2024-09-10T15:41:23.455 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T15:41:23.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T15:41:23.547 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899253 branch: wip-bharath7-testing-2024-09-10-1409 description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-low-osd-mem-target} rados supported-random-distro$/{centos_latest}} email: skanta@redhat.com first_in_suite: false job_id: '7899253' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath7-testing-2024-09-10-1409 ceph: conf: global: mon client directed command retry: 5 mon client hunt interval max multiple: 2 mon election default strategy: 3 mon mgr beacon grace: 90 ms bind msgr2: false ms inject socket failures: 1000 ms type: async mgr: debug mgr: 20 debug monc: 10 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true bluestore write v2: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - \(MON_DOWN\) mon_bind_msgr2: false sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath7-testing-2024-09-10-1409 sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 seed: 5245 sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath7-testing-2024-09-10-1409 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_bad1f558c327094897e4a32e99dd5cb4ae6ccd46/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 targets: smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0LLOr7kl6WLWImY5x6pbqLOhbca7rp3LWyJqOEgo6ahtG+msQKqJ5+e08DvifgRH1/AL7hxxIZb2fPg3HNEAU= tasks: - install: null - ceph: log-ignorelist: - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(SLOW_OPS\) - slow request - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - rep_lost_unfound_delete: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_15:30:27 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T15:41:23.547 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_bad1f558c327094897e4a32e99dd5cb4ae6ccd46/qa; will attempt to use it 2024-09-10T15:41:23.548 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_bad1f558c327094897e4a32e99dd5cb4ae6ccd46/qa/tasks 2024-09-10T15:41:23.548 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T15:41:23.550 INFO:teuthology.task.internal:Checking packages... 2024-09-10T15:41:23.573 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46' 2024-09-10T15:41:23.573 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T15:41:23.573 INFO:teuthology.packaging:ref: None 2024-09-10T15:41:23.573 INFO:teuthology.packaging:tag: None 2024-09-10T15:41:23.573 INFO:teuthology.packaging:branch: wip-bharath7-testing-2024-09-10-1409 2024-09-10T15:41:23.573 INFO:teuthology.packaging:sha1: bad1f558c327094897e4a32e99dd5cb4ae6ccd46 2024-09-10T15:41:23.573 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath7-testing-2024-09-10-1409 2024-09-10T15:41:23.719 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4858.gbad1f558 2024-09-10T15:41:23.721 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T15:41:23.728 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T15:41:23.728 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T15:41:23.773 INFO:teuthology.task.internal:Saving configuration 2024-09-10T15:41:23.786 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T15:41:23.799 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T15:41:23.825 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899253', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 15:36:37.862001', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0LLOr7kl6WLWImY5x6pbqLOhbca7rp3LWyJqOEgo6ahtG+msQKqJ5+e08DvifgRH1/AL7hxxIZb2fPg3HNEAU='} 2024-09-10T15:41:23.825 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T15:41:23.831 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2024-09-10T15:41:23.831 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T15:41:23.876 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcdad6cfbe0>, signals=[15]) 2024-09-10T15:41:23.876 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T15:41:23.884 INFO:teuthology.task.internal:Opening connections... 2024-09-10T15:41:23.884 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2024-09-10T15:41:23.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T15:41:23.965 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T15:41:23.972 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2024-09-10T15:41:24.051 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2024-09-10T15:41:24.051 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2024-09-10T15:41:24.109 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T15:41:24.110 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T15:41:24.111 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2024-09-10T15:41:24.129 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T15:41:24.137 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T15:41:24.148 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T15:41:24.148 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2024-09-10T15:41:24.168 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T15:41:24.173 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T15:41:24.174 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2024-09-10T15:41:24.304 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T15:41:24.335 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T15:41:24.349 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T15:41:24.349 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T15:41:24.349 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T15:41:24.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:41:24.367 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2024-09-10T15:41:24.427 INFO:teuthology.orchestra.run.smithi121.stdout:5.14.0-503.el9.x86_64 2024-09-10T15:41:24.427 INFO:teuthology.task.kernel:Running kernel on smithi121: 5.14.0-503.el9.x86_64 2024-09-10T15:41:24.427 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum install -y kernel 2024-09-10T15:41:27.032 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.2 MB 00:00 2024-09-10T15:41:30.399 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 20 MB 00:02 2024-09-10T15:41:34.808 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-10T15:41:37.125 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-10T15:41:37.912 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-10T15:41:43.276 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-10T15:41:45.232 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T15:41:45.233 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T15:41:45.317 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:41:45.328 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:41:45.328 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-09-10T15:41:45.328 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:41:45.328 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:41:45.328 INFO:teuthology.orchestra.run.smithi121.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T15:41:45.328 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-09-10T15:41:45.328 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T15:41:45.328 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T15:41:45.328 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T15:41:45.328 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:41:45.328 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:41:45.329 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:41:45.329 INFO:teuthology.orchestra.run.smithi121.stdout:Install 4 Packages 2024-09-10T15:41:45.329 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:41:45.329 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 79 M 2024-09-10T15:41:45.329 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 126 M 2024-09-10T15:41:45.329 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:41:45.511 INFO:teuthology.orchestra.run.smithi121.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 268 kB/s | 23 kB 00:00 2024-09-10T15:41:45.886 INFO:teuthology.orchestra.run.smithi121.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-10T15:41:46.295 INFO:teuthology.orchestra.run.smithi121.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 40 MB/s | 35 MB 00:00 2024-09-10T15:41:46.695 INFO:teuthology.orchestra.run.smithi121.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-10T15:41:46.695 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:41:46.695 INFO:teuthology.orchestra.run.smithi121.stdout:Total 58 MB/s | 79 MB 00:01 2024-09-10T15:41:47.382 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:41:47.740 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:41:47.740 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:41:48.676 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:41:48.676 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:41:51.140 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:41:51.778 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T15:41:51.862 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T15:41:54.681 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T15:41:54.824 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T15:41:59.643 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T15:41:59.741 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T15:42:04.585 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T15:42:42.724 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T15:42:42.763 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T15:42:43.102 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T15:42:43.102 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T15:42:43.102 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T15:42:43.102 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T15:42:43.962 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T15:42:43.962 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:42:43.962 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:42:43.962 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T15:42:43.963 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T15:42:43.963 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T15:42:43.963 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T15:42:43.963 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:42:43.963 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:42:44.330 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T15:42:44.429 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T15:42:44.430 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T15:42:44.430 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T15:42:44.430 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T15:42:44.430 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T15:42:44.430 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T15:42:44.430 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi121.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T15:42:44.431 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T15:42:44.489 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T15:42:44.490 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T15:42:44.490 DEBUG:teuthology.orchestra.run.smithi121:> sudo rpm -qi grub2-tools 2024-09-10T15:42:44.528 INFO:teuthology.orchestra.run.smithi121.stdout:Name : grub2-tools 2024-09-10T15:42:44.529 INFO:teuthology.orchestra.run.smithi121.stdout:Epoch : 1 2024-09-10T15:42:44.529 INFO:teuthology.orchestra.run.smithi121.stdout:Version : 2.06 2024-09-10T15:42:44.529 INFO:teuthology.orchestra.run.smithi121.stdout:Release : 82.el9 2024-09-10T15:42:44.529 INFO:teuthology.orchestra.run.smithi121.stdout:Architecture: x86_64 2024-09-10T15:42:44.529 INFO:teuthology.orchestra.run.smithi121.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T15:42:44.529 INFO:teuthology.orchestra.run.smithi121.stdout:Group : Unspecified 2024-09-10T15:42:44.529 INFO:teuthology.orchestra.run.smithi121.stdout:Size : 8274670 2024-09-10T15:42:44.529 INFO:teuthology.orchestra.run.smithi121.stdout:License : GPLv3+ 2024-09-10T15:42:44.529 INFO:teuthology.orchestra.run.smithi121.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T15:42:44.529 INFO:teuthology.orchestra.run.smithi121.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T15:42:44.529 INFO:teuthology.orchestra.run.smithi121.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T15:42:44.529 INFO:teuthology.orchestra.run.smithi121.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T15:42:44.530 INFO:teuthology.orchestra.run.smithi121.stdout:Packager : builder@centos.org 2024-09-10T15:42:44.530 INFO:teuthology.orchestra.run.smithi121.stdout:Vendor : CentOS 2024-09-10T15:42:44.530 INFO:teuthology.orchestra.run.smithi121.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T15:42:44.530 INFO:teuthology.orchestra.run.smithi121.stdout:Summary : Support tools for GRUB. 2024-09-10T15:42:44.530 INFO:teuthology.orchestra.run.smithi121.stdout:Description : 2024-09-10T15:42:44.530 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:42:44.530 INFO:teuthology.orchestra.run.smithi121.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T15:42:44.530 INFO:teuthology.orchestra.run.smithi121.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T15:42:44.530 INFO:teuthology.orchestra.run.smithi121.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T15:42:44.530 INFO:teuthology.orchestra.run.smithi121.stdout:hardware devices. 2024-09-10T15:42:44.530 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:42:44.530 INFO:teuthology.orchestra.run.smithi121.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T15:42:44.531 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T15:42:44.531 INFO:teuthology.task.kernel:Updating grub on smithi121 to boot 5.14.0-505.el9.x86_64 2024-09-10T15:42:44.531 DEBUG:teuthology.orchestra.run.smithi121:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T15:42:45.155 INFO:teuthology.orchestra.run.smithi121.stderr:Generating grub configuration file ... 2024-09-10T15:42:46.507 INFO:teuthology.orchestra.run.smithi121.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T15:42:46.539 INFO:teuthology.orchestra.run.smithi121.stderr:done 2024-09-10T15:42:46.546 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-09-10T15:42:46.561 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.XYq98AlSIm 2024-09-10T15:42:46.562 DEBUG:teuthology.orchestra.run.smithi121:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XYq98AlSIm 2024-09-10T15:42:46.629 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.XYq98AlSIm 2024-09-10T15:42:46.766 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.XYq98AlSIm is 6KB 2024-09-10T15:42:46.814 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.XYq98AlSIm 2024-09-10T15:42:46.829 DEBUG:teuthology.orchestra.run.smithi121:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T15:42:46.892 INFO:teuthology.orchestra.run.smithi121.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T15:42:46.892 INFO:teuthology.orchestra.run.smithi121.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T15:42:46.892 INFO:teuthology.orchestra.run.smithi121.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T15:42:46.892 INFO:teuthology.orchestra.run.smithi121.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T15:42:46.893 DEBUG:teuthology.orchestra.run.smithi121:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T15:42:47.055 DEBUG:teuthology.orchestra.run.smithi121:> sudo shutdown -r now 2024-09-10T15:43:17.083 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T15:43:17.084 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-09-10T15:43:17.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T15:43:35.485 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-09-10T15:43:44.493 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-09-10T15:43:44.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T15:44:44.554 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T15:44:56.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-09-10T15:44:56.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T15:44:56.933 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-09-10T15:44:57.288 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-09-10T15:44:57.288 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T15:44:57.289 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2024-09-10T15:44:57.304 INFO:teuthology.orchestra.run.smithi121.stdout:5.14.0-505.el9.x86_64 2024-09-10T15:44:57.305 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T15:44:57.305 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T15:44:57.305 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T15:44:58.306 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T15:44:58.307 DEBUG:teuthology.orchestra.run.smithi121:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T15:44:58.382 INFO:teuthology.orchestra.run.smithi121.stdout:ttyS1 2024-09-10T15:44:58.400 DEBUG:teuthology.parallel:result is None 2024-09-10T15:44:58.400 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T15:44:58.408 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T15:44:58.409 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T15:44:58.427 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T15:44:58.442 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T15:44:58.492 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T15:44:58.492 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T15:44:58.522 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T15:44:58.543 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T15:44:58.544 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T15:44:58.601 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T15:44:58.612 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T15:44:58.613 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T15:44:58.622 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T15:44:58.622 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T15:44:58.680 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T15:44:58.690 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T15:44:58.690 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T15:44:58.737 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T15:44:58.812 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T15:44:58.867 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T15:44:58.946 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T15:44:58.991 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:44:58.991 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T15:44:59.056 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2024-09-10T15:44:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T15:44:59.493 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T15:44:59.504 INFO:teuthology.task.internal:Starting timer... 2024-09-10T15:44:59.504 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T15:44:59.549 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T15:44:59.595 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2024-09-10T15:44:59.629 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2024-09-10T15:44:59.633 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T15:44:59.633 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2024-09-10T15:44:59.703 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2024-09-10T15:44:59.703 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi121.front.sepia.ceph.com': 'permissive'} 2024-09-10T15:44:59.703 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T15:44:59.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:44:59.769 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T15:44:59.769 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2024-09-10T15:44:59.845 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T15:44:59.855 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T15:44:59.858 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T15:44:59.964 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T15:44:59.974 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T15:44:59.975 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi121.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T15:48:39.529 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi121.front.sepia.ceph.com')] 2024-09-10T15:48:39.531 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-09-10T15:48:39.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T15:48:39.664 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-09-10T15:48:39.744 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-09-10T15:48:39.744 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T15:48:39.756 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T15:48:39.756 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T15:48:39.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T15:48:39.814 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T15:48:39.828 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T15:48:39.852 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2024-09-10T15:48:39.863 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2024-09-10T15:48:39.878 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T15:48:39.892 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T15:48:39.953 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-09-10T15:48:39.954 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T15:48:39.954 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-09-10T15:48:39.954 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T15:48:39.954 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T15:48:39.954 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T15:48:39.954 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T15:48:39.954 INFO:teuthology.run_tasks:Running task install... 2024-09-10T15:48:39.962 DEBUG:teuthology.task.install:project ceph 2024-09-10T15:48:39.962 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'}} 2024-09-10T15:48:39.963 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'} 2024-09-10T15:48:39.963 INFO:teuthology.task.install:Using flavor: default 2024-09-10T15:48:39.969 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T15:48:39.969 INFO:teuthology.task.install:extra packages: [] 2024-09-10T15:48:39.970 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46', 'tag': None, 'wait_for_package': False} 2024-09-10T15:48:39.970 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bad1f558c327094897e4a32e99dd5cb4ae6ccd46 2024-09-10T15:48:40.115 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-bharath7-testing-2024-09-10-1409/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/centos/9/flavors/default/ 2024-09-10T15:48:40.116 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4858.gbad1f558 2024-09-10T15:48:40.234 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-bharath7-testing-2024-09-10-1409/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-bharath7-testing-2024-09-10-1409/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-bharath7-testing-2024-09-10-1409/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T15:48:40.235 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:48:40.235 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T15:48:40.262 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T15:48:40.262 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bad1f558c327094897e4a32e99dd5cb4ae6ccd46/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T15:48:40.329 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T15:48:40.404 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T15:48:40.469 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2024-09-10T15:48:40.470 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2024-09-10T15:48:40.706 INFO:teuthology.orchestra.run.smithi121.stdout:45 files removed 2024-09-10T15:48:40.732 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw 2024-09-10T15:48:41.275 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 324 kB/s | 84 kB 00:00 2024-09-10T15:48:41.559 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-10T15:48:41.827 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-10T15:48:43.601 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-10T15:48:46.385 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-10T15:48:50.728 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-10T15:48:52.939 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:01 2024-09-10T15:48:53.710 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T15:48:59.057 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T15:49:01.407 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:49:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:01.424 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-10T15:49:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:49:01.424 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 11 M 2024-09-10T15:49:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2024-09-10T15:49:01.424 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 3.5 M 2024-09-10T15:49:01.424 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 3.2 M 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 5.5 M 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 22 M 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 25 k 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 755 k 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T15:49:01.425 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 509 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 5.5 M 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 45 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 143 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 161 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 321 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 306 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 99 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T15:49:01.426 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T15:49:01.427 INFO:teuthology.orchestra.run.smithi121.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T15:49:01.427 INFO:teuthology.orchestra.run.smithi121.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T15:49:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:Installing weak dependencies: 2024-09-10T15:49:01.427 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T15:49:01.427 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:49:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:Install 31 Packages 2024-09-10T15:49:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 2 Packages 2024-09-10T15:49:01.427 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:01.428 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 62 M 2024-09-10T15:49:01.428 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:49:02.324 INFO:teuthology.orchestra.run.smithi121.stdout:(1/33): ceph-base-19.3.0-4858.gbad1f558.el9.x86 7.7 MB/s | 5.5 MB 00:00 2024-09-10T15:49:02.391 INFO:teuthology.orchestra.run.smithi121.stdout:(2/33): ceph-selinux-19.3.0-4858.gbad1f558.el9. 378 kB/s | 25 kB 00:00 2024-09-10T15:49:02.508 INFO:teuthology.orchestra.run.smithi121.stdout:(3/33): ceph-radosgw-19.3.0-4858.gbad1f558.el9. 12 MB/s | 11 MB 00:00 2024-09-10T15:49:02.542 INFO:teuthology.orchestra.run.smithi121.stdout:(4/33): libcephfs2-19.3.0-4858.gbad1f558.el9.x8 4.9 MB/s | 755 kB 00:00 2024-09-10T15:49:02.567 INFO:teuthology.orchestra.run.smithi121.stdout:(5/33): libradosstriper1-19.3.0-4858.gbad1f558. 8.5 MB/s | 509 kB 00:00 2024-09-10T15:49:02.776 INFO:teuthology.orchestra.run.smithi121.stdout:(6/33): ceph-common-19.3.0-4858.gbad1f558.el9.x 19 MB/s | 22 MB 00:01 2024-09-10T15:49:02.801 INFO:teuthology.orchestra.run.smithi121.stdout:(7/33): python3-ceph-argparse-19.3.0-4858.gbad1 193 kB/s | 45 kB 00:00 2024-09-10T15:49:02.826 INFO:teuthology.orchestra.run.smithi121.stdout:(8/33): python3-ceph-common-19.3.0-4858.gbad1f5 2.8 MB/s | 143 kB 00:00 2024-09-10T15:49:02.851 INFO:teuthology.orchestra.run.smithi121.stdout:(9/33): python3-cephfs-19.3.0-4858.gbad1f558.el 3.1 MB/s | 161 kB 00:00 2024-09-10T15:49:02.877 INFO:teuthology.orchestra.run.smithi121.stdout:(10/33): python3-rados-19.3.0-4858.gbad1f558.el 6.2 MB/s | 321 kB 00:00 2024-09-10T15:49:02.910 INFO:teuthology.orchestra.run.smithi121.stdout:(11/33): python3-rbd-19.3.0-4858.gbad1f558.el9. 5.1 MB/s | 306 kB 00:00 2024-09-10T15:49:02.936 INFO:teuthology.orchestra.run.smithi121.stdout:(12/33): python3-rgw-19.3.0-4858.gbad1f558.el9. 1.7 MB/s | 99 kB 00:00 2024-09-10T15:49:03.020 INFO:teuthology.orchestra.run.smithi121.stdout:(13/33): librgw2-19.3.0-4858.gbad1f558.el9.x86_ 12 MB/s | 5.5 MB 00:00 2024-09-10T15:49:03.112 INFO:teuthology.orchestra.run.smithi121.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 533 kB/s | 107 kB 00:00 2024-09-10T15:49:03.137 INFO:teuthology.orchestra.run.smithi121.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 850 kB/s | 171 kB 00:00 2024-09-10T15:49:03.162 INFO:teuthology.orchestra.run.smithi121.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-09-10T15:49:03.179 INFO:teuthology.orchestra.run.smithi121.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-10T15:49:03.221 INFO:teuthology.orchestra.run.smithi121.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-10T15:49:03.263 INFO:teuthology.orchestra.run.smithi121.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-10T15:49:03.288 INFO:teuthology.orchestra.run.smithi121.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-10T15:49:03.313 INFO:teuthology.orchestra.run.smithi121.stdout:(21/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-09-10T15:49:03.339 INFO:teuthology.orchestra.run.smithi121.stdout:(22/33): qatlib-service-24.02.0-1.el9.x86_64.rp 476 kB/s | 36 kB 00:00 2024-09-10T15:49:03.364 INFO:teuthology.orchestra.run.smithi121.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-10T15:49:03.422 INFO:teuthology.orchestra.run.smithi121.stdout:(24/33): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.8 MB/s | 309 kB 00:00 2024-09-10T15:49:03.448 INFO:teuthology.orchestra.run.smithi121.stdout:(25/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-09-10T15:49:03.473 INFO:teuthology.orchestra.run.smithi121.stdout:(26/33): liboath-2.6.7-2.el9.x86_64.rpm 966 kB/s | 49 kB 00:00 2024-09-10T15:49:03.498 INFO:teuthology.orchestra.run.smithi121.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T15:49:03.532 INFO:teuthology.orchestra.run.smithi121.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-10T15:49:03.565 INFO:teuthology.orchestra.run.smithi121.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-10T15:49:03.616 INFO:teuthology.orchestra.run.smithi121.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-10T15:49:03.716 INFO:teuthology.orchestra.run.smithi121.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-10T15:49:03.800 INFO:teuthology.orchestra.run.smithi121.stdout:(32/33): librados2-19.3.0-4858.gbad1f558.el9.x8 15 MB/s | 3.5 MB 00:00 2024-09-10T15:49:03.866 INFO:teuthology.orchestra.run.smithi121.stdout:(33/33): librbd1-19.3.0-4858.gbad1f558.el9.x86_ 13 MB/s | 3.2 MB 00:00 2024-09-10T15:49:03.870 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:49:03.871 INFO:teuthology.orchestra.run.smithi121.stdout:Total 25 MB/s | 62 MB 00:02 2024-09-10T15:49:03.980 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:49:04.015 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:49:04.016 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:49:04.707 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:49:04.707 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:49:06.238 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:49:06.326 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-10T15:49:06.356 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-10T15:49:06.380 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-10T15:49:06.403 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-10T15:49:06.428 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-10T15:49:06.622 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-10T15:49:06.674 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-10T15:49:07.057 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-10T15:49:07.123 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 7/35 2024-09-10T15:49:07.199 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 7/35 2024-09-10T15:49:07.289 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:19.3.0-4858.gbad1f558.el9.x86_64 8/35 2024-09-10T15:49:07.316 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 9/35 2024-09-10T15:49:07.380 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 9/35 2024-09-10T15:49:07.422 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-10T15:49:07.468 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:19.3.0-4858.gbad1f558.el9. 11/35 2024-09-10T15:49:07.531 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:19.3.0-4858.gbad1f558.el9.x86_64 12/35 2024-09-10T15:49:07.569 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:19.3.0-4858.gbad1f558.el9.x86_6 13/35 2024-09-10T15:49:07.617 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4858.gbad1f558.el9.x86_6 13/35 2024-09-10T15:49:07.651 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-10T15:49:07.761 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-10T15:49:07.793 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-10T15:49:07.835 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-10T15:49:07.875 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/35 2024-09-10T15:49:07.904 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-10T15:49:07.937 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-10T15:49:07.983 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-10T15:49:08.306 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-10T15:49:08.359 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 23/35 2024-09-10T15:49:08.449 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 23/35 2024-09-10T15:49:08.480 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:19.3.0-4858.gbad1f558.el9.x86_64 24/35 2024-09-10T15:49:08.516 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-10T15:49:09.012 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-10T15:49:09.037 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:19.3.0-4858.gbad1f558.el9.x86_64 27/35 2024-09-10T15:49:09.108 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:19.3.0-4858.gbad1f558.el9.x86_64 27/35 2024-09-10T15:49:09.186 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:19.3.0-4858.gbad1f558.el9.x86_64 28/35 2024-09-10T15:49:09.231 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:19.3.0-4858.gbad1f558.el9.x8 29/35 2024-09-10T15:49:11.576 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 30/35 2024-09-10T15:49:11.610 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 30/35 2024-09-10T15:49:12.179 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 30/35 2024-09-10T15:49:12.213 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 31/35 2024-09-10T15:49:12.330 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 31/35 2024-09-10T15:49:12.330 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T15:49:12.330 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T15:49:12.330 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:12.365 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:19.3.0-4858.gbad1f558.el9.x86_64 32/35 2024-09-10T15:49:22.324 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4858.gbad1f558.el9.x86_64 32/35 2024-09-10T15:49:22.324 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2024-09-10T15:49:22.325 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2024-09-10T15:49:22.325 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2024-09-10T15:49:22.325 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2024-09-10T15:49:22.325 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2024-09-10T15:49:22.325 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /root 2024-09-10T15:49:22.325 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2024-09-10T15:49:22.325 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:23.342 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:19.3.0-4858.gbad1f558.el9.x86_64 33/35 2024-09-10T15:49:23.368 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4858.gbad1f558.el9.x86_64 33/35 2024-09-10T15:49:23.368 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:49:23.368 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T15:49:23.368 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T15:49:23.368 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T15:49:23.368 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:23.370 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-10T15:49:23.485 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-10T15:49:23.485 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-10T15:49:24.722 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-10T15:49:24.722 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 1/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 2/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:19.3.0-4858.gbad1f558.el9.x86_64 3/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:19.3.0-4858.gbad1f558.el9.x86_64 4/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 5/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:19.3.0-4858.gbad1f558.el9.x86_6 6/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:19.3.0-4858.gbad1f558.el9.x86_64 7/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4858.gbad1f558.el9. 8/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:19.3.0-4858.gbad1f558.el9.x8 9/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:19.3.0-4858.gbad1f558.el9.x86_64 10/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:19.3.0-4858.gbad1f558.el9.x86_64 11/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:19.3.0-4858.gbad1f558.el9.x86_64 12/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:19.3.0-4858.gbad1f558.el9.x86_64 13/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-10T15:49:24.723 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 24/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-10T15:49:24.724 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-10T15:49:24.725 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-10T15:49:24.726 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-10T15:49:24.726 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-10T15:49:24.726 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 32/35 2024-09-10T15:49:24.726 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-10T15:49:24.726 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 34/35 2024-09-10T15:49:25.660 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-10T15:49:25.660 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:25.660 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2024-09-10T15:49:25.660 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.660 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.660 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:49:25.660 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T15:49:25.661 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:25.662 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T15:49:25.663 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T15:49:25.663 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T15:49:25.663 INFO:teuthology.orchestra.run.smithi121.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T15:49:25.663 INFO:teuthology.orchestra.run.smithi121.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T15:49:25.663 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:25.663 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:49:26.020 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-test 2024-09-10T15:49:26.589 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:27 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:49:26.721 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:49:26.722 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-10T15:49:26.722 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:26.722 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:49:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 50 M 2024-09-10T15:49:26.722 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-09-10T15:49:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 161 k 2024-09-10T15:49:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T15:49:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T15:49:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:26.722 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:49:26.722 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:26.723 INFO:teuthology.orchestra.run.smithi121.stdout:Install 4 Packages 2024-09-10T15:49:26.723 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:26.723 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 50 M 2024-09-10T15:49:26.723 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 215 M 2024-09-10T15:49:26.723 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:49:27.192 INFO:teuthology.orchestra.run.smithi121.stdout:(1/4): libcephsqlite-19.3.0-4858.gbad1f558.el9. 734 kB/s | 161 kB 00:00 2024-09-10T15:49:27.217 INFO:teuthology.orchestra.run.smithi121.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-10T15:49:27.342 INFO:teuthology.orchestra.run.smithi121.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 424 kB/s | 64 kB 00:00 2024-09-10T15:49:28.526 INFO:teuthology.orchestra.run.smithi121.stdout:(4/4): ceph-test-19.3.0-4858.gbad1f558.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-10T15:49:28.528 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:49:28.528 INFO:teuthology.orchestra.run.smithi121.stdout:Total 28 MB/s | 50 MB 00:01 2024-09-10T15:49:28.556 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:49:28.576 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:49:28.576 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:49:28.953 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:49:28.954 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:49:29.496 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:49:29.578 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T15:49:29.626 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T15:49:29.653 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:19.3.0-4858.gbad1f558.el9.x86_64 3/4 2024-09-10T15:49:34.694 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4858.gbad1f558.el9.x86_64 3/4 2024-09-10T15:49:34.785 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:19.3.0-4858.gbad1f558.el9.x86_64 4/4 2024-09-10T15:49:37.216 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-test-2:19.3.0-4858.gbad1f558.el9.x86_64 4/4 2024-09-10T15:49:37.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:19.3.0-4858.gbad1f558.el9.x86_64 1/4 2024-09-10T15:49:37.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:19.3.0-4858.gbad1f558.el9.x86_64 2/4 2024-09-10T15:49:37.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T15:49:37.522 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T15:49:37.522 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:37.523 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:49:37.523 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:37.523 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:37.523 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T15:49:37.523 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T15:49:37.523 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:37.523 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:49:37.786 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph 2024-09-10T15:49:38.357 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:39 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:49:38.497 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:49:38.499 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:38.499 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-10T15:49:38.500 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:38.500 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:49:38.500 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 6.6 k 2024-09-10T15:49:38.500 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-09-10T15:49:38.500 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 2.4 M 2024-09-10T15:49:38.500 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 1.5 M 2024-09-10T15:49:38.500 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 291 k 2024-09-10T15:49:38.500 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 4.8 M 2024-09-10T15:49:38.500 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 17 M 2024-09-10T15:49:38.500 INFO:teuthology.orchestra.run.smithi121.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T15:49:38.500 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T15:49:38.500 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T15:49:38.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T15:49:38.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T15:49:38.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T15:49:38.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T15:49:38.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T15:49:38.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T15:49:38.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T15:49:38.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections 2024-09-10T15:49:38.501 INFO:teuthology.orchestra.run.smithi121.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T15:49:38.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T15:49:38.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T15:49:38.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T15:49:38.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T15:49:38.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T15:49:38.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T15:49:38.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T15:49:38.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T15:49:38.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T15:49:38.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T15:49:38.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T15:49:38.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T15:49:38.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T15:49:38.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T15:49:38.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T15:49:38.503 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T15:49:38.503 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T15:49:38.503 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T15:49:38.503 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T15:49:38.503 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T15:49:38.503 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:38.503 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:49:38.503 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:38.503 INFO:teuthology.orchestra.run.smithi121.stdout:Install 37 Packages 2024-09-10T15:49:38.503 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:38.504 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 31 M 2024-09-10T15:49:38.504 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 109 M 2024-09-10T15:49:38.504 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:49:38.843 INFO:teuthology.orchestra.run.smithi121.stdout:(1/37): ceph-19.3.0-4858.gbad1f558.el9.x86_64.r 58 kB/s | 6.6 kB 00:00 2024-09-10T15:49:39.035 INFO:teuthology.orchestra.run.smithi121.stdout:(2/37): ceph-mgr-19.3.0-4858.gbad1f558.el9.x86_ 5.0 MB/s | 1.5 MB 00:00 2024-09-10T15:49:39.085 INFO:teuthology.orchestra.run.smithi121.stdout:(3/37): ceph-mds-19.3.0-4858.gbad1f558.el9.x86_ 6.7 MB/s | 2.4 MB 00:00 2024-09-10T15:49:39.169 INFO:teuthology.orchestra.run.smithi121.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4858.gbad1 3.4 MB/s | 291 kB 00:00 2024-09-10T15:49:39.294 INFO:teuthology.orchestra.run.smithi121.stdout:(5/37): ceph-mon-19.3.0-4858.gbad1f558.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-10T15:49:39.328 INFO:teuthology.orchestra.run.smithi121.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-10T15:49:39.370 INFO:teuthology.orchestra.run.smithi121.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-10T15:49:39.428 INFO:teuthology.orchestra.run.smithi121.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.3 MB/s | 135 kB 00:00 2024-09-10T15:49:39.475 INFO:teuthology.orchestra.run.smithi121.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.6 MB/s | 125 kB 00:00 2024-09-10T15:49:39.521 INFO:teuthology.orchestra.run.smithi121.stdout:(10/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.6 MB/s | 215 kB 00:00 2024-09-10T15:49:39.679 INFO:teuthology.orchestra.run.smithi121.stdout:(11/37): ceph-osd-19.3.0-4858.gbad1f558.el9.x86 27 MB/s | 17 MB 00:00 2024-09-10T15:49:39.713 INFO:teuthology.orchestra.run.smithi121.stdout:(12/37): python3-cryptography-36.0.1-4.el9.x86_ 2.9 MB/s | 1.2 MB 00:00 2024-09-10T15:49:39.830 INFO:teuthology.orchestra.run.smithi121.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 298 kB/s | 35 kB 00:00 2024-09-10T15:49:39.855 INFO:teuthology.orchestra.run.smithi121.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 982 kB/s | 172 kB 00:00 2024-09-10T15:49:39.897 INFO:teuthology.orchestra.run.smithi121.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 501 kB/s | 188 kB 00:00 2024-09-10T15:49:39.922 INFO:teuthology.orchestra.run.smithi121.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 837 kB/s | 77 kB 00:00 2024-09-10T15:49:39.947 INFO:teuthology.orchestra.run.smithi121.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 553 kB/s | 51 kB 00:00 2024-09-10T15:49:39.990 INFO:teuthology.orchestra.run.smithi121.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 451 kB/s | 42 kB 00:00 2024-09-10T15:49:40.023 INFO:teuthology.orchestra.run.smithi121.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-10T15:49:40.048 INFO:teuthology.orchestra.run.smithi121.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 177 kB/s | 22 kB 00:00 2024-09-10T15:49:40.074 INFO:teuthology.orchestra.run.smithi121.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 519 kB/s | 43 kB 00:00 2024-09-10T15:49:40.107 INFO:teuthology.orchestra.run.smithi121.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-10T15:49:40.133 INFO:teuthology.orchestra.run.smithi121.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 208 kB/s | 11 kB 00:00 2024-09-10T15:49:40.158 INFO:teuthology.orchestra.run.smithi121.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 362 kB/s | 18 kB 00:00 2024-09-10T15:49:40.183 INFO:teuthology.orchestra.run.smithi121.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 463 kB/s | 23 kB 00:00 2024-09-10T15:49:40.209 INFO:teuthology.orchestra.run.smithi121.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-10T15:49:40.234 INFO:teuthology.orchestra.run.smithi121.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-10T15:49:40.259 INFO:teuthology.orchestra.run.smithi121.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-10T15:49:40.276 INFO:teuthology.orchestra.run.smithi121.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-10T15:49:40.301 INFO:teuthology.orchestra.run.smithi121.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 661 kB/s | 16 kB 00:00 2024-09-10T15:49:40.329 INFO:teuthology.orchestra.run.smithi121.stdout:(31/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-10T15:49:40.352 INFO:teuthology.orchestra.run.smithi121.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-10T15:49:40.377 INFO:teuthology.orchestra.run.smithi121.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-10T15:49:40.403 INFO:teuthology.orchestra.run.smithi121.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-10T15:49:40.428 INFO:teuthology.orchestra.run.smithi121.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 801 kB/s | 20 kB 00:00 2024-09-10T15:49:40.453 INFO:teuthology.orchestra.run.smithi121.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T15:49:40.478 INFO:teuthology.orchestra.run.smithi121.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-10T15:49:40.483 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:49:40.483 INFO:teuthology.orchestra.run.smithi121.stdout:Total 16 MB/s | 31 MB 00:01 2024-09-10T15:49:40.637 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:49:40.677 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:49:40.677 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:49:41.159 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:49:41.160 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:49:42.522 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:49:42.589 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-10T15:49:42.631 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-10T15:49:42.658 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-10T15:49:42.730 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-10T15:49:42.774 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-10T15:49:42.807 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-10T15:49:42.899 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-10T15:49:42.986 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-10T15:49:43.035 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-10T15:49:43.070 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-10T15:49:43.099 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-10T15:49:43.134 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-10T15:49:43.171 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-10T15:49:43.220 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-10T15:49:43.264 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-10T15:49:43.298 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-10T15:49:43.346 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-10T15:49:43.378 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-10T15:49:43.548 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-10T15:49:43.609 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-10T15:49:43.694 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-10T15:49:43.748 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-10T15:49:43.786 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-10T15:49:43.835 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-10T15:49:43.890 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-10T15:49:44.084 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-10T15:49:44.140 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-10T15:49:44.219 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-10T15:49:44.306 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-10T15:49:44.344 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-10T15:49:44.475 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-10T15:49:44.637 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4858.gbad1f558.el9. 32/37 2024-09-10T15:49:44.667 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 33/37 2024-09-10T15:49:44.693 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 33/37 2024-09-10T15:49:44.694 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:49:44.694 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T15:49:44.694 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T15:49:44.694 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T15:49:44.694 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:46.151 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:19.3.0-4858.gbad1f558.el9.x86_64 34/37 2024-09-10T15:49:46.178 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:19.3.0-4858.gbad1f558.el9.x86_64 34/37 2024-09-10T15:49:46.178 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:49:46.178 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T15:49:46.178 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T15:49:46.178 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T15:49:46.179 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:19.3.0-4858.gbad1f558.el9.x86_64 35/37 2024-09-10T15:49:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:19.3.0-4858.gbad1f558.el9.x86_64 35/37 2024-09-10T15:49:46.653 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:49:46.653 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T15:49:46.653 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T15:49:46.653 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T15:49:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:46.880 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:19.3.0-4858.gbad1f558.el9.x86_64 36/37 2024-09-10T15:49:46.903 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:19.3.0-4858.gbad1f558.el9.x86_64 36/37 2024-09-10T15:49:46.903 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:49:46.904 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T15:49:46.904 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T15:49:46.904 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T15:49:46.904 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:46.969 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:19.3.0-4858.gbad1f558.el9.x86_64 37/37 2024-09-10T15:49:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-2:19.3.0-4858.gbad1f558.el9.x86_64 37/37 2024-09-10T15:49:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:19.3.0-4858.gbad1f558.el9.x86_64 1/37 2024-09-10T15:49:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:19.3.0-4858.gbad1f558.el9.x86_64 2/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 3/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:19.3.0-4858.gbad1f558.el9.x86_64 4/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:19.3.0-4858.gbad1f558.el9.x86_64 5/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4858.gbad1f558.el9. 6/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-10T15:49:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-10T15:49:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-10T15:49:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-10T15:49:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-10T15:49:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-10T15:49:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-10T15:49:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-10T15:49:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-10T15:49:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-10T15:49:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-10T15:49:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-10T15:49:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-10T15:49:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-10T15:49:48.737 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-10T15:49:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-10T15:49:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-10T15:49:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-10T15:49:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-10T15:49:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-10T15:49:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-10T15:49:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-10T15:49:49.725 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-10T15:49:49.725 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:49.725 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:49:49.725 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:49.725 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:49.725 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T15:49:49.726 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T15:49:49.727 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T15:49:49.728 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T15:49:49.728 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T15:49:49.728 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T15:49:49.728 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T15:49:49.728 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T15:49:49.728 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T15:49:49.728 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T15:49:49.728 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:49.728 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:49:50.094 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-base 2024-09-10T15:49:50.663 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:51 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:49:50.753 INFO:teuthology.orchestra.run.smithi121.stdout:Package ceph-base-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:49:50.796 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:49:50.797 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-09-10T15:49:50.797 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:49:50.863 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install cephadm 2024-09-10T15:49:51.439 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:52 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:49:51.571 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:49:51.572 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:51.572 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-10T15:49:51.572 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:51.572 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:49:51.572 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 784 k 2024-09-10T15:49:51.572 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:51.572 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:49:51.572 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:51.572 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-09-10T15:49:51.572 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:51.573 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 784 k 2024-09-10T15:49:51.573 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 790 k 2024-09-10T15:49:51.573 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:49:51.830 INFO:teuthology.orchestra.run.smithi121.stdout:cephadm-19.3.0-4858.gbad1f558.el9.noarch.rpm 3.0 MB/s | 784 kB 00:00 2024-09-10T15:49:51.831 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:49:51.831 INFO:teuthology.orchestra.run.smithi121.stdout:Total 3.0 MB/s | 784 kB 00:00 2024-09-10T15:49:51.831 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:49:51.836 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:49:51.837 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:49:51.842 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:49:51.843 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:49:51.919 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:49:52.368 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:49:52.459 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:49:52.828 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:49:53.082 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:49:53.082 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:53.082 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:49:53.082 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:49:53.083 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:53.083 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:49:53.270 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T15:49:53.846 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:54 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:49:53.984 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:49:53.984 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:53.985 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2024-09-10T15:49:53.985 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:53.985 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:49:53.985 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 145 k 2024-09-10T15:49:53.985 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:53.985 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:49:53.985 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:53.985 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-09-10T15:49:53.985 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:53.985 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 145 k 2024-09-10T15:49:53.985 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 432 k 2024-09-10T15:49:53.985 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:49:54.189 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-immutable-object-cache-19.3.0-4858.gbad1f5 715 kB/s | 145 kB 00:00 2024-09-10T15:49:54.189 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:49:54.189 INFO:teuthology.orchestra.run.smithi121.stdout:Total 710 kB/s | 145 kB 00:00 2024-09-10T15:49:54.189 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:49:54.197 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:49:54.197 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:49:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:49:54.255 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:49:54.400 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:49:54.465 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4858.gbad1f558. 1/1 2024-09-10T15:49:54.490 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4858.gbad1f558. 1/1 2024-09-10T15:49:54.490 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:49:54.490 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T15:49:54.490 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:55.424 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4858.gbad1f558. 1/1 2024-09-10T15:49:55.425 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:55.425 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:49:55.425 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:49:55.425 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:55.425 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:49:55.651 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr 2024-09-10T15:49:56.227 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:57 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:49:56.332 INFO:teuthology.orchestra.run.smithi121.stdout:Package ceph-mgr-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:49:56.375 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:49:56.376 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-09-10T15:49:56.376 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:49:56.435 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T15:49:56.992 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:57 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:49:57.123 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:49:57.124 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:57.124 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-10T15:49:57.124 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:57.125 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:49:57.125 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 14 M 2024-09-10T15:49:57.125 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-09-10T15:49:57.125 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T15:49:57.125 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 34 k 2024-09-10T15:49:57.126 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 16 k 2024-09-10T15:49:57.126 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T15:49:57.126 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T15:49:57.126 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T15:49:57.126 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T15:49:57.126 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T15:49:57.126 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T15:49:57.127 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T15:49:57.127 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T15:49:57.127 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:57.127 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:49:57.127 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:49:57.127 INFO:teuthology.orchestra.run.smithi121.stdout:Install 12 Packages 2024-09-10T15:49:57.127 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:49:57.128 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 19 M 2024-09-10T15:49:57.128 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 159 M 2024-09-10T15:49:57.128 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:49:57.534 INFO:teuthology.orchestra.run.smithi121.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4858.gba 235 kB/s | 34 kB 00:00 2024-09-10T15:49:57.559 INFO:teuthology.orchestra.run.smithi121.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4858.gbad 96 kB/s | 16 kB 00:00 2024-09-10T15:49:57.768 INFO:teuthology.orchestra.run.smithi121.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 269 kB 00:00 2024-09-10T15:49:57.827 INFO:teuthology.orchestra.run.smithi121.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.4 MB/s | 1.0 MB 00:00 2024-09-10T15:49:57.877 INFO:teuthology.orchestra.run.smithi121.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-10T15:49:57.911 INFO:teuthology.orchestra.run.smithi121.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 586 kB/s | 19 kB 00:00 2024-09-10T15:49:58.078 INFO:teuthology.orchestra.run.smithi121.stdout:(7/12): ceph-mgr-dashboard-19.3.0-4858.gbad1f55 21 MB/s | 14 MB 00:00 2024-09-10T15:49:58.120 INFO:teuthology.orchestra.run.smithi121.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.4 MB/s | 863 kB 00:00 2024-09-10T15:49:58.153 INFO:teuthology.orchestra.run.smithi121.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.9 MB/s | 144 kB 00:00 2024-09-10T15:49:58.178 INFO:teuthology.orchestra.run.smithi121.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 542 kB/s | 31 kB 00:00 2024-09-10T15:49:58.235 INFO:teuthology.orchestra.run.smithi121.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.3 MB/s | 188 kB 00:00 2024-09-10T15:49:58.279 INFO:teuthology.orchestra.run.smithi121.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.5 MB/s | 2.0 MB 00:00 2024-09-10T15:49:58.283 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:49:58.283 INFO:teuthology.orchestra.run.smithi121.stdout:Total 17 MB/s | 19 MB 00:01 2024-09-10T15:49:58.330 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:49:58.349 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:49:58.349 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:49:58.629 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:49:58.629 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:49:59.218 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:49:59.304 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-10T15:49:59.340 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-10T15:49:59.386 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-10T15:49:59.417 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-10T15:49:59.521 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-10T15:49:59.652 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-10T15:49:59.745 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-10T15:49:59.797 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-10T15:49:59.825 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-10T15:49:59.858 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4858.gbad1f558.el9 10/12 2024-09-10T15:50:03.023 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4858.gbad1f558.el 11/12 2024-09-10T15:50:03.079 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4858.gbad1f558.el9.noa 12/12 2024-09-10T15:50:04.984 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4858.gbad1f558.el9.noa 12/12 2024-09-10T15:50:04.984 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4858.gbad1f558.el 1/12 2024-09-10T15:50:04.984 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4858.gbad1f558.el9.noa 2/12 2024-09-10T15:50:04.984 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4858.gbad1f558.el9 3/12 2024-09-10T15:50:04.984 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-10T15:50:04.985 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-10T15:50:04.985 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-10T15:50:04.985 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-10T15:50:04.985 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-10T15:50:04.985 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-10T15:50:04.985 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-10T15:50:04.985 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-10T15:50:05.297 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-10T15:50:05.297 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:05.297 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:50:05.297 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T15:50:05.297 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:50:05.297 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:50:05.298 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:50:05.298 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T15:50:05.298 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T15:50:05.298 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T15:50:05.298 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T15:50:05.298 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T15:50:05.298 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T15:50:05.298 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T15:50:05.298 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T15:50:05.298 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:05.298 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:05.522 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T15:50:06.081 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:07 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:06.214 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:06.215 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-09-10T15:50:06.215 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-10T15:50:06.215 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-09-10T15:50:06.215 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:50:06.215 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 7.4 M 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T15:50:06.216 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:06.217 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:50:06.217 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-09-10T15:50:06.217 INFO:teuthology.orchestra.run.smithi121.stdout:Install 12 Packages 2024-09-10T15:50:06.217 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:06.217 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 38 M 2024-09-10T15:50:06.217 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 212 M 2024-09-10T15:50:06.217 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:50:06.827 INFO:teuthology.orchestra.run.smithi121.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 369 kB/s | 191 kB 00:00 2024-09-10T15:50:06.936 INFO:teuthology.orchestra.run.smithi121.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-09-10T15:50:07.136 INFO:teuthology.orchestra.run.smithi121.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 968 kB/s | 801 kB 00:00 2024-09-10T15:50:07.219 INFO:teuthology.orchestra.run.smithi121.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 219 kB/s | 18 kB 00:00 2024-09-10T15:50:07.295 INFO:teuthology.orchestra.run.smithi121.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 513 kB/s | 38 kB 00:00 2024-09-10T15:50:07.687 INFO:teuthology.orchestra.run.smithi121.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.0 MB/s | 3.0 MB 00:00 2024-09-10T15:50:07.787 INFO:teuthology.orchestra.run.smithi121.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-10T15:50:07.921 INFO:teuthology.orchestra.run.smithi121.stdout:(8/12): ceph-mgr-diskprediction-local-19.3.0-48 4.6 MB/s | 7.4 MB 00:01 2024-09-10T15:50:08.088 INFO:teuthology.orchestra.run.smithi121.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.5 MB/s | 250 kB 00:00 2024-09-10T15:50:08.696 INFO:teuthology.orchestra.run.smithi121.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.5 MB/s | 4.9 MB 00:01 2024-09-10T15:50:08.829 INFO:teuthology.orchestra.run.smithi121.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.9 MB/s | 5.1 MB 00:01 2024-09-10T15:50:11.221 INFO:teuthology.orchestra.run.smithi121.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.2 MB/s | 16 MB 00:03 2024-09-10T15:50:11.222 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:50:11.222 INFO:teuthology.orchestra.run.smithi121.stdout:Total 7.6 MB/s | 38 MB 00:05 2024-09-10T15:50:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:50:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:50:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:50:12.029 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:50:12.029 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:50:12.802 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:50:12.885 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T15:50:12.927 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T15:50:13.026 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T15:50:13.062 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T15:50:13.248 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T15:50:13.277 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T15:50:13.355 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T15:50:14.026 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T15:50:14.141 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T15:50:15.379 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T15:50:16.402 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T15:50:16.436 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4858.gbad1f 12/12 2024-09-10T15:50:18.945 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4858.gbad1f 12/12 2024-09-10T15:50:18.945 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4858.gbad1f 1/12 2024-09-10T15:50:18.945 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T15:50:18.945 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T15:50:18.945 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T15:50:18.945 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T15:50:18.946 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T15:50:18.946 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T15:50:18.946 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T15:50:18.946 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T15:50:18.946 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T15:50:18.946 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T15:50:19.397 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T15:50:19.397 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:19.397 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:50:19.397 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:50:19.397 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T15:50:19.397 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T15:50:19.398 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T15:50:19.398 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T15:50:19.398 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T15:50:19.398 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T15:50:19.398 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T15:50:19.398 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T15:50:19.398 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T15:50:19.398 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T15:50:19.398 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T15:50:19.398 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:19.398 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:19.768 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-rook 2024-09-10T15:50:20.353 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:21 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:20.487 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:20.488 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:20.488 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-10T15:50:20.488 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:20.488 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:50:20.488 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 49 k 2024-09-10T15:50:20.488 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-09-10T15:50:20.488 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T15:50:20.488 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:50:20.489 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:20.490 INFO:teuthology.orchestra.run.smithi121.stdout:Install 13 Packages 2024-09-10T15:50:20.490 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:20.490 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 2.2 M 2024-09-10T15:50:20.490 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 26 M 2024-09-10T15:50:20.491 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:50:20.871 INFO:teuthology.orchestra.run.smithi121.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 209 kB/s | 26 kB 00:00 2024-09-10T15:50:20.904 INFO:teuthology.orchestra.run.smithi121.stdout:(2/13): ceph-mgr-rook-19.3.0-4858.gbad1f558.el9 316 kB/s | 49 kB 00:00 2024-09-10T15:50:20.930 INFO:teuthology.orchestra.run.smithi121.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 106 kB/s | 19 kB 00:00 2024-09-10T15:50:20.980 INFO:teuthology.orchestra.run.smithi121.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-10T15:50:21.047 INFO:teuthology.orchestra.run.smithi121.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.3 MB/s | 279 kB 00:00 2024-09-10T15:50:21.072 INFO:teuthology.orchestra.run.smithi121.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 583 kB/s | 54 kB 00:00 2024-09-10T15:50:21.097 INFO:teuthology.orchestra.run.smithi121.stdout:(7/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 824 kB/s | 159 kB 00:00 2024-09-10T15:50:21.148 INFO:teuthology.orchestra.run.smithi121.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 320 kB/s | 32 kB 00:00 2024-09-10T15:50:21.173 INFO:teuthology.orchestra.run.smithi121.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-10T15:50:21.198 INFO:teuthology.orchestra.run.smithi121.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T15:50:21.232 INFO:teuthology.orchestra.run.smithi121.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.8 MB/s | 220 kB 00:00 2024-09-10T15:50:21.257 INFO:teuthology.orchestra.run.smithi121.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-10T15:50:21.291 INFO:teuthology.orchestra.run.smithi121.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-10T15:50:21.293 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:50:21.294 INFO:teuthology.orchestra.run.smithi121.stdout:Total 2.8 MB/s | 2.2 MB 00:00 2024-09-10T15:50:21.358 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:50:21.374 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:50:21.374 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:50:21.465 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:50:21.465 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:50:21.971 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:50:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T15:50:22.192 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T15:50:22.248 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T15:50:22.285 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T15:50:22.326 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T15:50:22.429 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T15:50:22.538 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T15:50:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T15:50:23.319 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T15:50:23.375 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T15:50:23.409 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T15:50:23.457 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T15:50:23.470 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:19.3.0-4858.gbad1f558.el9.noarch 13/13 2024-09-10T15:50:24.282 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4858.gbad1f558.el9.noarch 13/13 2024-09-10T15:50:24.282 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4858.gbad1f558.el9.noarch 1/13 2024-09-10T15:50:24.282 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T15:50:24.282 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T15:50:24.282 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T15:50:24.282 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T15:50:24.282 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T15:50:24.282 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T15:50:24.282 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T15:50:24.282 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T15:50:24.282 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T15:50:24.282 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T15:50:24.283 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T15:50:24.609 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T15:50:24.609 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:24.609 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:50:24.609 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:50:24.609 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T15:50:24.609 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T15:50:24.609 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T15:50:24.609 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T15:50:24.609 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T15:50:24.610 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T15:50:24.610 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T15:50:24.610 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T15:50:24.610 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T15:50:24.610 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T15:50:24.610 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T15:50:24.610 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T15:50:24.610 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:24.610 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:24.823 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T15:50:25.385 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:26 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:25.519 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:25.520 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:25.520 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-10T15:50:25.520 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:25.520 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:50:25.520 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 154 k 2024-09-10T15:50:25.521 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-09-10T15:50:25.521 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T15:50:25.521 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T15:50:25.521 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T15:50:25.521 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T15:50:25.521 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T15:50:25.521 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:25.521 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:50:25.521 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:25.521 INFO:teuthology.orchestra.run.smithi121.stdout:Install 6 Packages 2024-09-10T15:50:25.521 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:25.522 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 7.0 M 2024-09-10T15:50:25.522 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 33 M 2024-09-10T15:50:25.522 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:50:25.902 INFO:teuthology.orchestra.run.smithi121.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4858.gbad1f558.e 748 kB/s | 154 kB 00:00 2024-09-10T15:50:25.952 INFO:teuthology.orchestra.run.smithi121.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-10T15:50:25.985 INFO:teuthology.orchestra.run.smithi121.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.1 MB/s | 58 kB 00:00 2024-09-10T15:50:26.011 INFO:teuthology.orchestra.run.smithi121.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-10T15:50:26.669 INFO:teuthology.orchestra.run.smithi121.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 255 kB/s | 248 kB 00:00 2024-09-10T15:50:27.578 INFO:teuthology.orchestra.run.smithi121.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-09-10T15:50:27.580 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:50:27.580 INFO:teuthology.orchestra.run.smithi121.stdout:Total 3.4 MB/s | 7.0 MB 00:02 2024-09-10T15:50:27.652 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:50:27.663 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:50:27.663 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:50:27.764 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:50:27.764 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:50:28.077 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:50:28.242 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T15:50:28.288 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T15:50:28.813 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T15:50:28.907 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T15:50:28.976 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T15:50:29.009 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 6/6 2024-09-10T15:50:29.793 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 6/6 2024-09-10T15:50:29.793 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 1/6 2024-09-10T15:50:29.793 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T15:50:29.793 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T15:50:29.793 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T15:50:29.793 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T15:50:30.066 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T15:50:30.066 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:30.066 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:50:30.066 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:50:30.066 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T15:50:30.066 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T15:50:30.067 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T15:50:30.067 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T15:50:30.067 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T15:50:30.067 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:30.067 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:30.308 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-fuse 2024-09-10T15:50:30.868 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:31 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:30.999 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:31.000 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:31.000 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-10T15:50:31.000 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:31.000 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:50:31.000 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 901 k 2024-09-10T15:50:31.000 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-09-10T15:50:31.000 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T15:50:31.000 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:31.000 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:50:31.000 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:31.000 INFO:teuthology.orchestra.run.smithi121.stdout:Install 2 Packages 2024-09-10T15:50:31.001 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:31.001 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 981 k 2024-09-10T15:50:31.001 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 2.9 M 2024-09-10T15:50:31.001 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:50:31.246 INFO:teuthology.orchestra.run.smithi121.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 597 kB/s | 80 kB 00:00 2024-09-10T15:50:31.388 INFO:teuthology.orchestra.run.smithi121.stdout:(2/2): ceph-fuse-19.3.0-4858.gbad1f558.el9.x86_ 3.2 MB/s | 901 kB 00:00 2024-09-10T15:50:31.388 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:50:31.389 INFO:teuthology.orchestra.run.smithi121.stdout:Total 2.5 MB/s | 981 kB 00:00 2024-09-10T15:50:31.400 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:50:31.421 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:50:31.421 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:50:31.500 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:50:31.500 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:50:31.681 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:50:31.824 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T15:50:31.886 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 2/2 2024-09-10T15:50:32.564 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 2/2 2024-09-10T15:50:32.565 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 1/2 2024-09-10T15:50:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T15:50:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:32.843 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:50:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T15:50:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:32.843 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:33.133 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-volume 2024-09-10T15:50:33.704 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:33.840 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:33.841 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:33.841 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-10T15:50:33.841 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:33.841 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:50:33.841 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume noarch 2:19.3.0-4858.gbad1f558.el9 ceph-noarch 285 k 2024-09-10T15:50:33.841 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:33.842 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:50:33.842 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:33.842 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-09-10T15:50:33.842 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:33.842 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 285 k 2024-09-10T15:50:33.842 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 1.3 M 2024-09-10T15:50:33.842 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:50:34.068 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-volume-19.3.0-4858.gbad1f558.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-10T15:50:34.068 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:50:34.069 INFO:teuthology.orchestra.run.smithi121.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-10T15:50:34.069 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:50:34.077 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:50:34.077 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:50:34.102 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:50:34.102 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:50:34.325 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:50:34.394 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-volume-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:50:34.410 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-volume-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:50:34.410 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:50:34.410 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T15:50:34.410 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:35.426 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-volume-2:19.3.0-4858.gbad1f558.el9.noarch 1/1 2024-09-10T15:50:35.426 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:35.426 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:50:35.426 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume-2:19.3.0-4858.gbad1f558.el9.noarch 2024-09-10T15:50:35.426 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:35.427 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:35.696 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librados-devel 2024-09-10T15:50:36.278 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:36.416 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:36.417 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:36.417 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2024-09-10T15:50:36.417 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:36.417 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:50:36.417 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 126 k 2024-09-10T15:50:36.418 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:36.418 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:50:36.418 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:36.418 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-09-10T15:50:36.418 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:36.418 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 126 k 2024-09-10T15:50:36.418 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 456 k 2024-09-10T15:50:36.418 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:50:36.606 INFO:teuthology.orchestra.run.smithi121.stdout:librados-devel-19.3.0-4858.gbad1f558.el9.x86_64 673 kB/s | 126 kB 00:00 2024-09-10T15:50:36.607 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:50:36.607 INFO:teuthology.orchestra.run.smithi121.stdout:Total 668 kB/s | 126 kB 00:00 2024-09-10T15:50:36.607 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:50:36.613 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:50:36.613 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:50:36.676 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:50:36.677 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:50:36.839 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:50:36.946 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:37.389 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:37.641 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:37.641 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:37.641 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:50:37.641 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:50:37.641 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:37.642 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:37.911 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install libcephfs2 2024-09-10T15:50:38.485 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:38.576 INFO:teuthology.orchestra.run.smithi121.stdout:Package libcephfs2-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:50:38.620 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:38.622 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-09-10T15:50:38.622 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:38.681 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install libcephfs-devel 2024-09-10T15:50:39.245 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:39.378 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:39.379 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:39.379 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2024-09-10T15:50:39.379 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:39.379 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:50:39.379 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 32 k 2024-09-10T15:50:39.380 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:39.380 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:50:39.380 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:39.380 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-09-10T15:50:39.380 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:39.380 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 32 k 2024-09-10T15:50:39.380 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 142 k 2024-09-10T15:50:39.380 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:50:39.535 INFO:teuthology.orchestra.run.smithi121.stdout:libcephfs-devel-19.3.0-4858.gbad1f558.el9.x86_6 205 kB/s | 32 kB 00:00 2024-09-10T15:50:39.535 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:50:39.536 INFO:teuthology.orchestra.run.smithi121.stdout:Total 203 kB/s | 32 kB 00:00 2024-09-10T15:50:39.536 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:50:39.539 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:50:39.539 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:50:39.557 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:50:39.557 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:50:39.645 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:50:39.758 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:40.140 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:40.362 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:40.363 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:40.363 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:50:40.363 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:50:40.363 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:40.363 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:40.524 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librados2 2024-09-10T15:50:41.079 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:42 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:41.168 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:50:41.210 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:41.211 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-09-10T15:50:41.211 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:41.273 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librbd1 2024-09-10T15:50:41.838 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:42 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:41.929 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:50:41.973 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:41.974 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-09-10T15:50:41.974 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:42.032 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rados 2024-09-10T15:50:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:43 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:42.684 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rados-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:50:42.727 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:42.728 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-09-10T15:50:42.728 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:42.790 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rgw 2024-09-10T15:50:43.351 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:44 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:43.441 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rgw-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:50:43.484 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:43.485 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-09-10T15:50:43.485 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:43.547 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-cephfs 2024-09-10T15:50:44.108 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:45 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:44.198 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-cephfs-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:50:44.240 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:44.242 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-09-10T15:50:44.242 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:44.305 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rbd 2024-09-10T15:50:44.866 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:45 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:44.956 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rbd-2:19.3.0-4858.gbad1f558.el9.x86_64 is already installed. 2024-09-10T15:50:44.999 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:45.000 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-09-10T15:50:45.000 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:45.064 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-fuse 2024-09-10T15:50:45.627 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:46 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:45.759 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:45.760 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:45.760 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-09-10T15:50:45.760 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:45.760 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:50:45.760 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 85 k 2024-09-10T15:50:45.760 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:45.760 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:50:45.760 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:45.760 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-09-10T15:50:45.761 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:45.761 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 85 k 2024-09-10T15:50:45.761 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 231 k 2024-09-10T15:50:45.761 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:50:45.997 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-fuse-19.3.0-4858.gbad1f558.el9.x86_64.rpm 361 kB/s | 85 kB 00:00 2024-09-10T15:50:45.997 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:50:45.997 INFO:teuthology.orchestra.run.smithi121.stdout:Total 359 kB/s | 85 kB 00:00 2024-09-10T15:50:45.997 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:50:46.004 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:50:46.004 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:50:46.069 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:50:46.069 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:50:46.215 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:50:46.319 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:46.679 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:46.939 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:46.939 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:46.939 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:50:46.939 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:50:46.939 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:46.939 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:47.242 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-mirror 2024-09-10T15:50:47.835 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:48 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:47.977 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:47.978 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:47.978 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2024-09-10T15:50:47.978 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:47.978 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:50:47.978 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 3.2 M 2024-09-10T15:50:47.978 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:47.978 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:50:47.978 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:47.979 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-09-10T15:50:47.979 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:47.979 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 3.2 M 2024-09-10T15:50:47.979 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 14 M 2024-09-10T15:50:47.979 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:50:48.317 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-mirror-19.3.0-4858.gbad1f558.el9.x86_64.rpm 9.4 MB/s | 3.2 MB 00:00 2024-09-10T15:50:48.318 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:50:48.318 INFO:teuthology.orchestra.run.smithi121.stdout:Total 9.4 MB/s | 3.2 MB 00:00 2024-09-10T15:50:48.319 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:50:48.328 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:50:48.328 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:50:48.417 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:50:48.417 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:50:48.864 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:50:48.935 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:48.961 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:48.961 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T15:50:48.961 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T15:50:48.961 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T15:50:48.962 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T15:50:48.962 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:50.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:50.087 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:50.087 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:50:50.087 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:50:50.087 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:50.087 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:50.363 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-nbd 2024-09-10T15:50:50.949 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:51 ago on Tue 10 Sep 2024 03:48:59 PM UTC. 2024-09-10T15:50:51.082 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-10T15:50:51.083 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:51.083 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-09-10T15:50:51.083 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:51.083 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-10T15:50:51.083 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:19.3.0-4858.gbad1f558.el9 ceph 170 k 2024-09-10T15:50:51.083 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:51.083 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-10T15:50:51.083 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-09-10T15:50:51.083 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-09-10T15:50:51.083 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:51.083 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 170 k 2024-09-10T15:50:51.084 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 490 k 2024-09-10T15:50:51.084 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-10T15:50:51.321 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-nbd-19.3.0-4858.gbad1f558.el9.x86_64.rpm 717 kB/s | 170 kB 00:00 2024-09-10T15:50:51.322 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-10T15:50:51.322 INFO:teuthology.orchestra.run.smithi121.stdout:Total 713 kB/s | 170 kB 00:00 2024-09-10T15:50:51.322 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-10T15:50:51.330 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-10T15:50:51.330 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-10T15:50:51.395 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-10T15:50:51.395 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-10T15:50:51.547 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-10T15:50:51.644 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:52.080 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:52.346 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:19.3.0-4858.gbad1f558.el9.x86_64 1/1 2024-09-10T15:50:52.346 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:52.346 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-10T15:50:52.347 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:19.3.0-4858.gbad1f558.el9.x86_64 2024-09-10T15:50:52.347 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:52.347 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-10T15:50:52.560 DEBUG:teuthology.parallel:result is None 2024-09-10T15:50:52.560 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bad1f558c327094897e4a32e99dd5cb4ae6ccd46 2024-09-10T15:50:52.703 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T15:50:52.724 INFO:teuthology.orchestra.run.smithi121.stdout:19.3.0-4858.gbad1f558.el9 2024-09-10T15:50:52.725 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4858.gbad1f558.el9 2024-09-10T15:50:52.725 INFO:teuthology.task.install:The correct ceph version 19.3.0-4858.gbad1f558 is installed. 2024-09-10T15:50:52.727 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T15:50:52.727 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:50:52.727 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T15:50:52.790 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T15:50:52.790 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:50:52.790 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T15:50:52.855 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T15:50:52.919 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T15:50:52.920 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:50:52.920 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T15:50:52.983 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T15:50:53.048 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T15:50:53.049 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:50:53.049 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T15:50:53.112 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T15:50:53.176 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T15:50:53.259 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T15:50:53.259 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /var/log/ceph 2024-09-10T15:50:53.283 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T15:50:53.284 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T15:50:53.349 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T15:50:53.349 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T15:50:53.422 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T15:50:53.422 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon client hunt interval max multiple': 2, 'mon election default strategy': 3, 'mon mgr beacon grace': 90, 'ms bind msgr2': False, 'ms inject socket failures': 1000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2024-09-10T15:50:53.422 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899253', 'branch': 'wip-bharath7-testing-2024-09-10-1409', 'description': 'rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-low-osd-mem-target} rados supported-random-distro$/{centos_latest}}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7899253', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath7-testing-2024-09-10-1409'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon client hunt interval max multiple': 2, 'mon election default strategy': 3, 'mon mgr beacon grace': 90, 'ms bind msgr2': False, 'ms inject socket failures': 1000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'mon_bind_msgr2': False, 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath7-testing-2024-09-10-1409', 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2']], 'seed': 5245, 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath7-testing-2024-09-10-1409', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_bad1f558c327094897e4a32e99dd5cb4ae6ccd46/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46', 'targets': {'smithi121.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0LLOr7kl6WLWImY5x6pbqLOhbca7rp3LWyJqOEgo6ahtG+msQKqJ5+e08DvifgRH1/AL7hxxIZb2fPg3HNEAU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon client directed command retry': 5, 'mon client hunt interval max multiple': 2, 'mon election default strategy': 3, 'mon mgr beacon grace': 90, 'ms bind msgr2': False, 'ms inject socket failures': 1000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'mon_bind_msgr2': False, 'sha1': 'bad1f558c327094897e4a32e99dd5cb4ae6ccd46', 'cluster': 'ceph'}}, {'rep_lost_unfound_delete': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_15:30:27', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T15:50:53.423 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T15:50:53.470 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T15:50:53.535 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:50:53.535 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T15:50:53.591 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T15:50:53.591 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2024-09-10T15:50:53.647 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T15:50:53.647 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T15:50:53.647 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-10T15:50:53.647 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T15:50:53.647 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T15:50:53.647 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-10 15:50:49.220733032 +0000 2024-09-10T15:50:53.647 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-10 15:48:19.736198630 +0000 2024-09-10T15:50:53.647 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-10 15:48:19.736198630 +0000 2024-09-10T15:50:53.647 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-10 15:48:19.736198630 +0000 2024-09-10T15:50:53.647 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T15:50:53.711 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-10T15:50:53.711 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-10T15:50:53.712 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000204196 s, 2.5 MB/s 2024-09-10T15:50:53.713 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T15:50:53.769 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2024-09-10T15:50:53.825 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T15:50:53.826 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T15:50:53.826 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-10T15:50:53.826 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T15:50:53.826 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T15:50:53.826 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-10 15:50:49.220733032 +0000 2024-09-10T15:50:53.826 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-10 15:48:20.063194010 +0000 2024-09-10T15:50:53.826 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-10 15:48:20.063194010 +0000 2024-09-10T15:50:53.826 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-10 15:48:20.063194010 +0000 2024-09-10T15:50:53.826 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T15:50:53.889 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-10T15:50:53.889 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-10T15:50:53.890 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000211545 s, 2.4 MB/s 2024-09-10T15:50:53.891 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T15:50:53.947 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2024-09-10T15:50:54.002 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T15:50:54.002 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T15:50:54.002 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-10T15:50:54.002 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T15:50:54.002 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T15:50:54.003 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-10 15:50:49.221733018 +0000 2024-09-10T15:50:54.003 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-10 15:48:20.388189419 +0000 2024-09-10T15:50:54.003 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-10 15:48:20.388189419 +0000 2024-09-10T15:50:54.003 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-10 15:48:20.388189419 +0000 2024-09-10T15:50:54.003 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T15:50:54.065 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-10T15:50:54.065 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-10T15:50:54.065 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000204366 s, 2.5 MB/s 2024-09-10T15:50:54.066 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T15:50:54.123 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2024-09-10T15:50:54.178 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T15:50:54.179 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T15:50:54.179 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-10T15:50:54.179 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T15:50:54.179 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T15:50:54.179 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-10 15:50:49.221733018 +0000 2024-09-10T15:50:54.179 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-10 15:48:20.703184968 +0000 2024-09-10T15:50:54.179 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-10 15:48:20.703184968 +0000 2024-09-10T15:50:54.179 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-10 15:48:20.703184968 +0000 2024-09-10T15:50:54.180 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T15:50:54.242 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-10T15:50:54.243 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-10T15:50:54.243 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000261409 s, 2.0 MB/s 2024-09-10T15:50:54.244 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T15:50:54.300 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-10T15:50:54.300 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-10T15:50:54.300 INFO:tasks.ceph:Generating config... 2024-09-10T15:50:54.302 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-10T15:50:54.302 INFO:tasks.ceph:[global] mon client hunt interval max multiple = 2 2024-09-10T15:50:54.302 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-10T15:50:54.302 INFO:tasks.ceph:[global] mon mgr beacon grace = 90 2024-09-10T15:50:54.302 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-09-10T15:50:54.303 INFO:tasks.ceph:[global] ms inject socket failures = 1000 2024-09-10T15:50:54.303 INFO:tasks.ceph:[global] ms type = async 2024-09-10T15:50:54.303 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T15:50:54.303 INFO:tasks.ceph:[mgr] debug monc = 10 2024-09-10T15:50:54.303 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T15:50:54.303 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T15:50:54.303 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T15:50:54.303 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T15:50:54.303 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-10T15:50:54.303 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-10T15:50:54.303 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-10T15:50:54.303 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T15:50:54.303 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] bluestore write v2 = True 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-10T15:50:54.304 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T15:50:54.305 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-10T15:50:54.305 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-10T15:50:54.305 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-10T15:50:54.305 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T15:50:54.305 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-10T15:50:54.305 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-10T15:50:54.305 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T15:50:54.305 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T15:50:54.384 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T15:50:54.387 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T15:50:54.429 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T15:50:54.494 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.121:6789]'), ('mon.b', '[v1:172.21.15.121:6790]'), ('mon.c', '[v1:172.21.15.121:6791]')] 2024-09-10T15:50:54.495 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.121:6789],[v1:172.21.15.121:6790],[v1:172.21.15.121:6791]', 'mon client directed command retry': 5, 'mon client hunt interval max multiple': 2, 'mon election default strategy': 3, 'mon mgr beacon grace': 90, 'ms bind msgr2': False, 'ms inject socket failures': 1000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug monc': 10}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-10T15:50:54.495 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:50:54.496 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T15:50:54.550 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.121:6789]' --addv b '[v1:172.21.15.121:6790]' --addv c '[v1:172.21.15.121:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:50:54.637 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:50:54.637 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: generated fsid 199e91a6-cca4-4c41-a5fa-011e971838b5 2024-09-10T15:50:54.637 INFO:teuthology.orchestra.run.smithi121.stdout:setting min_mon_release = quincy 2024-09-10T15:50:54.637 INFO:teuthology.orchestra.run.smithi121.stdout:epoch 0 2024-09-10T15:50:54.637 INFO:teuthology.orchestra.run.smithi121.stdout:fsid 199e91a6-cca4-4c41-a5fa-011e971838b5 2024-09-10T15:50:54.637 INFO:teuthology.orchestra.run.smithi121.stdout:last_changed 2024-09-10T15:50:54.636609+0000 2024-09-10T15:50:54.637 INFO:teuthology.orchestra.run.smithi121.stdout:created 2024-09-10T15:50:54.636609+0000 2024-09-10T15:50:54.637 INFO:teuthology.orchestra.run.smithi121.stdout:min_mon_release 17 (quincy) 2024-09-10T15:50:54.637 INFO:teuthology.orchestra.run.smithi121.stdout:election_strategy: 3 2024-09-10T15:50:54.638 INFO:teuthology.orchestra.run.smithi121.stdout:0: v1:172.21.15.121:6789/0 mon.a 2024-09-10T15:50:54.638 INFO:teuthology.orchestra.run.smithi121.stdout:1: v1:172.21.15.121:6790/0 mon.b 2024-09-10T15:50:54.638 INFO:teuthology.orchestra.run.smithi121.stdout:2: v1:172.21.15.121:6791/0 mon.c 2024-09-10T15:50:54.638 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T15:50:54.639 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T15:50:54.654 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 199e91a6-cca4-4c41-a5fa-011e971838b5... 2024-09-10T15:50:54.656 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T15:50:54.736 INFO:teuthology.orchestra.run.smithi121.stdout:[global] 2024-09-10T15:50:54.736 INFO:teuthology.orchestra.run.smithi121.stdout: chdir = "" 2024-09-10T15:50:54.737 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T15:50:54.737 INFO:teuthology.orchestra.run.smithi121.stdout: auth supported = cephx 2024-09-10T15:50:54.737 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.737 INFO:teuthology.orchestra.run.smithi121.stdout: filestore xattr use omap = true 2024-09-10T15:50:54.737 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.737 INFO:teuthology.orchestra.run.smithi121.stdout: mon clock drift allowed = 1.000 2024-09-10T15:50:54.737 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.737 INFO:teuthology.orchestra.run.smithi121.stdout: osd crush chooseleaf type = 0 2024-09-10T15:50:54.737 INFO:teuthology.orchestra.run.smithi121.stdout: auth debug = true 2024-09-10T15:50:54.737 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.737 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on old message = true 2024-09-10T15:50:54.737 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on bug = true 2024-09-10T15:50:54.738 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.738 INFO:teuthology.orchestra.run.smithi121.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T15:50:54.738 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn max object skew = 0 2024-09-10T15:50:54.738 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.738 INFO:teuthology.orchestra.run.smithi121.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T15:50:54.738 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T15:50:54.738 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.738 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default size = 2 2024-09-10T15:50:54.739 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.739 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow primary affinity = true 2024-09-10T15:50:54.739 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow pg remap = true 2024-09-10T15:50:54.739 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on legacy crush tunables = false 2024-09-10T15:50:54.739 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on crush straw calc version zero = false 2024-09-10T15:50:54.739 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on no sortbitwise = false 2024-09-10T15:50:54.739 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = false 2024-09-10T15:50:54.739 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on too few osds = false 2024-09-10T15:50:54.739 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T15:50:54.739 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T15:50:54.739 INFO:teuthology.orchestra.run.smithi121.stdout: mon_allow_pool_size_one = true 2024-09-10T15:50:54.740 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.740 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T15:50:54.740 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.740 INFO:teuthology.orchestra.run.smithi121.stdout: osd default data pool replay window = 5 2024-09-10T15:50:54.740 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.740 INFO:teuthology.orchestra.run.smithi121.stdout: mon allow pool delete = true 2024-09-10T15:50:54.740 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.740 INFO:teuthology.orchestra.run.smithi121.stdout: mon cluster log file level = debug 2024-09-10T15:50:54.740 INFO:teuthology.orchestra.run.smithi121.stdout: debug asserts on shutdown = true 2024-09-10T15:50:54.740 INFO:teuthology.orchestra.run.smithi121.stdout: mon health detail to clog = false 2024-09-10T15:50:54.740 INFO:teuthology.orchestra.run.smithi121.stdout: mon host = "[v1:172.21.15.121:6789],[v1:172.21.15.121:6790],[v1:172.21.15.121:6791]" 2024-09-10T15:50:54.741 INFO:teuthology.orchestra.run.smithi121.stdout: mon client directed command retry = 5 2024-09-10T15:50:54.741 INFO:teuthology.orchestra.run.smithi121.stdout: mon client hunt interval max multiple = 2 2024-09-10T15:50:54.741 INFO:teuthology.orchestra.run.smithi121.stdout: mon election default strategy = 3 2024-09-10T15:50:54.741 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr beacon grace = 90 2024-09-10T15:50:54.741 INFO:teuthology.orchestra.run.smithi121.stdout: ms bind msgr2 = False 2024-09-10T15:50:54.741 INFO:teuthology.orchestra.run.smithi121.stdout: ms inject socket failures = 1000 2024-09-10T15:50:54.741 INFO:teuthology.orchestra.run.smithi121.stdout: ms type = async 2024-09-10T15:50:54.741 INFO:teuthology.orchestra.run.smithi121.stdout: fsid = 199e91a6-cca4-4c41-a5fa-011e971838b5 2024-09-10T15:50:54.741 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.741 INFO:teuthology.orchestra.run.smithi121.stdout:[osd] 2024-09-10T15:50:54.741 INFO:teuthology.orchestra.run.smithi121.stdout: osd journal size = 100 2024-09-10T15:50:54.742 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.742 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub load threshold = 5.0 2024-09-10T15:50:54.742 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub max interval = 600 2024-09-10T15:50:54.742 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock profile = high_recovery_ops 2024-09-10T15:50:54.742 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.742 INFO:teuthology.orchestra.run.smithi121.stdout: osd recover clone overlap = true 2024-09-10T15:50:54.742 INFO:teuthology.orchestra.run.smithi121.stdout: osd recovery max chunk = 1048576 2024-09-10T15:50:54.742 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.742 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug shutdown = true 2024-09-10T15:50:54.742 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug op order = true 2024-09-10T15:50:54.742 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify stray on activate = true 2024-09-10T15:50:54.742 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug trim objects = true 2024-09-10T15:50:54.743 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.743 INFO:teuthology.orchestra.run.smithi121.stdout: osd open classes on start = true 2024-09-10T15:50:54.743 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug pg log writeout = true 2024-09-10T15:50:54.743 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.743 INFO:teuthology.orchestra.run.smithi121.stdout: osd deep scrub update digest min age = 30 2024-09-10T15:50:54.743 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.743 INFO:teuthology.orchestra.run.smithi121.stdout: osd map max advance = 10 2024-09-10T15:50:54.743 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.743 INFO:teuthology.orchestra.run.smithi121.stdout: journal zero on create = true 2024-09-10T15:50:54.743 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.743 INFO:teuthology.orchestra.run.smithi121.stdout: filestore ondisk finisher threads = 3 2024-09-10T15:50:54.743 INFO:teuthology.orchestra.run.smithi121.stdout: filestore apply finisher threads = 3 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: bdev debug aio = true 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug misdirected ops = true 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: bdev async discard = True 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: bdev enable discard = True 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore block size = 96636764160 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore fsck on mount = True 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore write v2 = True 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore zero block detection = True 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluefs = 20 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluestore = 20 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: debug osd = 20 2024-09-10T15:50:54.744 INFO:teuthology.orchestra.run.smithi121.stdout: debug rocksdb = 10 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd full ratio = 0.9 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd nearfull ratio = 0.8 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify cached snaps = True 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify missing on start = True 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: osd failsafe full ratio = 0.95 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock override recovery settings = True 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: osd memory target = 1610612736 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: osd objectstore = bluestore 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: osd op queue = debug_random 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: osd op queue cut off = debug_random 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr] 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-09-10T15:50:54.745 INFO:teuthology.orchestra.run.smithi121.stdout: debug mgr = 20 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout: mgr/telemetry/nag = false 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout: debug monc = 10 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout:[mon] 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout: debug paxos = 20 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-09-10T15:50:54.746 INFO:teuthology.orchestra.run.smithi121.stdout: mon data avail warn = 5 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr mkfs grace = 240 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd reporter subtree level = osd 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd prime pg temp = true 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.747 INFO:teuthology.orchestra.run.smithi121.stdout: # 1m isn't quite enough 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout: mon_down_mkfs_grace = 2m 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_filestore_osds = false 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout: mon scrub interval = 300 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout:[client] 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout: rgw cache enabled = true 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable ops log = true 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable usage log = true 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.a] 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.b] 2024-09-10T15:50:54.748 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.c] 2024-09-10T15:50:54.749 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T15:50:54.750 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T15:50:54.845 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T15:50:54.846 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:50:54.846 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T15:50:54.861 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:50:54.861 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T15:50:54.916 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi121.front.sepia.ceph.com 2024-09-10T15:50:54.917 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:50:54.917 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T15:50:54.917 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T15:50:54.991 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:50:54.992 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:50:55.047 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T15:50:55.047 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T15:50:55.047 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T15:50:55.138 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T15:50:55.140 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T15:50:55.140 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T15:50:55.141 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T15:50:55.141 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-10T15:50:55.141 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T15:50:55.165 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-10T15:50:55.165 INFO:tasks.ceph:role: osd.0 2024-09-10T15:50:55.166 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com 2024-09-10T15:50:55.166 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T15:50:55.237 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T15:50:55.237 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T15:50:55.237 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T15:50:55.238 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T15:50:55.238 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T15:50:55.238 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-09-10T15:50:55.238 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T15:50:55.238 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T15:50:55.238 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T15:50:55.238 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T15:50:55.753 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-09-10T15:50:55.755 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-09-10T15:50:55.755 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T15:50:55.874 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T15:50:55.909 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T15:50:55.976 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-10T15:50:55.976 INFO:tasks.ceph:role: osd.1 2024-09-10T15:50:55.976 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com 2024-09-10T15:50:55.976 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T15:50:56.050 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T15:50:56.050 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T15:50:56.050 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T15:50:56.050 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T15:50:56.051 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T15:50:56.051 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-09-10T15:50:56.051 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T15:50:56.051 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T15:50:56.051 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T15:50:56.051 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T15:50:56.512 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-09-10T15:50:56.514 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-09-10T15:50:56.514 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T15:50:56.610 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T15:50:56.641 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T15:50:56.709 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-10T15:50:56.710 INFO:tasks.ceph:role: osd.2 2024-09-10T15:50:56.710 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com 2024-09-10T15:50:56.710 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T15:50:56.782 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T15:50:56.783 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T15:50:56.783 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T15:50:56.783 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T15:50:56.783 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T15:50:56.783 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-09-10T15:50:56.783 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T15:50:56.783 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T15:50:56.783 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T15:50:56.783 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T15:50:57.277 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-09-10T15:50:57.279 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-09-10T15:50:57.279 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T15:50:57.432 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T15:50:57.466 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:50:57.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T15:50:57.570+0000 7f99aa72b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T15:50:57.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T15:50:57.575+0000 7f99aa72b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T15:50:57.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T15:50:57.575+0000 7f99aa72b640 -1 bdev(0x5580f2909800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T15:50:57.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T15:50:57.575+0000 7f99aa72b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T15:51:00.976 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T15:51:01.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:51:01.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T15:51:01.093+0000 7fc434616640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T15:51:01.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T15:51:01.093+0000 7fc434616640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T15:51:01.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T15:51:01.093+0000 7fc434616640 -1 bdev(0x55ef7cd93800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T15:51:01.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T15:51:01.093+0000 7fc434616640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T15:51:04.260 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T15:51:04.288 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:51:04.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T15:51:04.377+0000 7fd54661c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T15:51:04.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T15:51:04.377+0000 7fd54661c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T15:51:04.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T15:51:04.377+0000 7fd54661c640 -1 bdev(0x55a2a9299800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T15:51:04.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-10T15:51:04.377+0000 7fd54661c640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T15:51:07.531 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T15:51:07.561 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T15:51:07.561 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:51:07.561 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T15:51:07.629 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:51:07.629 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T15:51:07.697 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:51:07.697 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T15:51:07.764 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:51:07.764 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T15:51:07.830 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T15:51:07.830 DEBUG:teuthology.orchestra.run.smithi121:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T15:51:07.895 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.x] 2024-09-10T15:51:07.895 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDfauBmxwc3CBAAzj1J0H2CMBET9dQPCshoFg== 2024-09-10T15:51:07.896 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.0] 2024-09-10T15:51:07.896 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDhauBm3VINIhAATPT2BrTBAwfMUsyv4LhLGw== 2024-09-10T15:51:07.896 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.1] 2024-09-10T15:51:07.896 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDlauBmaFSeBRAASKBoaZhbXnJFDtdm7T6xuA== 2024-09-10T15:51:07.896 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.2] 2024-09-10T15:51:07.896 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDoauBmAqSHFhAAKXoSUBNzvB/onZQSDhVAKg== 2024-09-10T15:51:07.897 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T15:51:07.989 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T15:51:08.040 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T15:51:08.091 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T15:51:08.141 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T15:51:08.142 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T15:51:08.166 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T15:51:08.263 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T15:51:08.287 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T15:51:08.352 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T15:51:08.447 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T15:51:08.472 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T15:51:08.537 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T15:51:08.633 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T15:51:08.657 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T15:51:08.713 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T15:51:08.714 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T15:51:08.714 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T15:51:08.757 INFO:tasks.ceph.mon.a:Started 2024-09-10T15:51:08.758 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T15:51:08.758 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T15:51:08.760 INFO:tasks.ceph.mon.b:Started 2024-09-10T15:51:08.760 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T15:51:08.760 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T15:51:08.762 INFO:tasks.ceph.mon.c:Started 2024-09-10T15:51:08.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph config set mgr mgr_pool false --force 2024-09-10T15:51:14.168 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T15:51:14.169 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T15:51:14.169 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T15:51:14.172 INFO:tasks.ceph.mgr.x:Started 2024-09-10T15:51:14.172 DEBUG:tasks.ceph:set 0 configs 2024-09-10T15:51:14.172 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T15:51:14.173 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T15:51:14.348 INFO:teuthology.orchestra.run.smithi121.stderr:adjusted tunables profile to default 2024-09-10T15:51:14.359 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T15:51:14.360 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T15:51:14.360 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:51:14.360 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T15:51:14.385 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:51:14.385 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T15:51:14.450 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-10T15:51:14.450 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T15:51:14.514 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 53b58107-009b-46ac-921d-2e8d9580081e 0 2024-09-10T15:51:14.736 INFO:teuthology.orchestra.run.smithi121.stdout:0 2024-09-10T15:51:14.747 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new aced9681-6a62-49a4-a1e2-6ff7d9a544ef 1 2024-09-10T15:51:14.930 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-09-10T15:51:14.941 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 0cfba235-9896-429e-8396-69dfeb7a44bb 2 2024-09-10T15:51:15.121 INFO:teuthology.orchestra.run.smithi121.stdout:2 2024-09-10T15:51:15.132 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T15:51:15.132 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T15:51:15.134 INFO:tasks.ceph.osd.0:Started 2024-09-10T15:51:15.135 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T15:51:15.135 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T15:51:15.137 INFO:tasks.ceph.osd.1:Started 2024-09-10T15:51:15.137 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T15:51:15.137 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T15:51:15.140 INFO:tasks.ceph.osd.2:Started 2024-09-10T15:51:15.140 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:51:15.357 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:15.357 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":5,"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","created":"2024-09-10T15:51:14.060262+0000","modified":"2024-09-10T15:51:15.118607+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T15:51:15.118607+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"53b58107-009b-46ac-921d-2e8d9580081e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"aced9681-6a62-49a4-a1e2-6ff7d9a544ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0cfba235-9896-429e-8396-69dfeb7a44bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:51:15.368 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T15:51:15.368 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T15:51:15.670 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T15:51:15.828 INFO:teuthology.misc.health.smithi121.stdout: 2024-09-10T15:51:15.828 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":5,"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","created":"2024-09-10T15:51:14.060262+0000","modified":"2024-09-10T15:51:15.118607+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T15:51:15.118607+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"53b58107-009b-46ac-921d-2e8d9580081e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"aced9681-6a62-49a4-a1e2-6ff7d9a544ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0cfba235-9896-429e-8396-69dfeb7a44bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:51:15.838 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-10T15:51:16.603 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-09-10T15:51:16.601+0000 7fb79d6d6640 -1 Falling back to public interface 2024-09-10T15:51:16.829 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-09-10T15:51:16.828+0000 7f925b233640 -1 Falling back to public interface 2024-09-10T15:51:17.104 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:51:17.102+0000 7f3604413640 -1 Falling back to public interface 2024-09-10T15:51:19.421 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-09-10T15:51:19.419+0000 7f834de17640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T15:51:19.422 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-09-10T15:51:19.419+0000 7f834de17640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T15:51:20.390 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-09-10T15:51:20.388+0000 7fb79d6d6640 -1 osd.2 0 log_to_monitors true 2024-09-10T15:51:20.619 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-09-10T15:51:20.617+0000 7f925b233640 -1 osd.1 0 log_to_monitors true 2024-09-10T15:51:21.154 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:51:21.152+0000 7f3604413640 -1 osd.0 0 log_to_monitors true 2024-09-10T15:51:22.132 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-09-10T15:51:22.131+0000 7fb799665640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T15:51:22.137 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-09-10T15:51:22.135+0000 7fb791484640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:51:22.141 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T15:51:22.414 INFO:teuthology.misc.health.smithi121.stdout: 2024-09-10T15:51:22.414 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":7,"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","created":"2024-09-10T15:51:14.060262+0000","modified":"2024-09-10T15:51:21.419936+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T15:51:15.118607+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"53b58107-009b-46ac-921d-2e8d9580081e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"aced9681-6a62-49a4-a1e2-6ff7d9a544ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0cfba235-9896-429e-8396-69dfeb7a44bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:51:22.427 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-10T15:51:23.577 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:51:23.575+0000 7f35ffd06640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T15:51:23.582 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:51:23.581+0000 7f35f7b25640 -1 osd.0 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:51:23.585 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-09-10T15:51:23.583+0000 7f92579d6640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T15:51:23.589 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-09-10T15:51:23.588+0000 7f924efe1640 -1 osd.1 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:51:28.730 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T15:51:29.015 INFO:teuthology.misc.health.smithi121.stdout: 2024-09-10T15:51:29.015 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":11,"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","created":"2024-09-10T15:51:14.060262+0000","modified":"2024-09-10T15:51:25.584286+0000","last_up_change":"2024-09-10T15:51:24.577743+0000","last_in_change":"2024-09-10T15:51:15.118607+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"53b58107-009b-46ac-921d-2e8d9580081e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6808","nonce":2908859504}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6809","nonce":2908859504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6811","nonce":2908859504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6810","nonce":2908859504}]},"public_addr":"172.21.15.121:6808/2908859504","cluster_addr":"172.21.15.121:6809/2908859504","heartbeat_back_addr":"172.21.15.121:6811/2908859504","heartbeat_front_addr":"172.21.15.121:6810/2908859504","state":["exists","up"]},{"osd":1,"uuid":"aced9681-6a62-49a4-a1e2-6ff7d9a544ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6804","nonce":3301493939}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6805","nonce":3301493939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6807","nonce":3301493939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6806","nonce":3301493939}]},"public_addr":"172.21.15.121:6804/3301493939","cluster_addr":"172.21.15.121:6805/3301493939","heartbeat_back_addr":"172.21.15.121:6807/3301493939","heartbeat_front_addr":"172.21.15.121:6806/3301493939","state":["exists","up"]},{"osd":2,"uuid":"0cfba235-9896-429e-8396-69dfeb7a44bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6800","nonce":1473565644}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6801","nonce":1473565644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6803","nonce":1473565644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6802","nonce":1473565644}]},"public_addr":"172.21.15.121:6800/1473565644","cluster_addr":"172.21.15.121:6801/1473565644","heartbeat_back_addr":"172.21.15.121:6803/1473565644","heartbeat_front_addr":"172.21.15.121:6802/1473565644","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:22.164346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.644539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.413080+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:51:29.027 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-10T15:51:29.027 INFO:tasks.ceph:Creating RBD pool 2024-09-10T15:51:29.027 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-10T15:51:29.607 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'rbd' created 2024-09-10T15:51:29.621 DEBUG:teuthology.orchestra.run.smithi121:> rbd --cluster ceph pool init rbd 2024-09-10T15:51:32.636 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T15:51:32.637 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-10T15:51:32.638 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T15:51:32.928 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:32.941 INFO:teuthology.orchestra.run.smithi121.stdout:[{"version":5,"timestamp":"2024-09-10T15:51:23.584909+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"7992.768442"}]}] 2024-09-10T15:51:32.941 INFO:tasks.ceph_manager:config epoch is 5 2024-09-10T15:51:32.941 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T15:51:32.942 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T15:51:32.942 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T15:51:33.271 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:33.282 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":4,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6812","nonce":2431597205}]},"active_addr":"172.21.15.121:6812/2431597205","active_change":"2024-09-10T15:51:18.409383+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":690359194}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":4260189282}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":2700798139}]}]} 2024-09-10T15:51:33.286 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T15:51:33.287 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T15:51:33.287 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:51:33.570 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:33.570 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":15,"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","created":"2024-09-10T15:51:14.060262+0000","modified":"2024-09-10T15:51:32.612253+0000","last_up_change":"2024-09-10T15:51:24.577743+0000","last_in_change":"2024-09-10T15:51:15.118607+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:51:29.314004+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"53b58107-009b-46ac-921d-2e8d9580081e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6808","nonce":2908859504}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6809","nonce":2908859504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6811","nonce":2908859504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6810","nonce":2908859504}]},"public_addr":"172.21.15.121:6808/2908859504","cluster_addr":"172.21.15.121:6809/2908859504","heartbeat_back_addr":"172.21.15.121:6811/2908859504","heartbeat_front_addr":"172.21.15.121:6810/2908859504","state":["exists","up"]},{"osd":1,"uuid":"aced9681-6a62-49a4-a1e2-6ff7d9a544ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6804","nonce":3301493939}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6805","nonce":3301493939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6807","nonce":3301493939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6806","nonce":3301493939}]},"public_addr":"172.21.15.121:6804/3301493939","cluster_addr":"172.21.15.121:6805/3301493939","heartbeat_back_addr":"172.21.15.121:6807/3301493939","heartbeat_front_addr":"172.21.15.121:6806/3301493939","state":["exists","up"]},{"osd":2,"uuid":"0cfba235-9896-429e-8396-69dfeb7a44bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6800","nonce":1473565644}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6801","nonce":1473565644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6803","nonce":1473565644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6802","nonce":1473565644}]},"public_addr":"172.21.15.121:6800/1473565644","cluster_addr":"172.21.15.121:6801/1473565644","heartbeat_back_addr":"172.21.15.121:6803/1473565644","heartbeat_front_addr":"172.21.15.121:6802/1473565644","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:22.164346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.644539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.413080+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:51:33.582 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T15:51:33.583 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:51:33.863 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:33.863 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":15,"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","created":"2024-09-10T15:51:14.060262+0000","modified":"2024-09-10T15:51:32.612253+0000","last_up_change":"2024-09-10T15:51:24.577743+0000","last_in_change":"2024-09-10T15:51:15.118607+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:51:29.314004+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"53b58107-009b-46ac-921d-2e8d9580081e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6808","nonce":2908859504}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6809","nonce":2908859504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6811","nonce":2908859504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6810","nonce":2908859504}]},"public_addr":"172.21.15.121:6808/2908859504","cluster_addr":"172.21.15.121:6809/2908859504","heartbeat_back_addr":"172.21.15.121:6811/2908859504","heartbeat_front_addr":"172.21.15.121:6810/2908859504","state":["exists","up"]},{"osd":1,"uuid":"aced9681-6a62-49a4-a1e2-6ff7d9a544ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6804","nonce":3301493939}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6805","nonce":3301493939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6807","nonce":3301493939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6806","nonce":3301493939}]},"public_addr":"172.21.15.121:6804/3301493939","cluster_addr":"172.21.15.121:6805/3301493939","heartbeat_back_addr":"172.21.15.121:6807/3301493939","heartbeat_front_addr":"172.21.15.121:6806/3301493939","state":["exists","up"]},{"osd":2,"uuid":"0cfba235-9896-429e-8396-69dfeb7a44bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6800","nonce":1473565644}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6801","nonce":1473565644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6803","nonce":1473565644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6802","nonce":1473565644}]},"public_addr":"172.21.15.121:6800/1473565644","cluster_addr":"172.21.15.121:6801/1473565644","heartbeat_back_addr":"172.21.15.121:6803/1473565644","heartbeat_front_addr":"172.21.15.121:6802/1473565644","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:22.164346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.644539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.413080+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:51:33.876 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:51:33.877 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T15:51:33.877 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T15:51:34.011 INFO:teuthology.orchestra.run.smithi121.stdout:34359738371 2024-09-10T15:51:34.011 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:51:34.030 INFO:teuthology.orchestra.run.smithi121.stdout:42949672965 2024-09-10T15:51:34.030 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:51:34.032 INFO:teuthology.orchestra.run.smithi121.stdout:42949672964 2024-09-10T15:51:34.032 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:51:34.310 INFO:teuthology.orchestra.run.smithi121.stdout:34359738370 2024-09-10T15:51:34.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738370 for osd.2 2024-09-10T15:51:34.381 INFO:teuthology.orchestra.run.smithi121.stdout:42949672963 2024-09-10T15:51:34.385 INFO:teuthology.orchestra.run.smithi121.stdout:42949672964 2024-09-10T15:51:34.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672964 got 42949672963 for osd.0 2024-09-10T15:51:34.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672964 for osd.1 2024-09-10T15:51:35.323 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:51:35.395 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:51:35.398 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:51:35.639 INFO:teuthology.orchestra.run.smithi121.stdout:34359738371 2024-09-10T15:51:35.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738371 for osd.2 2024-09-10T15:51:35.652 DEBUG:teuthology.parallel:result is None 2024-09-10T15:51:35.701 INFO:teuthology.orchestra.run.smithi121.stdout:42949672964 2024-09-10T15:51:35.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672964 got 42949672964 for osd.0 2024-09-10T15:51:35.713 DEBUG:teuthology.parallel:result is None 2024-09-10T15:51:35.743 INFO:teuthology.orchestra.run.smithi121.stdout:42949672965 2024-09-10T15:51:35.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672965 for osd.1 2024-09-10T15:51:35.756 DEBUG:teuthology.parallel:result is None 2024-09-10T15:51:35.756 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T15:51:35.756 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:51:36.073 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:36.074 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:51:36.085 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-10T15:51:34.416425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81464,"kb_used_data":392,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034056,"statfs":{"total":289910292480,"available":289826873344,"internally_reserved":0,"allocated":401408,"data_stored":92210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.634342+0000","last_change":"2024-09-10T15:51:32.634606+0000","last_active":"2024-09-10T15:51:32.634342+0000","last_peered":"2024-09-10T15:51:32.634342+0000","last_clean":"2024-09-10T15:51:32.634342+0000","last_became_active":"2024-09-10T15:51:30.614886+0000","last_became_peered":"2024-09-10T15:51:30.614886+0000","last_unstale":"2024-09-10T15:51:32.634342+0000","last_undegraded":"2024-09-10T15:51:32.634342+0000","last_fullsized":"2024-09-10T15:51:32.634342+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:28.735655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.633804+0000","last_change":"2024-09-10T15:51:32.633999+0000","last_active":"2024-09-10T15:51:32.633804+0000","last_peered":"2024-09-10T15:51:32.633804+0000","last_clean":"2024-09-10T15:51:32.633804+0000","last_became_active":"2024-09-10T15:51:30.613428+0000","last_became_peered":"2024-09-10T15:51:30.613428+0000","last_unstale":"2024-09-10T15:51:32.633804+0000","last_undegraded":"2024-09-10T15:51:32.633804+0000","last_fullsized":"2024-09-10T15:51:32.633804+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:18.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.623172+0000","last_change":"2024-09-10T15:51:32.623172+0000","last_active":"2024-09-10T15:51:32.623172+0000","last_peered":"2024-09-10T15:51:32.623172+0000","last_clean":"2024-09-10T15:51:32.623172+0000","last_became_active":"2024-09-10T15:51:30.613434+0000","last_became_peered":"2024-09-10T15:51:30.613434+0000","last_unstale":"2024-09-10T15:51:32.623172+0000","last_undegraded":"2024-09-10T15:51:32.623172+0000","last_fullsized":"2024-09-10T15:51:32.623172+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:58.317152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.633758+0000","last_change":"2024-09-10T15:51:32.634015+0000","last_active":"2024-09-10T15:51:32.633758+0000","last_peered":"2024-09-10T15:51:32.633758+0000","last_clean":"2024-09-10T15:51:32.633758+0000","last_became_active":"2024-09-10T15:51:30.613543+0000","last_became_peered":"2024-09-10T15:51:30.613543+0000","last_unstale":"2024-09-10T15:51:32.633758+0000","last_undegraded":"2024-09-10T15:51:32.633758+0000","last_fullsized":"2024-09-10T15:51:32.633758+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:26.573051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"13'1","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.634061+0000","last_change":"2024-09-10T15:51:32.634287+0000","last_active":"2024-09-10T15:51:32.634061+0000","last_peered":"2024-09-10T15:51:32.634061+0000","last_clean":"2024-09-10T15:51:32.634061+0000","last_became_active":"2024-09-10T15:51:30.616135+0000","last_became_peered":"2024-09-10T15:51:30.616135+0000","last_unstale":"2024-09-10T15:51:32.634061+0000","last_undegraded":"2024-09-10T15:51:32.634061+0000","last_fullsized":"2024-09-10T15:51:32.634061+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:18.351885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"15'2","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.629990+0000","last_change":"2024-09-10T15:51:32.629990+0000","last_active":"2024-09-10T15:51:32.629990+0000","last_peered":"2024-09-10T15:51:32.629990+0000","last_clean":"2024-09-10T15:51:32.629990+0000","last_became_active":"2024-09-10T15:51:30.617003+0000","last_became_peered":"2024-09-10T15:51:30.617003+0000","last_unstale":"2024-09-10T15:51:32.629990+0000","last_undegraded":"2024-09-10T15:51:32.629990+0000","last_fullsized":"2024-09-10T15:51:32.629990+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:50.477908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.623176+0000","last_change":"2024-09-10T15:51:32.623176+0000","last_active":"2024-09-10T15:51:32.623176+0000","last_peered":"2024-09-10T15:51:32.623176+0000","last_clean":"2024-09-10T15:51:32.623176+0000","last_became_active":"2024-09-10T15:51:30.613566+0000","last_became_peered":"2024-09-10T15:51:30.613566+0000","last_unstale":"2024-09-10T15:51:32.623176+0000","last_undegraded":"2024-09-10T15:51:32.623176+0000","last_fullsized":"2024-09-10T15:51:32.623176+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:05.565044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.634106+0000","last_change":"2024-09-10T15:51:32.634333+0000","last_active":"2024-09-10T15:51:32.634106+0000","last_peered":"2024-09-10T15:51:32.634106+0000","last_clean":"2024-09-10T15:51:32.634106+0000","last_became_active":"2024-09-10T15:51:30.613634+0000","last_became_peered":"2024-09-10T15:51:30.613634+0000","last_unstale":"2024-09-10T15:51:32.634106+0000","last_undegraded":"2024-09-10T15:51:32.634106+0000","last_fullsized":"2024-09-10T15:51:32.634106+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:50.070951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672965,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":135168,"data_stored":30743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672964,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":135168,"data_stored":30743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":131072,"data_stored":30724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:51:36.086 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:51:36.366 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:36.367 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:51:36.378 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-10T15:51:34.416425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81464,"kb_used_data":392,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034056,"statfs":{"total":289910292480,"available":289826873344,"internally_reserved":0,"allocated":401408,"data_stored":92210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.634342+0000","last_change":"2024-09-10T15:51:32.634606+0000","last_active":"2024-09-10T15:51:32.634342+0000","last_peered":"2024-09-10T15:51:32.634342+0000","last_clean":"2024-09-10T15:51:32.634342+0000","last_became_active":"2024-09-10T15:51:30.614886+0000","last_became_peered":"2024-09-10T15:51:30.614886+0000","last_unstale":"2024-09-10T15:51:32.634342+0000","last_undegraded":"2024-09-10T15:51:32.634342+0000","last_fullsized":"2024-09-10T15:51:32.634342+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:28.735655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.633804+0000","last_change":"2024-09-10T15:51:32.633999+0000","last_active":"2024-09-10T15:51:32.633804+0000","last_peered":"2024-09-10T15:51:32.633804+0000","last_clean":"2024-09-10T15:51:32.633804+0000","last_became_active":"2024-09-10T15:51:30.613428+0000","last_became_peered":"2024-09-10T15:51:30.613428+0000","last_unstale":"2024-09-10T15:51:32.633804+0000","last_undegraded":"2024-09-10T15:51:32.633804+0000","last_fullsized":"2024-09-10T15:51:32.633804+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:18.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.623172+0000","last_change":"2024-09-10T15:51:32.623172+0000","last_active":"2024-09-10T15:51:32.623172+0000","last_peered":"2024-09-10T15:51:32.623172+0000","last_clean":"2024-09-10T15:51:32.623172+0000","last_became_active":"2024-09-10T15:51:30.613434+0000","last_became_peered":"2024-09-10T15:51:30.613434+0000","last_unstale":"2024-09-10T15:51:32.623172+0000","last_undegraded":"2024-09-10T15:51:32.623172+0000","last_fullsized":"2024-09-10T15:51:32.623172+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:58.317152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.633758+0000","last_change":"2024-09-10T15:51:32.634015+0000","last_active":"2024-09-10T15:51:32.633758+0000","last_peered":"2024-09-10T15:51:32.633758+0000","last_clean":"2024-09-10T15:51:32.633758+0000","last_became_active":"2024-09-10T15:51:30.613543+0000","last_became_peered":"2024-09-10T15:51:30.613543+0000","last_unstale":"2024-09-10T15:51:32.633758+0000","last_undegraded":"2024-09-10T15:51:32.633758+0000","last_fullsized":"2024-09-10T15:51:32.633758+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:26.573051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"13'1","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.634061+0000","last_change":"2024-09-10T15:51:32.634287+0000","last_active":"2024-09-10T15:51:32.634061+0000","last_peered":"2024-09-10T15:51:32.634061+0000","last_clean":"2024-09-10T15:51:32.634061+0000","last_became_active":"2024-09-10T15:51:30.616135+0000","last_became_peered":"2024-09-10T15:51:30.616135+0000","last_unstale":"2024-09-10T15:51:32.634061+0000","last_undegraded":"2024-09-10T15:51:32.634061+0000","last_fullsized":"2024-09-10T15:51:32.634061+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:18.351885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"15'2","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.629990+0000","last_change":"2024-09-10T15:51:32.629990+0000","last_active":"2024-09-10T15:51:32.629990+0000","last_peered":"2024-09-10T15:51:32.629990+0000","last_clean":"2024-09-10T15:51:32.629990+0000","last_became_active":"2024-09-10T15:51:30.617003+0000","last_became_peered":"2024-09-10T15:51:30.617003+0000","last_unstale":"2024-09-10T15:51:32.629990+0000","last_undegraded":"2024-09-10T15:51:32.629990+0000","last_fullsized":"2024-09-10T15:51:32.629990+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:50.477908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.623176+0000","last_change":"2024-09-10T15:51:32.623176+0000","last_active":"2024-09-10T15:51:32.623176+0000","last_peered":"2024-09-10T15:51:32.623176+0000","last_clean":"2024-09-10T15:51:32.623176+0000","last_became_active":"2024-09-10T15:51:30.613566+0000","last_became_peered":"2024-09-10T15:51:30.613566+0000","last_unstale":"2024-09-10T15:51:32.623176+0000","last_undegraded":"2024-09-10T15:51:32.623176+0000","last_fullsized":"2024-09-10T15:51:32.623176+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:05.565044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.634106+0000","last_change":"2024-09-10T15:51:32.634333+0000","last_active":"2024-09-10T15:51:32.634106+0000","last_peered":"2024-09-10T15:51:32.634106+0000","last_clean":"2024-09-10T15:51:32.634106+0000","last_became_active":"2024-09-10T15:51:30.613634+0000","last_became_peered":"2024-09-10T15:51:30.613634+0000","last_unstale":"2024-09-10T15:51:32.634106+0000","last_undegraded":"2024-09-10T15:51:32.634106+0000","last_fullsized":"2024-09-10T15:51:32.634106+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:50.070951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672965,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":135168,"data_stored":30743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672964,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":135168,"data_stored":30743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":131072,"data_stored":30724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:51:36.379 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T15:51:36.379 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T15:51:36.379 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T15:51:36.379 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T15:51:36.696 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:36.696 INFO:teuthology.orchestra.run.smithi121.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T15:51:36.708 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T15:51:36.708 INFO:teuthology.run_tasks:Running task rep_lost_unfound_delete... 2024-09-10T15:51:36.720 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:51:37.002 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:37.003 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":15,"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","created":"2024-09-10T15:51:14.060262+0000","modified":"2024-09-10T15:51:32.612253+0000","last_up_change":"2024-09-10T15:51:24.577743+0000","last_in_change":"2024-09-10T15:51:15.118607+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:51:29.314004+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"53b58107-009b-46ac-921d-2e8d9580081e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6808","nonce":2908859504}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6809","nonce":2908859504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6811","nonce":2908859504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6810","nonce":2908859504}]},"public_addr":"172.21.15.121:6808/2908859504","cluster_addr":"172.21.15.121:6809/2908859504","heartbeat_back_addr":"172.21.15.121:6811/2908859504","heartbeat_front_addr":"172.21.15.121:6810/2908859504","state":["exists","up"]},{"osd":1,"uuid":"aced9681-6a62-49a4-a1e2-6ff7d9a544ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6804","nonce":3301493939}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6805","nonce":3301493939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6807","nonce":3301493939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6806","nonce":3301493939}]},"public_addr":"172.21.15.121:6804/3301493939","cluster_addr":"172.21.15.121:6805/3301493939","heartbeat_back_addr":"172.21.15.121:6807/3301493939","heartbeat_front_addr":"172.21.15.121:6806/3301493939","state":["exists","up"]},{"osd":2,"uuid":"0cfba235-9896-429e-8396-69dfeb7a44bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6800","nonce":1473565644}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6801","nonce":1473565644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6803","nonce":1473565644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6802","nonce":1473565644}]},"public_addr":"172.21.15.121:6800/1473565644","cluster_addr":"172.21.15.121:6801/1473565644","heartbeat_back_addr":"172.21.15.121:6803/1473565644","heartbeat_front_addr":"172.21.15.121:6802/1473565644","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:22.164346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.644539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.413080+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:51:37.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-10T15:51:29.314004+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 15, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T15:51:37.016 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-10T15:51:37.300 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 8 2024-09-10T15:51:37.312 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-10T15:51:37.593 INFO:teuthology.orchestra.run.smithi121.stdout:epoch 15 2024-09-10T15:51:37.593 INFO:teuthology.orchestra.run.smithi121.stdout:fsid 199e91a6-cca4-4c41-a5fa-011e971838b5 2024-09-10T15:51:37.593 INFO:teuthology.orchestra.run.smithi121.stdout:created 2024-09-10T15:51:14.060262+0000 2024-09-10T15:51:37.593 INFO:teuthology.orchestra.run.smithi121.stdout:modified 2024-09-10T15:51:32.612253+0000 2024-09-10T15:51:37.593 INFO:teuthology.orchestra.run.smithi121.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T15:51:37.593 INFO:teuthology.orchestra.run.smithi121.stdout:crush_version 5 2024-09-10T15:51:37.593 INFO:teuthology.orchestra.run.smithi121.stdout:full_ratio 0.95 2024-09-10T15:51:37.594 INFO:teuthology.orchestra.run.smithi121.stdout:backfillfull_ratio 0.9 2024-09-10T15:51:37.594 INFO:teuthology.orchestra.run.smithi121.stdout:nearfull_ratio 0.85 2024-09-10T15:51:37.594 INFO:teuthology.orchestra.run.smithi121.stdout:require_min_compat_client luminous 2024-09-10T15:51:37.594 INFO:teuthology.orchestra.run.smithi121.stdout:min_compat_client jewel 2024-09-10T15:51:37.594 INFO:teuthology.orchestra.run.smithi121.stdout:require_osd_release squid 2024-09-10T15:51:37.594 INFO:teuthology.orchestra.run.smithi121.stdout:stretch_mode_enabled false 2024-09-10T15:51:37.594 INFO:teuthology.orchestra.run.smithi121.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 15 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2024-09-10T15:51:37.594 INFO:teuthology.orchestra.run.smithi121.stdout: removed_snaps_queue [2~1] 2024-09-10T15:51:37.594 INFO:teuthology.orchestra.run.smithi121.stdout:max_osd 3 2024-09-10T15:51:37.594 INFO:teuthology.orchestra.run.smithi121.stdout:osd.0 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v1:172.21.15.121:6808/2908859504 v1:172.21.15.121:6809/2908859504 exists,up 53b58107-009b-46ac-921d-2e8d9580081e 2024-09-10T15:51:37.594 INFO:teuthology.orchestra.run.smithi121.stdout:osd.1 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v1:172.21.15.121:6804/3301493939 v1:172.21.15.121:6805/3301493939 exists,up aced9681-6a62-49a4-a1e2-6ff7d9a544ef 2024-09-10T15:51:37.594 INFO:teuthology.orchestra.run.smithi121.stdout:osd.2 up in weight 1 up_from 8 up_thru 12 down_at 0 last_clean_interval [0,0) v1:172.21.15.121:6800/1473565644 v1:172.21.15.121:6801/1473565644 exists,up 0cfba235-9896-429e-8396-69dfeb7a44bb 2024-09-10T15:51:37.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:['osd.0 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v1:172.21.15.121:6808/2908859504 v1:172.21.15.121:6809/2908859504 exists,up 53b58107-009b-46ac-921d-2e8d9580081e', 'osd.1 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v1:172.21.15.121:6804/3301493939 v1:172.21.15.121:6805/3301493939 exists,up aced9681-6a62-49a4-a1e2-6ff7d9a544ef', 'osd.2 up in weight 1 up_from 8 up_thru 12 down_at 0 last_clean_interval [0,0) v1:172.21.15.121:6800/1473565644 v1:172.21.15.121:6801/1473565644 exists,up 0cfba235-9896-429e-8396-69dfeb7a44bb'] 2024-09-10T15:51:37.605 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:51:37.606 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T15:51:37.606 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T15:51:37.743 INFO:teuthology.orchestra.run.smithi121.stdout:34359738373 2024-09-10T15:51:37.744 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:51:37.745 INFO:teuthology.orchestra.run.smithi121.stdout:42949672966 2024-09-10T15:51:37.745 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:51:37.746 INFO:teuthology.orchestra.run.smithi121.stdout:42949672967 2024-09-10T15:51:37.746 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:51:38.084 INFO:teuthology.orchestra.run.smithi121.stdout:42949672965 2024-09-10T15:51:38.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672966 got 42949672965 for osd.0 2024-09-10T15:51:38.103 INFO:teuthology.orchestra.run.smithi121.stdout:34359738372 2024-09-10T15:51:38.115 INFO:teuthology.orchestra.run.smithi121.stdout:42949672966 2024-09-10T15:51:38.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738372 for osd.2 2024-09-10T15:51:38.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672967 got 42949672966 for osd.1 2024-09-10T15:51:39.104 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:51:39.117 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:51:39.129 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:51:39.417 INFO:teuthology.orchestra.run.smithi121.stdout:42949672966 2024-09-10T15:51:39.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672966 got 42949672966 for osd.0 2024-09-10T15:51:39.434 DEBUG:teuthology.parallel:result is None 2024-09-10T15:51:39.465 INFO:teuthology.orchestra.run.smithi121.stdout:42949672967 2024-09-10T15:51:39.474 INFO:teuthology.orchestra.run.smithi121.stdout:34359738373 2024-09-10T15:51:39.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672967 got 42949672967 for osd.1 2024-09-10T15:51:39.481 DEBUG:teuthology.parallel:result is None 2024-09-10T15:51:39.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738373 for osd.2 2024-09-10T15:51:39.487 DEBUG:teuthology.parallel:result is None 2024-09-10T15:51:39.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-10T15:51:39.488 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:51:39.807 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:39.807 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:51:39.819 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-10T15:51:38.844135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81464,"kb_used_data":392,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034056,"statfs":{"total":289910292480,"available":289826873344,"internally_reserved":0,"allocated":401408,"data_stored":92210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.427710"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.634342+0000","last_change":"2024-09-10T15:51:32.634606+0000","last_active":"2024-09-10T15:51:32.634342+0000","last_peered":"2024-09-10T15:51:32.634342+0000","last_clean":"2024-09-10T15:51:32.634342+0000","last_became_active":"2024-09-10T15:51:30.614886+0000","last_became_peered":"2024-09-10T15:51:30.614886+0000","last_unstale":"2024-09-10T15:51:32.634342+0000","last_undegraded":"2024-09-10T15:51:32.634342+0000","last_fullsized":"2024-09-10T15:51:32.634342+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:28.735655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.633804+0000","last_change":"2024-09-10T15:51:32.633999+0000","last_active":"2024-09-10T15:51:32.633804+0000","last_peered":"2024-09-10T15:51:32.633804+0000","last_clean":"2024-09-10T15:51:32.633804+0000","last_became_active":"2024-09-10T15:51:30.613428+0000","last_became_peered":"2024-09-10T15:51:30.613428+0000","last_unstale":"2024-09-10T15:51:32.633804+0000","last_undegraded":"2024-09-10T15:51:32.633804+0000","last_fullsized":"2024-09-10T15:51:32.633804+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:18.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.623172+0000","last_change":"2024-09-10T15:51:32.623172+0000","last_active":"2024-09-10T15:51:32.623172+0000","last_peered":"2024-09-10T15:51:32.623172+0000","last_clean":"2024-09-10T15:51:32.623172+0000","last_became_active":"2024-09-10T15:51:30.613434+0000","last_became_peered":"2024-09-10T15:51:30.613434+0000","last_unstale":"2024-09-10T15:51:32.623172+0000","last_undegraded":"2024-09-10T15:51:32.623172+0000","last_fullsized":"2024-09-10T15:51:32.623172+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:58.317152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.633758+0000","last_change":"2024-09-10T15:51:32.634015+0000","last_active":"2024-09-10T15:51:32.633758+0000","last_peered":"2024-09-10T15:51:32.633758+0000","last_clean":"2024-09-10T15:51:32.633758+0000","last_became_active":"2024-09-10T15:51:30.613543+0000","last_became_peered":"2024-09-10T15:51:30.613543+0000","last_unstale":"2024-09-10T15:51:32.633758+0000","last_undegraded":"2024-09-10T15:51:32.633758+0000","last_fullsized":"2024-09-10T15:51:32.633758+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:26.573051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"13'1","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.634061+0000","last_change":"2024-09-10T15:51:32.634287+0000","last_active":"2024-09-10T15:51:32.634061+0000","last_peered":"2024-09-10T15:51:32.634061+0000","last_clean":"2024-09-10T15:51:32.634061+0000","last_became_active":"2024-09-10T15:51:30.616135+0000","last_became_peered":"2024-09-10T15:51:30.616135+0000","last_unstale":"2024-09-10T15:51:32.634061+0000","last_undegraded":"2024-09-10T15:51:32.634061+0000","last_fullsized":"2024-09-10T15:51:32.634061+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:18.351885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"15'2","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.629990+0000","last_change":"2024-09-10T15:51:32.629990+0000","last_active":"2024-09-10T15:51:32.629990+0000","last_peered":"2024-09-10T15:51:32.629990+0000","last_clean":"2024-09-10T15:51:32.629990+0000","last_became_active":"2024-09-10T15:51:30.617003+0000","last_became_peered":"2024-09-10T15:51:30.617003+0000","last_unstale":"2024-09-10T15:51:32.629990+0000","last_undegraded":"2024-09-10T15:51:32.629990+0000","last_fullsized":"2024-09-10T15:51:32.629990+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:50.477908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.623176+0000","last_change":"2024-09-10T15:51:32.623176+0000","last_active":"2024-09-10T15:51:32.623176+0000","last_peered":"2024-09-10T15:51:32.623176+0000","last_clean":"2024-09-10T15:51:32.623176+0000","last_became_active":"2024-09-10T15:51:30.613566+0000","last_became_peered":"2024-09-10T15:51:30.613566+0000","last_unstale":"2024-09-10T15:51:32.623176+0000","last_undegraded":"2024-09-10T15:51:32.623176+0000","last_fullsized":"2024-09-10T15:51:32.623176+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:05.565044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.634106+0000","last_change":"2024-09-10T15:51:32.634333+0000","last_active":"2024-09-10T15:51:32.634106+0000","last_peered":"2024-09-10T15:51:32.634106+0000","last_clean":"2024-09-10T15:51:32.634106+0000","last_became_active":"2024-09-10T15:51:30.613634+0000","last_became_peered":"2024-09-10T15:51:30.613634+0000","last_unstale":"2024-09-10T15:51:32.634106+0000","last_undegraded":"2024-09-10T15:51:32.634106+0000","last_fullsized":"2024-09-10T15:51:32.634106+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:50.070951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672967,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":135168,"data_stored":30743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672966,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":135168,"data_stored":30743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":131072,"data_stored":30724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:51:39.820 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:51:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:40.101 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:51:40.113 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-10T15:51:38.844135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81464,"kb_used_data":392,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034056,"statfs":{"total":289910292480,"available":289826873344,"internally_reserved":0,"allocated":401408,"data_stored":92210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.427710"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.634342+0000","last_change":"2024-09-10T15:51:32.634606+0000","last_active":"2024-09-10T15:51:32.634342+0000","last_peered":"2024-09-10T15:51:32.634342+0000","last_clean":"2024-09-10T15:51:32.634342+0000","last_became_active":"2024-09-10T15:51:30.614886+0000","last_became_peered":"2024-09-10T15:51:30.614886+0000","last_unstale":"2024-09-10T15:51:32.634342+0000","last_undegraded":"2024-09-10T15:51:32.634342+0000","last_fullsized":"2024-09-10T15:51:32.634342+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:28.735655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.633804+0000","last_change":"2024-09-10T15:51:32.633999+0000","last_active":"2024-09-10T15:51:32.633804+0000","last_peered":"2024-09-10T15:51:32.633804+0000","last_clean":"2024-09-10T15:51:32.633804+0000","last_became_active":"2024-09-10T15:51:30.613428+0000","last_became_peered":"2024-09-10T15:51:30.613428+0000","last_unstale":"2024-09-10T15:51:32.633804+0000","last_undegraded":"2024-09-10T15:51:32.633804+0000","last_fullsized":"2024-09-10T15:51:32.633804+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:18.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.623172+0000","last_change":"2024-09-10T15:51:32.623172+0000","last_active":"2024-09-10T15:51:32.623172+0000","last_peered":"2024-09-10T15:51:32.623172+0000","last_clean":"2024-09-10T15:51:32.623172+0000","last_became_active":"2024-09-10T15:51:30.613434+0000","last_became_peered":"2024-09-10T15:51:30.613434+0000","last_unstale":"2024-09-10T15:51:32.623172+0000","last_undegraded":"2024-09-10T15:51:32.623172+0000","last_fullsized":"2024-09-10T15:51:32.623172+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:58.317152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.633758+0000","last_change":"2024-09-10T15:51:32.634015+0000","last_active":"2024-09-10T15:51:32.633758+0000","last_peered":"2024-09-10T15:51:32.633758+0000","last_clean":"2024-09-10T15:51:32.633758+0000","last_became_active":"2024-09-10T15:51:30.613543+0000","last_became_peered":"2024-09-10T15:51:30.613543+0000","last_unstale":"2024-09-10T15:51:32.633758+0000","last_undegraded":"2024-09-10T15:51:32.633758+0000","last_fullsized":"2024-09-10T15:51:32.633758+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:26.573051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"13'1","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.634061+0000","last_change":"2024-09-10T15:51:32.634287+0000","last_active":"2024-09-10T15:51:32.634061+0000","last_peered":"2024-09-10T15:51:32.634061+0000","last_clean":"2024-09-10T15:51:32.634061+0000","last_became_active":"2024-09-10T15:51:30.616135+0000","last_became_peered":"2024-09-10T15:51:30.616135+0000","last_unstale":"2024-09-10T15:51:32.634061+0000","last_undegraded":"2024-09-10T15:51:32.634061+0000","last_fullsized":"2024-09-10T15:51:32.634061+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:18.351885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"15'2","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.629990+0000","last_change":"2024-09-10T15:51:32.629990+0000","last_active":"2024-09-10T15:51:32.629990+0000","last_peered":"2024-09-10T15:51:32.629990+0000","last_clean":"2024-09-10T15:51:32.629990+0000","last_became_active":"2024-09-10T15:51:30.617003+0000","last_became_peered":"2024-09-10T15:51:30.617003+0000","last_unstale":"2024-09-10T15:51:32.629990+0000","last_undegraded":"2024-09-10T15:51:32.629990+0000","last_fullsized":"2024-09-10T15:51:32.629990+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:50.477908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.623176+0000","last_change":"2024-09-10T15:51:32.623176+0000","last_active":"2024-09-10T15:51:32.623176+0000","last_peered":"2024-09-10T15:51:32.623176+0000","last_clean":"2024-09-10T15:51:32.623176+0000","last_became_active":"2024-09-10T15:51:30.613566+0000","last_became_peered":"2024-09-10T15:51:30.613566+0000","last_unstale":"2024-09-10T15:51:32.623176+0000","last_undegraded":"2024-09-10T15:51:32.623176+0000","last_fullsized":"2024-09-10T15:51:32.623176+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:05.565044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-10T15:51:32.634106+0000","last_change":"2024-09-10T15:51:32.634333+0000","last_active":"2024-09-10T15:51:32.634106+0000","last_peered":"2024-09-10T15:51:32.634106+0000","last_clean":"2024-09-10T15:51:32.634106+0000","last_became_active":"2024-09-10T15:51:30.613634+0000","last_became_peered":"2024-09-10T15:51:30.613634+0000","last_unstale":"2024-09-10T15:51:32.634106+0000","last_undegraded":"2024-09-10T15:51:32.634106+0000","last_fullsized":"2024-09-10T15:51:32.634106+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:50.070951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672967,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":135168,"data_stored":30743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672966,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":135168,"data_stored":30743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":131072,"data_stored":30724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:51:40.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-10T15:51:40.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:creating pool_name unfounddel_pool 2024-09-10T15:51:40.114 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unfounddel_pool 16 2024-09-10T15:51:40.859 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'unfounddel_pool' created 2024-09-10T15:51:40.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unfounddel_pool rados --yes-i-really-mean-it || true 2024-09-10T15:51:41.857 INFO:teuthology.orchestra.run.smithi121.stderr:enabled application 'rados' on pool 'unfounddel_pool' 2024-09-10T15:51:42.881 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T15:51:42.881 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T15:51:42.911 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T15:51:42.911 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-10T15:51:43.253 INFO:tasks.ceph.mon.c.smithi121.stderr:2024-09-10T15:51:43.251+0000 7fb1890bc640 -1 mon.c@2(leader).osd e19 definitely_dead 0 2024-09-10T15:51:43.879 INFO:teuthology.orchestra.run.smithi121.stderr:marked down osd.2. 2024-09-10T15:51:43.895 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-10T15:51:44.188 INFO:tasks.ceph.mon.c.smithi121.stderr:2024-09-10T15:51:44.186+0000 7fb1890bc640 -1 mon.c@2(leader).osd e20 definitely_dead 0 2024-09-10T15:51:44.891 INFO:teuthology.orchestra.run.smithi121.stderr:marked out osd.2. 2024-09-10T15:51:44.906 INFO:tasks.util.rados:rados -p unfounddel_pool put dummy /etc/fstab 2024-09-10T15:51:44.906 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put dummy /etc/fstab 2024-09-10T15:51:44.955 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:51:44.956 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T15:51:45.087 INFO:teuthology.orchestra.run.smithi121.stdout:42949672968 2024-09-10T15:51:45.087 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:51:45.088 INFO:teuthology.orchestra.run.smithi121.stdout:42949672969 2024-09-10T15:51:45.088 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:51:45.420 INFO:teuthology.orchestra.run.smithi121.stdout:42949672967 2024-09-10T15:51:45.420 INFO:teuthology.orchestra.run.smithi121.stdout:42949672968 2024-09-10T15:51:45.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672968 got 42949672967 for osd.0 2024-09-10T15:51:45.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672969 got 42949672968 for osd.1 2024-09-10T15:51:46.435 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:51:46.436 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:51:46.727 INFO:teuthology.orchestra.run.smithi121.stdout:42949672968 2024-09-10T15:51:46.728 INFO:teuthology.orchestra.run.smithi121.stdout:42949672967 2024-09-10T15:51:46.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672969 got 42949672968 for osd.1 2024-09-10T15:51:46.741 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672968 got 42949672967 for osd.0 2024-09-10T15:51:47.741 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:51:47.742 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:51:48.044 INFO:teuthology.orchestra.run.smithi121.stdout:42949672969 2024-09-10T15:51:48.045 INFO:teuthology.orchestra.run.smithi121.stdout:42949672970 2024-09-10T15:51:48.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672969 got 42949672970 for osd.1 2024-09-10T15:51:48.059 DEBUG:teuthology.parallel:result is None 2024-09-10T15:51:48.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672968 got 42949672969 for osd.0 2024-09-10T15:51:48.059 DEBUG:teuthology.parallel:result is None 2024-09-10T15:51:48.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-10T15:51:48.060 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:51:48.338 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:48.338 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:51:48.350 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-10T15:51:46.418437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54404,"kb_used_data":356,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689276,"statfs":{"total":193273528320,"available":193217818624,"internally_reserved":0,"allocated":364544,"data_stored":93882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000566"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.895083+0000","last_change":"2024-09-10T15:51:41.867143+0000","last_active":"2024-09-10T15:51:44.895083+0000","last_peered":"2024-09-10T15:51:44.895083+0000","last_clean":"2024-09-10T15:51:44.895083+0000","last_became_active":"2024-09-10T15:51:41.866879+0000","last_became_peered":"2024-09-10T15:51:41.866879+0000","last_unstale":"2024-09-10T15:51:44.895083+0000","last_undegraded":"2024-09-10T15:51:44.895083+0000","last_fullsized":"2024-09-10T15:51:44.895083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:09.819843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895509+0000","last_change":"2024-09-10T15:51:44.308511+0000","last_active":"2024-09-10T15:51:44.308511+0000","last_peered":"2024-09-10T15:51:41.869848+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.869661+0000","last_became_peered":"2024-09-10T15:51:41.869661+0000","last_unstale":"2024-09-10T15:51:44.895509+0000","last_undegraded":"2024-09-10T15:51:44.895509+0000","last_fullsized":"2024-09-10T15:51:44.895509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889769+0000","last_change":"2024-09-10T15:51:43.883276+0000","last_active":"2024-09-10T15:51:43.883276+0000","last_peered":"2024-09-10T15:51:43.882835+0000","last_clean":"2024-09-10T15:51:43.882835+0000","last_became_active":"2024-09-10T15:51:41.868597+0000","last_became_peered":"2024-09-10T15:51:41.868597+0000","last_unstale":"2024-09-10T15:51:44.889769+0000","last_undegraded":"2024-09-10T15:51:44.889769+0000","last_fullsized":"2024-09-10T15:51:44.889769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895589+0000","last_change":"2024-09-10T15:51:44.308231+0000","last_active":"2024-09-10T15:51:44.308231+0000","last_peered":"2024-09-10T15:51:41.868682+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.868468+0000","last_became_peered":"2024-09-10T15:51:41.868468+0000","last_unstale":"2024-09-10T15:51:44.895589+0000","last_undegraded":"2024-09-10T15:51:44.895589+0000","last_fullsized":"2024-09-10T15:51:44.895589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895105+0000","last_change":"2024-09-10T15:51:44.308258+0000","last_active":"2024-09-10T15:51:44.308258+0000","last_peered":"2024-09-10T15:51:42.870041+0000","last_clean":"2024-09-10T15:51:42.870041+0000","last_became_active":"2024-09-10T15:51:41.865185+0000","last_became_peered":"2024-09-10T15:51:41.865185+0000","last_unstale":"2024-09-10T15:51:44.895105+0000","last_undegraded":"2024-09-10T15:51:44.895105+0000","last_fullsized":"2024-09-10T15:51:44.895105+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894060+0000","last_change":"2024-09-10T15:51:41.869027+0000","last_active":"2024-09-10T15:51:44.894060+0000","last_peered":"2024-09-10T15:51:44.894060+0000","last_clean":"2024-09-10T15:51:44.894060+0000","last_became_active":"2024-09-10T15:51:41.868747+0000","last_became_peered":"2024-09-10T15:51:41.868747+0000","last_unstale":"2024-09-10T15:51:44.894060+0000","last_undegraded":"2024-09-10T15:51:44.894060+0000","last_fullsized":"2024-09-10T15:51:44.894060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:38.255269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889447+0000","last_change":"2024-09-10T15:51:41.869751+0000","last_active":"2024-09-10T15:51:44.889447+0000","last_peered":"2024-09-10T15:51:44.889447+0000","last_clean":"2024-09-10T15:51:44.889447+0000","last_became_active":"2024-09-10T15:51:41.869474+0000","last_became_peered":"2024-09-10T15:51:41.869474+0000","last_unstale":"2024-09-10T15:51:44.889447+0000","last_undegraded":"2024-09-10T15:51:44.889447+0000","last_fullsized":"2024-09-10T15:51:44.889447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:54.279052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889377+0000","last_change":"2024-09-10T15:51:44.308154+0000","last_active":"2024-09-10T15:51:44.308154+0000","last_peered":"2024-09-10T15:51:42.870214+0000","last_clean":"2024-09-10T15:51:42.870214+0000","last_became_active":"2024-09-10T15:51:41.864310+0000","last_became_peered":"2024-09-10T15:51:41.864310+0000","last_unstale":"2024-09-10T15:51:44.889377+0000","last_undegraded":"2024-09-10T15:51:44.889377+0000","last_fullsized":"2024-09-10T15:51:44.889377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889046+0000","last_change":"2024-09-10T15:51:32.634333+0000","last_active":"2024-09-10T15:51:44.889046+0000","last_peered":"2024-09-10T15:51:44.889046+0000","last_clean":"2024-09-10T15:51:44.889046+0000","last_became_active":"2024-09-10T15:51:30.613634+0000","last_became_peered":"2024-09-10T15:51:30.613634+0000","last_unstale":"2024-09-10T15:51:44.889046+0000","last_undegraded":"2024-09-10T15:51:44.889046+0000","last_fullsized":"2024-09-10T15:51:44.889046+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:50.070951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889787+0000","last_change":"2024-09-10T15:51:43.883355+0000","last_active":"2024-09-10T15:51:43.883355+0000","last_peered":"2024-09-10T15:51:43.882616+0000","last_clean":"2024-09-10T15:51:43.882616+0000","last_became_active":"2024-09-10T15:51:41.868604+0000","last_became_peered":"2024-09-10T15:51:41.868604+0000","last_unstale":"2024-09-10T15:51:44.889787+0000","last_undegraded":"2024-09-10T15:51:44.889787+0000","last_fullsized":"2024-09-10T15:51:44.889787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.894541+0000","last_change":"2024-09-10T15:51:44.308168+0000","last_active":"2024-09-10T15:51:44.308168+0000","last_peered":"2024-09-10T15:51:34.000525+0000","last_clean":"2024-09-10T15:51:34.000525+0000","last_became_active":"2024-09-10T15:51:30.613566+0000","last_became_peered":"2024-09-10T15:51:30.613566+0000","last_unstale":"2024-09-10T15:51:44.894541+0000","last_undegraded":"2024-09-10T15:51:44.894541+0000","last_fullsized":"2024-09-10T15:51:44.894541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894872+0000","last_change":"2024-09-10T15:51:41.867323+0000","last_active":"2024-09-10T15:51:44.894872+0000","last_peered":"2024-09-10T15:51:44.894872+0000","last_clean":"2024-09-10T15:51:44.894872+0000","last_became_active":"2024-09-10T15:51:41.867050+0000","last_became_peered":"2024-09-10T15:51:41.867050+0000","last_unstale":"2024-09-10T15:51:44.894872+0000","last_undegraded":"2024-09-10T15:51:44.894872+0000","last_fullsized":"2024-09-10T15:51:44.894872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:58.326451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894880+0000","last_change":"2024-09-10T15:51:32.629990+0000","last_active":"2024-09-10T15:51:44.894880+0000","last_peered":"2024-09-10T15:51:44.894880+0000","last_clean":"2024-09-10T15:51:44.894880+0000","last_became_active":"2024-09-10T15:51:30.617003+0000","last_became_peered":"2024-09-10T15:51:30.617003+0000","last_unstale":"2024-09-10T15:51:44.894880+0000","last_undegraded":"2024-09-10T15:51:44.894880+0000","last_fullsized":"2024-09-10T15:51:44.894880+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:50.477908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889391+0000","last_change":"2024-09-10T15:51:43.883490+0000","last_active":"2024-09-10T15:51:43.883490+0000","last_peered":"2024-09-10T15:51:41.867177+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.866984+0000","last_became_peered":"2024-09-10T15:51:41.866984+0000","last_unstale":"2024-09-10T15:51:44.889391+0000","last_undegraded":"2024-09-10T15:51:44.889391+0000","last_fullsized":"2024-09-10T15:51:44.889391+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":30,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:51:43.883454+0000","last_change":"2024-09-10T15:51:43.883454+0000","last_active":"2024-09-10T15:51:43.883454+0000","last_peered":"2024-09-10T15:51:43.882793+0000","last_clean":"2024-09-10T15:51:43.882793+0000","last_became_active":"2024-09-10T15:51:30.616135+0000","last_became_peered":"2024-09-10T15:51:30.616135+0000","last_unstale":"2024-09-10T15:51:43.883454+0000","last_undegraded":"2024-09-10T15:51:43.883454+0000","last_fullsized":"2024-09-10T15:51:43.883454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889380+0000","last_change":"2024-09-10T15:51:43.883632+0000","last_active":"2024-09-10T15:51:43.883632+0000","last_peered":"2024-09-10T15:51:41.867187+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.866987+0000","last_became_peered":"2024-09-10T15:51:41.866987+0000","last_unstale":"2024-09-10T15:51:44.889380+0000","last_undegraded":"2024-09-10T15:51:44.889380+0000","last_fullsized":"2024-09-10T15:51:44.889380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894260+0000","last_change":"2024-09-10T15:51:41.869802+0000","last_active":"2024-09-10T15:51:44.894260+0000","last_peered":"2024-09-10T15:51:44.894260+0000","last_clean":"2024-09-10T15:51:44.894260+0000","last_became_active":"2024-09-10T15:51:41.869551+0000","last_became_peered":"2024-09-10T15:51:41.869551+0000","last_unstale":"2024-09-10T15:51:44.894260+0000","last_undegraded":"2024-09-10T15:51:44.894260+0000","last_fullsized":"2024-09-10T15:51:44.894260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:49.796567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889352+0000","last_change":"2024-09-10T15:51:32.634015+0000","last_active":"2024-09-10T15:51:44.889352+0000","last_peered":"2024-09-10T15:51:44.889352+0000","last_clean":"2024-09-10T15:51:44.889352+0000","last_became_active":"2024-09-10T15:51:30.613543+0000","last_became_peered":"2024-09-10T15:51:30.613543+0000","last_unstale":"2024-09-10T15:51:44.889352+0000","last_undegraded":"2024-09-10T15:51:44.889352+0000","last_fullsized":"2024-09-10T15:51:44.889352+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:26.573051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889276+0000","last_change":"2024-09-10T15:51:41.866039+0000","last_active":"2024-09-10T15:51:44.889276+0000","last_peered":"2024-09-10T15:51:44.889276+0000","last_clean":"2024-09-10T15:51:44.889276+0000","last_became_active":"2024-09-10T15:51:41.865716+0000","last_became_peered":"2024-09-10T15:51:41.865716+0000","last_unstale":"2024-09-10T15:51:44.889276+0000","last_undegraded":"2024-09-10T15:51:44.889276+0000","last_fullsized":"2024-09-10T15:51:44.889276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:29.481067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895527+0000","last_change":"2024-09-10T15:51:44.308400+0000","last_active":"2024-09-10T15:51:44.308400+0000","last_peered":"2024-09-10T15:51:34.000546+0000","last_clean":"2024-09-10T15:51:34.000546+0000","last_became_active":"2024-09-10T15:51:30.613434+0000","last_became_peered":"2024-09-10T15:51:30.613434+0000","last_unstale":"2024-09-10T15:51:44.895527+0000","last_undegraded":"2024-09-10T15:51:44.895527+0000","last_fullsized":"2024-09-10T15:51:44.895527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894150+0000","last_change":"2024-09-10T15:51:41.870267+0000","last_active":"2024-09-10T15:51:44.894150+0000","last_peered":"2024-09-10T15:51:44.894150+0000","last_clean":"2024-09-10T15:51:44.894150+0000","last_became_active":"2024-09-10T15:51:41.868802+0000","last_became_peered":"2024-09-10T15:51:41.868802+0000","last_unstale":"2024-09-10T15:51:44.894150+0000","last_undegraded":"2024-09-10T15:51:44.894150+0000","last_fullsized":"2024-09-10T15:51:44.894150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:14.820633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889334+0000","last_change":"2024-09-10T15:51:32.633999+0000","last_active":"2024-09-10T15:51:44.889334+0000","last_peered":"2024-09-10T15:51:44.889334+0000","last_clean":"2024-09-10T15:51:44.889334+0000","last_became_active":"2024-09-10T15:51:30.613428+0000","last_became_peered":"2024-09-10T15:51:30.613428+0000","last_unstale":"2024-09-10T15:51:44.889334+0000","last_undegraded":"2024-09-10T15:51:44.889334+0000","last_fullsized":"2024-09-10T15:51:44.889334+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:18.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:51:43.884058+0000","last_change":"2024-09-10T15:51:43.884058+0000","last_active":"2024-09-10T15:51:43.884058+0000","last_peered":"2024-09-10T15:51:43.883639+0000","last_clean":"2024-09-10T15:51:43.883639+0000","last_became_active":"2024-09-10T15:51:30.614886+0000","last_became_peered":"2024-09-10T15:51:30.614886+0000","last_unstale":"2024-09-10T15:51:43.884058+0000","last_undegraded":"2024-09-10T15:51:43.884058+0000","last_fullsized":"2024-09-10T15:51:43.884058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.952468+0000","last_change":"2024-09-10T15:51:41.869011+0000","last_active":"2024-09-10T15:51:44.952468+0000","last_peered":"2024-09-10T15:51:44.952468+0000","last_clean":"2024-09-10T15:51:44.952468+0000","last_became_active":"2024-09-10T15:51:41.868056+0000","last_became_peered":"2024-09-10T15:51:41.868056+0000","last_unstale":"2024-09-10T15:51:44.952468+0000","last_undegraded":"2024-09-10T15:51:44.952468+0000","last_fullsized":"2024-09-10T15:51:44.952468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:04.162191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672970,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":45312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672969,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":188416,"data_stored":48570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:51:48.351 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:51:48.634 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:48.635 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:51:48.646 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-10T15:51:48.418788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54404,"kb_used_data":356,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689276,"statfs":{"total":193273528320,"available":193217818624,"internally_reserved":0,"allocated":364544,"data_stored":93882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.550124"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.895083+0000","last_change":"2024-09-10T15:51:41.867143+0000","last_active":"2024-09-10T15:51:44.895083+0000","last_peered":"2024-09-10T15:51:44.895083+0000","last_clean":"2024-09-10T15:51:44.895083+0000","last_became_active":"2024-09-10T15:51:41.866879+0000","last_became_peered":"2024-09-10T15:51:41.866879+0000","last_unstale":"2024-09-10T15:51:44.895083+0000","last_undegraded":"2024-09-10T15:51:44.895083+0000","last_fullsized":"2024-09-10T15:51:44.895083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:09.819843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895509+0000","last_change":"2024-09-10T15:51:44.308511+0000","last_active":"2024-09-10T15:51:44.308511+0000","last_peered":"2024-09-10T15:51:41.869848+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.869661+0000","last_became_peered":"2024-09-10T15:51:41.869661+0000","last_unstale":"2024-09-10T15:51:44.895509+0000","last_undegraded":"2024-09-10T15:51:44.895509+0000","last_fullsized":"2024-09-10T15:51:44.895509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889769+0000","last_change":"2024-09-10T15:51:43.883276+0000","last_active":"2024-09-10T15:51:43.883276+0000","last_peered":"2024-09-10T15:51:43.882835+0000","last_clean":"2024-09-10T15:51:43.882835+0000","last_became_active":"2024-09-10T15:51:41.868597+0000","last_became_peered":"2024-09-10T15:51:41.868597+0000","last_unstale":"2024-09-10T15:51:44.889769+0000","last_undegraded":"2024-09-10T15:51:44.889769+0000","last_fullsized":"2024-09-10T15:51:44.889769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895589+0000","last_change":"2024-09-10T15:51:44.308231+0000","last_active":"2024-09-10T15:51:44.308231+0000","last_peered":"2024-09-10T15:51:41.868682+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.868468+0000","last_became_peered":"2024-09-10T15:51:41.868468+0000","last_unstale":"2024-09-10T15:51:44.895589+0000","last_undegraded":"2024-09-10T15:51:44.895589+0000","last_fullsized":"2024-09-10T15:51:44.895589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895105+0000","last_change":"2024-09-10T15:51:44.308258+0000","last_active":"2024-09-10T15:51:44.308258+0000","last_peered":"2024-09-10T15:51:42.870041+0000","last_clean":"2024-09-10T15:51:42.870041+0000","last_became_active":"2024-09-10T15:51:41.865185+0000","last_became_peered":"2024-09-10T15:51:41.865185+0000","last_unstale":"2024-09-10T15:51:44.895105+0000","last_undegraded":"2024-09-10T15:51:44.895105+0000","last_fullsized":"2024-09-10T15:51:44.895105+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894060+0000","last_change":"2024-09-10T15:51:41.869027+0000","last_active":"2024-09-10T15:51:44.894060+0000","last_peered":"2024-09-10T15:51:44.894060+0000","last_clean":"2024-09-10T15:51:44.894060+0000","last_became_active":"2024-09-10T15:51:41.868747+0000","last_became_peered":"2024-09-10T15:51:41.868747+0000","last_unstale":"2024-09-10T15:51:44.894060+0000","last_undegraded":"2024-09-10T15:51:44.894060+0000","last_fullsized":"2024-09-10T15:51:44.894060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:38.255269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889447+0000","last_change":"2024-09-10T15:51:41.869751+0000","last_active":"2024-09-10T15:51:44.889447+0000","last_peered":"2024-09-10T15:51:44.889447+0000","last_clean":"2024-09-10T15:51:44.889447+0000","last_became_active":"2024-09-10T15:51:41.869474+0000","last_became_peered":"2024-09-10T15:51:41.869474+0000","last_unstale":"2024-09-10T15:51:44.889447+0000","last_undegraded":"2024-09-10T15:51:44.889447+0000","last_fullsized":"2024-09-10T15:51:44.889447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:54.279052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889377+0000","last_change":"2024-09-10T15:51:44.308154+0000","last_active":"2024-09-10T15:51:44.308154+0000","last_peered":"2024-09-10T15:51:42.870214+0000","last_clean":"2024-09-10T15:51:42.870214+0000","last_became_active":"2024-09-10T15:51:41.864310+0000","last_became_peered":"2024-09-10T15:51:41.864310+0000","last_unstale":"2024-09-10T15:51:44.889377+0000","last_undegraded":"2024-09-10T15:51:44.889377+0000","last_fullsized":"2024-09-10T15:51:44.889377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889046+0000","last_change":"2024-09-10T15:51:32.634333+0000","last_active":"2024-09-10T15:51:44.889046+0000","last_peered":"2024-09-10T15:51:44.889046+0000","last_clean":"2024-09-10T15:51:44.889046+0000","last_became_active":"2024-09-10T15:51:30.613634+0000","last_became_peered":"2024-09-10T15:51:30.613634+0000","last_unstale":"2024-09-10T15:51:44.889046+0000","last_undegraded":"2024-09-10T15:51:44.889046+0000","last_fullsized":"2024-09-10T15:51:44.889046+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:50.070951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889787+0000","last_change":"2024-09-10T15:51:43.883355+0000","last_active":"2024-09-10T15:51:43.883355+0000","last_peered":"2024-09-10T15:51:43.882616+0000","last_clean":"2024-09-10T15:51:43.882616+0000","last_became_active":"2024-09-10T15:51:41.868604+0000","last_became_peered":"2024-09-10T15:51:41.868604+0000","last_unstale":"2024-09-10T15:51:44.889787+0000","last_undegraded":"2024-09-10T15:51:44.889787+0000","last_fullsized":"2024-09-10T15:51:44.889787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.894541+0000","last_change":"2024-09-10T15:51:44.308168+0000","last_active":"2024-09-10T15:51:44.308168+0000","last_peered":"2024-09-10T15:51:34.000525+0000","last_clean":"2024-09-10T15:51:34.000525+0000","last_became_active":"2024-09-10T15:51:30.613566+0000","last_became_peered":"2024-09-10T15:51:30.613566+0000","last_unstale":"2024-09-10T15:51:44.894541+0000","last_undegraded":"2024-09-10T15:51:44.894541+0000","last_fullsized":"2024-09-10T15:51:44.894541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894872+0000","last_change":"2024-09-10T15:51:41.867323+0000","last_active":"2024-09-10T15:51:44.894872+0000","last_peered":"2024-09-10T15:51:44.894872+0000","last_clean":"2024-09-10T15:51:44.894872+0000","last_became_active":"2024-09-10T15:51:41.867050+0000","last_became_peered":"2024-09-10T15:51:41.867050+0000","last_unstale":"2024-09-10T15:51:44.894872+0000","last_undegraded":"2024-09-10T15:51:44.894872+0000","last_fullsized":"2024-09-10T15:51:44.894872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:58.326451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894880+0000","last_change":"2024-09-10T15:51:32.629990+0000","last_active":"2024-09-10T15:51:44.894880+0000","last_peered":"2024-09-10T15:51:44.894880+0000","last_clean":"2024-09-10T15:51:44.894880+0000","last_became_active":"2024-09-10T15:51:30.617003+0000","last_became_peered":"2024-09-10T15:51:30.617003+0000","last_unstale":"2024-09-10T15:51:44.894880+0000","last_undegraded":"2024-09-10T15:51:44.894880+0000","last_fullsized":"2024-09-10T15:51:44.894880+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:50.477908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889391+0000","last_change":"2024-09-10T15:51:43.883490+0000","last_active":"2024-09-10T15:51:43.883490+0000","last_peered":"2024-09-10T15:51:41.867177+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.866984+0000","last_became_peered":"2024-09-10T15:51:41.866984+0000","last_unstale":"2024-09-10T15:51:44.889391+0000","last_undegraded":"2024-09-10T15:51:44.889391+0000","last_fullsized":"2024-09-10T15:51:44.889391+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":30,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:51:43.883454+0000","last_change":"2024-09-10T15:51:43.883454+0000","last_active":"2024-09-10T15:51:43.883454+0000","last_peered":"2024-09-10T15:51:43.882793+0000","last_clean":"2024-09-10T15:51:43.882793+0000","last_became_active":"2024-09-10T15:51:30.616135+0000","last_became_peered":"2024-09-10T15:51:30.616135+0000","last_unstale":"2024-09-10T15:51:43.883454+0000","last_undegraded":"2024-09-10T15:51:43.883454+0000","last_fullsized":"2024-09-10T15:51:43.883454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889380+0000","last_change":"2024-09-10T15:51:43.883632+0000","last_active":"2024-09-10T15:51:43.883632+0000","last_peered":"2024-09-10T15:51:41.867187+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.866987+0000","last_became_peered":"2024-09-10T15:51:41.866987+0000","last_unstale":"2024-09-10T15:51:44.889380+0000","last_undegraded":"2024-09-10T15:51:44.889380+0000","last_fullsized":"2024-09-10T15:51:44.889380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894260+0000","last_change":"2024-09-10T15:51:41.869802+0000","last_active":"2024-09-10T15:51:44.894260+0000","last_peered":"2024-09-10T15:51:44.894260+0000","last_clean":"2024-09-10T15:51:44.894260+0000","last_became_active":"2024-09-10T15:51:41.869551+0000","last_became_peered":"2024-09-10T15:51:41.869551+0000","last_unstale":"2024-09-10T15:51:44.894260+0000","last_undegraded":"2024-09-10T15:51:44.894260+0000","last_fullsized":"2024-09-10T15:51:44.894260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:49.796567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889352+0000","last_change":"2024-09-10T15:51:32.634015+0000","last_active":"2024-09-10T15:51:44.889352+0000","last_peered":"2024-09-10T15:51:44.889352+0000","last_clean":"2024-09-10T15:51:44.889352+0000","last_became_active":"2024-09-10T15:51:30.613543+0000","last_became_peered":"2024-09-10T15:51:30.613543+0000","last_unstale":"2024-09-10T15:51:44.889352+0000","last_undegraded":"2024-09-10T15:51:44.889352+0000","last_fullsized":"2024-09-10T15:51:44.889352+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:26.573051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889276+0000","last_change":"2024-09-10T15:51:41.866039+0000","last_active":"2024-09-10T15:51:44.889276+0000","last_peered":"2024-09-10T15:51:44.889276+0000","last_clean":"2024-09-10T15:51:44.889276+0000","last_became_active":"2024-09-10T15:51:41.865716+0000","last_became_peered":"2024-09-10T15:51:41.865716+0000","last_unstale":"2024-09-10T15:51:44.889276+0000","last_undegraded":"2024-09-10T15:51:44.889276+0000","last_fullsized":"2024-09-10T15:51:44.889276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:29.481067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895527+0000","last_change":"2024-09-10T15:51:44.308400+0000","last_active":"2024-09-10T15:51:44.308400+0000","last_peered":"2024-09-10T15:51:34.000546+0000","last_clean":"2024-09-10T15:51:34.000546+0000","last_became_active":"2024-09-10T15:51:30.613434+0000","last_became_peered":"2024-09-10T15:51:30.613434+0000","last_unstale":"2024-09-10T15:51:44.895527+0000","last_undegraded":"2024-09-10T15:51:44.895527+0000","last_fullsized":"2024-09-10T15:51:44.895527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894150+0000","last_change":"2024-09-10T15:51:41.870267+0000","last_active":"2024-09-10T15:51:44.894150+0000","last_peered":"2024-09-10T15:51:44.894150+0000","last_clean":"2024-09-10T15:51:44.894150+0000","last_became_active":"2024-09-10T15:51:41.868802+0000","last_became_peered":"2024-09-10T15:51:41.868802+0000","last_unstale":"2024-09-10T15:51:44.894150+0000","last_undegraded":"2024-09-10T15:51:44.894150+0000","last_fullsized":"2024-09-10T15:51:44.894150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:14.820633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889334+0000","last_change":"2024-09-10T15:51:32.633999+0000","last_active":"2024-09-10T15:51:44.889334+0000","last_peered":"2024-09-10T15:51:44.889334+0000","last_clean":"2024-09-10T15:51:44.889334+0000","last_became_active":"2024-09-10T15:51:30.613428+0000","last_became_peered":"2024-09-10T15:51:30.613428+0000","last_unstale":"2024-09-10T15:51:44.889334+0000","last_undegraded":"2024-09-10T15:51:44.889334+0000","last_fullsized":"2024-09-10T15:51:44.889334+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:18.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:51:43.884058+0000","last_change":"2024-09-10T15:51:43.884058+0000","last_active":"2024-09-10T15:51:43.884058+0000","last_peered":"2024-09-10T15:51:43.883639+0000","last_clean":"2024-09-10T15:51:43.883639+0000","last_became_active":"2024-09-10T15:51:30.614886+0000","last_became_peered":"2024-09-10T15:51:30.614886+0000","last_unstale":"2024-09-10T15:51:43.884058+0000","last_undegraded":"2024-09-10T15:51:43.884058+0000","last_fullsized":"2024-09-10T15:51:43.884058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.952468+0000","last_change":"2024-09-10T15:51:41.869011+0000","last_active":"2024-09-10T15:51:44.952468+0000","last_peered":"2024-09-10T15:51:44.952468+0000","last_clean":"2024-09-10T15:51:44.952468+0000","last_became_active":"2024-09-10T15:51:41.868056+0000","last_became_peered":"2024-09-10T15:51:41.868056+0000","last_unstale":"2024-09-10T15:51:44.952468+0000","last_undegraded":"2024-09-10T15:51:44.952468+0000","last_fullsized":"2024-09-10T15:51:44.952468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:04.162191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672970,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":45312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672969,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":188416,"data_stored":48570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:51:48.647 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:51:48.926 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:48.927 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:51:48.939 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-10T15:51:48.418788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54404,"kb_used_data":356,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689276,"statfs":{"total":193273528320,"available":193217818624,"internally_reserved":0,"allocated":364544,"data_stored":93882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.550124"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.895083+0000","last_change":"2024-09-10T15:51:41.867143+0000","last_active":"2024-09-10T15:51:44.895083+0000","last_peered":"2024-09-10T15:51:44.895083+0000","last_clean":"2024-09-10T15:51:44.895083+0000","last_became_active":"2024-09-10T15:51:41.866879+0000","last_became_peered":"2024-09-10T15:51:41.866879+0000","last_unstale":"2024-09-10T15:51:44.895083+0000","last_undegraded":"2024-09-10T15:51:44.895083+0000","last_fullsized":"2024-09-10T15:51:44.895083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:09.819843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895509+0000","last_change":"2024-09-10T15:51:44.308511+0000","last_active":"2024-09-10T15:51:44.308511+0000","last_peered":"2024-09-10T15:51:41.869848+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.869661+0000","last_became_peered":"2024-09-10T15:51:41.869661+0000","last_unstale":"2024-09-10T15:51:44.895509+0000","last_undegraded":"2024-09-10T15:51:44.895509+0000","last_fullsized":"2024-09-10T15:51:44.895509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889769+0000","last_change":"2024-09-10T15:51:43.883276+0000","last_active":"2024-09-10T15:51:43.883276+0000","last_peered":"2024-09-10T15:51:43.882835+0000","last_clean":"2024-09-10T15:51:43.882835+0000","last_became_active":"2024-09-10T15:51:41.868597+0000","last_became_peered":"2024-09-10T15:51:41.868597+0000","last_unstale":"2024-09-10T15:51:44.889769+0000","last_undegraded":"2024-09-10T15:51:44.889769+0000","last_fullsized":"2024-09-10T15:51:44.889769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895589+0000","last_change":"2024-09-10T15:51:44.308231+0000","last_active":"2024-09-10T15:51:44.308231+0000","last_peered":"2024-09-10T15:51:41.868682+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.868468+0000","last_became_peered":"2024-09-10T15:51:41.868468+0000","last_unstale":"2024-09-10T15:51:44.895589+0000","last_undegraded":"2024-09-10T15:51:44.895589+0000","last_fullsized":"2024-09-10T15:51:44.895589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895105+0000","last_change":"2024-09-10T15:51:44.308258+0000","last_active":"2024-09-10T15:51:44.308258+0000","last_peered":"2024-09-10T15:51:42.870041+0000","last_clean":"2024-09-10T15:51:42.870041+0000","last_became_active":"2024-09-10T15:51:41.865185+0000","last_became_peered":"2024-09-10T15:51:41.865185+0000","last_unstale":"2024-09-10T15:51:44.895105+0000","last_undegraded":"2024-09-10T15:51:44.895105+0000","last_fullsized":"2024-09-10T15:51:44.895105+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894060+0000","last_change":"2024-09-10T15:51:41.869027+0000","last_active":"2024-09-10T15:51:44.894060+0000","last_peered":"2024-09-10T15:51:44.894060+0000","last_clean":"2024-09-10T15:51:44.894060+0000","last_became_active":"2024-09-10T15:51:41.868747+0000","last_became_peered":"2024-09-10T15:51:41.868747+0000","last_unstale":"2024-09-10T15:51:44.894060+0000","last_undegraded":"2024-09-10T15:51:44.894060+0000","last_fullsized":"2024-09-10T15:51:44.894060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:38.255269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889447+0000","last_change":"2024-09-10T15:51:41.869751+0000","last_active":"2024-09-10T15:51:44.889447+0000","last_peered":"2024-09-10T15:51:44.889447+0000","last_clean":"2024-09-10T15:51:44.889447+0000","last_became_active":"2024-09-10T15:51:41.869474+0000","last_became_peered":"2024-09-10T15:51:41.869474+0000","last_unstale":"2024-09-10T15:51:44.889447+0000","last_undegraded":"2024-09-10T15:51:44.889447+0000","last_fullsized":"2024-09-10T15:51:44.889447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:54.279052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889377+0000","last_change":"2024-09-10T15:51:44.308154+0000","last_active":"2024-09-10T15:51:44.308154+0000","last_peered":"2024-09-10T15:51:42.870214+0000","last_clean":"2024-09-10T15:51:42.870214+0000","last_became_active":"2024-09-10T15:51:41.864310+0000","last_became_peered":"2024-09-10T15:51:41.864310+0000","last_unstale":"2024-09-10T15:51:44.889377+0000","last_undegraded":"2024-09-10T15:51:44.889377+0000","last_fullsized":"2024-09-10T15:51:44.889377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889046+0000","last_change":"2024-09-10T15:51:32.634333+0000","last_active":"2024-09-10T15:51:44.889046+0000","last_peered":"2024-09-10T15:51:44.889046+0000","last_clean":"2024-09-10T15:51:44.889046+0000","last_became_active":"2024-09-10T15:51:30.613634+0000","last_became_peered":"2024-09-10T15:51:30.613634+0000","last_unstale":"2024-09-10T15:51:44.889046+0000","last_undegraded":"2024-09-10T15:51:44.889046+0000","last_fullsized":"2024-09-10T15:51:44.889046+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:50.070951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889787+0000","last_change":"2024-09-10T15:51:43.883355+0000","last_active":"2024-09-10T15:51:43.883355+0000","last_peered":"2024-09-10T15:51:43.882616+0000","last_clean":"2024-09-10T15:51:43.882616+0000","last_became_active":"2024-09-10T15:51:41.868604+0000","last_became_peered":"2024-09-10T15:51:41.868604+0000","last_unstale":"2024-09-10T15:51:44.889787+0000","last_undegraded":"2024-09-10T15:51:44.889787+0000","last_fullsized":"2024-09-10T15:51:44.889787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.894541+0000","last_change":"2024-09-10T15:51:44.308168+0000","last_active":"2024-09-10T15:51:44.308168+0000","last_peered":"2024-09-10T15:51:34.000525+0000","last_clean":"2024-09-10T15:51:34.000525+0000","last_became_active":"2024-09-10T15:51:30.613566+0000","last_became_peered":"2024-09-10T15:51:30.613566+0000","last_unstale":"2024-09-10T15:51:44.894541+0000","last_undegraded":"2024-09-10T15:51:44.894541+0000","last_fullsized":"2024-09-10T15:51:44.894541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894872+0000","last_change":"2024-09-10T15:51:41.867323+0000","last_active":"2024-09-10T15:51:44.894872+0000","last_peered":"2024-09-10T15:51:44.894872+0000","last_clean":"2024-09-10T15:51:44.894872+0000","last_became_active":"2024-09-10T15:51:41.867050+0000","last_became_peered":"2024-09-10T15:51:41.867050+0000","last_unstale":"2024-09-10T15:51:44.894872+0000","last_undegraded":"2024-09-10T15:51:44.894872+0000","last_fullsized":"2024-09-10T15:51:44.894872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:58.326451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894880+0000","last_change":"2024-09-10T15:51:32.629990+0000","last_active":"2024-09-10T15:51:44.894880+0000","last_peered":"2024-09-10T15:51:44.894880+0000","last_clean":"2024-09-10T15:51:44.894880+0000","last_became_active":"2024-09-10T15:51:30.617003+0000","last_became_peered":"2024-09-10T15:51:30.617003+0000","last_unstale":"2024-09-10T15:51:44.894880+0000","last_undegraded":"2024-09-10T15:51:44.894880+0000","last_fullsized":"2024-09-10T15:51:44.894880+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:50.477908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889391+0000","last_change":"2024-09-10T15:51:43.883490+0000","last_active":"2024-09-10T15:51:43.883490+0000","last_peered":"2024-09-10T15:51:41.867177+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.866984+0000","last_became_peered":"2024-09-10T15:51:41.866984+0000","last_unstale":"2024-09-10T15:51:44.889391+0000","last_undegraded":"2024-09-10T15:51:44.889391+0000","last_fullsized":"2024-09-10T15:51:44.889391+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":30,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:51:43.883454+0000","last_change":"2024-09-10T15:51:43.883454+0000","last_active":"2024-09-10T15:51:43.883454+0000","last_peered":"2024-09-10T15:51:43.882793+0000","last_clean":"2024-09-10T15:51:43.882793+0000","last_became_active":"2024-09-10T15:51:30.616135+0000","last_became_peered":"2024-09-10T15:51:30.616135+0000","last_unstale":"2024-09-10T15:51:43.883454+0000","last_undegraded":"2024-09-10T15:51:43.883454+0000","last_fullsized":"2024-09-10T15:51:43.883454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889380+0000","last_change":"2024-09-10T15:51:43.883632+0000","last_active":"2024-09-10T15:51:43.883632+0000","last_peered":"2024-09-10T15:51:41.867187+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.866987+0000","last_became_peered":"2024-09-10T15:51:41.866987+0000","last_unstale":"2024-09-10T15:51:44.889380+0000","last_undegraded":"2024-09-10T15:51:44.889380+0000","last_fullsized":"2024-09-10T15:51:44.889380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894260+0000","last_change":"2024-09-10T15:51:41.869802+0000","last_active":"2024-09-10T15:51:44.894260+0000","last_peered":"2024-09-10T15:51:44.894260+0000","last_clean":"2024-09-10T15:51:44.894260+0000","last_became_active":"2024-09-10T15:51:41.869551+0000","last_became_peered":"2024-09-10T15:51:41.869551+0000","last_unstale":"2024-09-10T15:51:44.894260+0000","last_undegraded":"2024-09-10T15:51:44.894260+0000","last_fullsized":"2024-09-10T15:51:44.894260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:49.796567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889352+0000","last_change":"2024-09-10T15:51:32.634015+0000","last_active":"2024-09-10T15:51:44.889352+0000","last_peered":"2024-09-10T15:51:44.889352+0000","last_clean":"2024-09-10T15:51:44.889352+0000","last_became_active":"2024-09-10T15:51:30.613543+0000","last_became_peered":"2024-09-10T15:51:30.613543+0000","last_unstale":"2024-09-10T15:51:44.889352+0000","last_undegraded":"2024-09-10T15:51:44.889352+0000","last_fullsized":"2024-09-10T15:51:44.889352+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:26.573051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889276+0000","last_change":"2024-09-10T15:51:41.866039+0000","last_active":"2024-09-10T15:51:44.889276+0000","last_peered":"2024-09-10T15:51:44.889276+0000","last_clean":"2024-09-10T15:51:44.889276+0000","last_became_active":"2024-09-10T15:51:41.865716+0000","last_became_peered":"2024-09-10T15:51:41.865716+0000","last_unstale":"2024-09-10T15:51:44.889276+0000","last_undegraded":"2024-09-10T15:51:44.889276+0000","last_fullsized":"2024-09-10T15:51:44.889276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:29.481067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895527+0000","last_change":"2024-09-10T15:51:44.308400+0000","last_active":"2024-09-10T15:51:44.308400+0000","last_peered":"2024-09-10T15:51:34.000546+0000","last_clean":"2024-09-10T15:51:34.000546+0000","last_became_active":"2024-09-10T15:51:30.613434+0000","last_became_peered":"2024-09-10T15:51:30.613434+0000","last_unstale":"2024-09-10T15:51:44.895527+0000","last_undegraded":"2024-09-10T15:51:44.895527+0000","last_fullsized":"2024-09-10T15:51:44.895527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894150+0000","last_change":"2024-09-10T15:51:41.870267+0000","last_active":"2024-09-10T15:51:44.894150+0000","last_peered":"2024-09-10T15:51:44.894150+0000","last_clean":"2024-09-10T15:51:44.894150+0000","last_became_active":"2024-09-10T15:51:41.868802+0000","last_became_peered":"2024-09-10T15:51:41.868802+0000","last_unstale":"2024-09-10T15:51:44.894150+0000","last_undegraded":"2024-09-10T15:51:44.894150+0000","last_fullsized":"2024-09-10T15:51:44.894150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:14.820633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889334+0000","last_change":"2024-09-10T15:51:32.633999+0000","last_active":"2024-09-10T15:51:44.889334+0000","last_peered":"2024-09-10T15:51:44.889334+0000","last_clean":"2024-09-10T15:51:44.889334+0000","last_became_active":"2024-09-10T15:51:30.613428+0000","last_became_peered":"2024-09-10T15:51:30.613428+0000","last_unstale":"2024-09-10T15:51:44.889334+0000","last_undegraded":"2024-09-10T15:51:44.889334+0000","last_fullsized":"2024-09-10T15:51:44.889334+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:18.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:51:43.884058+0000","last_change":"2024-09-10T15:51:43.884058+0000","last_active":"2024-09-10T15:51:43.884058+0000","last_peered":"2024-09-10T15:51:43.883639+0000","last_clean":"2024-09-10T15:51:43.883639+0000","last_became_active":"2024-09-10T15:51:30.614886+0000","last_became_peered":"2024-09-10T15:51:30.614886+0000","last_unstale":"2024-09-10T15:51:43.884058+0000","last_undegraded":"2024-09-10T15:51:43.884058+0000","last_fullsized":"2024-09-10T15:51:43.884058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.952468+0000","last_change":"2024-09-10T15:51:41.869011+0000","last_active":"2024-09-10T15:51:44.952468+0000","last_peered":"2024-09-10T15:51:44.952468+0000","last_clean":"2024-09-10T15:51:44.952468+0000","last_became_active":"2024-09-10T15:51:41.868056+0000","last_became_peered":"2024-09-10T15:51:41.868056+0000","last_unstale":"2024-09-10T15:51:44.952468+0000","last_undegraded":"2024-09-10T15:51:44.952468+0000","last_fullsized":"2024-09-10T15:51:44.952468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:04.162191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672970,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":45312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672969,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":188416,"data_stored":48570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:51:51.942 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:51:52.235 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:52.236 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:51:52.247 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-10T15:51:50.419065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54404,"kb_used_data":356,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689276,"statfs":{"total":193273528320,"available":193217818624,"internally_reserved":0,"allocated":364544,"data_stored":93882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.541070"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.895083+0000","last_change":"2024-09-10T15:51:41.867143+0000","last_active":"2024-09-10T15:51:44.895083+0000","last_peered":"2024-09-10T15:51:44.895083+0000","last_clean":"2024-09-10T15:51:44.895083+0000","last_became_active":"2024-09-10T15:51:41.866879+0000","last_became_peered":"2024-09-10T15:51:41.866879+0000","last_unstale":"2024-09-10T15:51:44.895083+0000","last_undegraded":"2024-09-10T15:51:44.895083+0000","last_fullsized":"2024-09-10T15:51:44.895083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:09.819843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895509+0000","last_change":"2024-09-10T15:51:44.308511+0000","last_active":"2024-09-10T15:51:44.308511+0000","last_peered":"2024-09-10T15:51:41.869848+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.869661+0000","last_became_peered":"2024-09-10T15:51:41.869661+0000","last_unstale":"2024-09-10T15:51:44.895509+0000","last_undegraded":"2024-09-10T15:51:44.895509+0000","last_fullsized":"2024-09-10T15:51:44.895509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889769+0000","last_change":"2024-09-10T15:51:43.883276+0000","last_active":"2024-09-10T15:51:43.883276+0000","last_peered":"2024-09-10T15:51:43.882835+0000","last_clean":"2024-09-10T15:51:43.882835+0000","last_became_active":"2024-09-10T15:51:41.868597+0000","last_became_peered":"2024-09-10T15:51:41.868597+0000","last_unstale":"2024-09-10T15:51:44.889769+0000","last_undegraded":"2024-09-10T15:51:44.889769+0000","last_fullsized":"2024-09-10T15:51:44.889769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895589+0000","last_change":"2024-09-10T15:51:44.308231+0000","last_active":"2024-09-10T15:51:44.308231+0000","last_peered":"2024-09-10T15:51:41.868682+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.868468+0000","last_became_peered":"2024-09-10T15:51:41.868468+0000","last_unstale":"2024-09-10T15:51:44.895589+0000","last_undegraded":"2024-09-10T15:51:44.895589+0000","last_fullsized":"2024-09-10T15:51:44.895589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895105+0000","last_change":"2024-09-10T15:51:44.308258+0000","last_active":"2024-09-10T15:51:44.308258+0000","last_peered":"2024-09-10T15:51:42.870041+0000","last_clean":"2024-09-10T15:51:42.870041+0000","last_became_active":"2024-09-10T15:51:41.865185+0000","last_became_peered":"2024-09-10T15:51:41.865185+0000","last_unstale":"2024-09-10T15:51:44.895105+0000","last_undegraded":"2024-09-10T15:51:44.895105+0000","last_fullsized":"2024-09-10T15:51:44.895105+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894060+0000","last_change":"2024-09-10T15:51:41.869027+0000","last_active":"2024-09-10T15:51:44.894060+0000","last_peered":"2024-09-10T15:51:44.894060+0000","last_clean":"2024-09-10T15:51:44.894060+0000","last_became_active":"2024-09-10T15:51:41.868747+0000","last_became_peered":"2024-09-10T15:51:41.868747+0000","last_unstale":"2024-09-10T15:51:44.894060+0000","last_undegraded":"2024-09-10T15:51:44.894060+0000","last_fullsized":"2024-09-10T15:51:44.894060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:38.255269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889447+0000","last_change":"2024-09-10T15:51:41.869751+0000","last_active":"2024-09-10T15:51:44.889447+0000","last_peered":"2024-09-10T15:51:44.889447+0000","last_clean":"2024-09-10T15:51:44.889447+0000","last_became_active":"2024-09-10T15:51:41.869474+0000","last_became_peered":"2024-09-10T15:51:41.869474+0000","last_unstale":"2024-09-10T15:51:44.889447+0000","last_undegraded":"2024-09-10T15:51:44.889447+0000","last_fullsized":"2024-09-10T15:51:44.889447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:54.279052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889377+0000","last_change":"2024-09-10T15:51:44.308154+0000","last_active":"2024-09-10T15:51:44.308154+0000","last_peered":"2024-09-10T15:51:42.870214+0000","last_clean":"2024-09-10T15:51:42.870214+0000","last_became_active":"2024-09-10T15:51:41.864310+0000","last_became_peered":"2024-09-10T15:51:41.864310+0000","last_unstale":"2024-09-10T15:51:44.889377+0000","last_undegraded":"2024-09-10T15:51:44.889377+0000","last_fullsized":"2024-09-10T15:51:44.889377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889046+0000","last_change":"2024-09-10T15:51:32.634333+0000","last_active":"2024-09-10T15:51:44.889046+0000","last_peered":"2024-09-10T15:51:44.889046+0000","last_clean":"2024-09-10T15:51:44.889046+0000","last_became_active":"2024-09-10T15:51:30.613634+0000","last_became_peered":"2024-09-10T15:51:30.613634+0000","last_unstale":"2024-09-10T15:51:44.889046+0000","last_undegraded":"2024-09-10T15:51:44.889046+0000","last_fullsized":"2024-09-10T15:51:44.889046+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:50.070951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889787+0000","last_change":"2024-09-10T15:51:43.883355+0000","last_active":"2024-09-10T15:51:43.883355+0000","last_peered":"2024-09-10T15:51:43.882616+0000","last_clean":"2024-09-10T15:51:43.882616+0000","last_became_active":"2024-09-10T15:51:41.868604+0000","last_became_peered":"2024-09-10T15:51:41.868604+0000","last_unstale":"2024-09-10T15:51:44.889787+0000","last_undegraded":"2024-09-10T15:51:44.889787+0000","last_fullsized":"2024-09-10T15:51:44.889787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.894541+0000","last_change":"2024-09-10T15:51:44.308168+0000","last_active":"2024-09-10T15:51:44.308168+0000","last_peered":"2024-09-10T15:51:34.000525+0000","last_clean":"2024-09-10T15:51:34.000525+0000","last_became_active":"2024-09-10T15:51:30.613566+0000","last_became_peered":"2024-09-10T15:51:30.613566+0000","last_unstale":"2024-09-10T15:51:44.894541+0000","last_undegraded":"2024-09-10T15:51:44.894541+0000","last_fullsized":"2024-09-10T15:51:44.894541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894872+0000","last_change":"2024-09-10T15:51:41.867323+0000","last_active":"2024-09-10T15:51:44.894872+0000","last_peered":"2024-09-10T15:51:44.894872+0000","last_clean":"2024-09-10T15:51:44.894872+0000","last_became_active":"2024-09-10T15:51:41.867050+0000","last_became_peered":"2024-09-10T15:51:41.867050+0000","last_unstale":"2024-09-10T15:51:44.894872+0000","last_undegraded":"2024-09-10T15:51:44.894872+0000","last_fullsized":"2024-09-10T15:51:44.894872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:58.326451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894880+0000","last_change":"2024-09-10T15:51:32.629990+0000","last_active":"2024-09-10T15:51:44.894880+0000","last_peered":"2024-09-10T15:51:44.894880+0000","last_clean":"2024-09-10T15:51:44.894880+0000","last_became_active":"2024-09-10T15:51:30.617003+0000","last_became_peered":"2024-09-10T15:51:30.617003+0000","last_unstale":"2024-09-10T15:51:44.894880+0000","last_undegraded":"2024-09-10T15:51:44.894880+0000","last_fullsized":"2024-09-10T15:51:44.894880+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:50.477908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889391+0000","last_change":"2024-09-10T15:51:43.883490+0000","last_active":"2024-09-10T15:51:43.883490+0000","last_peered":"2024-09-10T15:51:41.867177+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.866984+0000","last_became_peered":"2024-09-10T15:51:41.866984+0000","last_unstale":"2024-09-10T15:51:44.889391+0000","last_undegraded":"2024-09-10T15:51:44.889391+0000","last_fullsized":"2024-09-10T15:51:44.889391+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":30,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:51:43.883454+0000","last_change":"2024-09-10T15:51:43.883454+0000","last_active":"2024-09-10T15:51:43.883454+0000","last_peered":"2024-09-10T15:51:43.882793+0000","last_clean":"2024-09-10T15:51:43.882793+0000","last_became_active":"2024-09-10T15:51:30.616135+0000","last_became_peered":"2024-09-10T15:51:30.616135+0000","last_unstale":"2024-09-10T15:51:43.883454+0000","last_undegraded":"2024-09-10T15:51:43.883454+0000","last_fullsized":"2024-09-10T15:51:43.883454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.889380+0000","last_change":"2024-09-10T15:51:43.883632+0000","last_active":"2024-09-10T15:51:43.883632+0000","last_peered":"2024-09-10T15:51:41.867187+0000","last_clean":"2024-09-10T15:51:40.844370+0000","last_became_active":"2024-09-10T15:51:41.866987+0000","last_became_peered":"2024-09-10T15:51:41.866987+0000","last_unstale":"2024-09-10T15:51:44.889380+0000","last_undegraded":"2024-09-10T15:51:44.889380+0000","last_fullsized":"2024-09-10T15:51:44.889380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894260+0000","last_change":"2024-09-10T15:51:41.869802+0000","last_active":"2024-09-10T15:51:44.894260+0000","last_peered":"2024-09-10T15:51:44.894260+0000","last_clean":"2024-09-10T15:51:44.894260+0000","last_became_active":"2024-09-10T15:51:41.869551+0000","last_became_peered":"2024-09-10T15:51:41.869551+0000","last_unstale":"2024-09-10T15:51:44.894260+0000","last_undegraded":"2024-09-10T15:51:44.894260+0000","last_fullsized":"2024-09-10T15:51:44.894260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:49.796567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889352+0000","last_change":"2024-09-10T15:51:32.634015+0000","last_active":"2024-09-10T15:51:44.889352+0000","last_peered":"2024-09-10T15:51:44.889352+0000","last_clean":"2024-09-10T15:51:44.889352+0000","last_became_active":"2024-09-10T15:51:30.613543+0000","last_became_peered":"2024-09-10T15:51:30.613543+0000","last_unstale":"2024-09-10T15:51:44.889352+0000","last_undegraded":"2024-09-10T15:51:44.889352+0000","last_fullsized":"2024-09-10T15:51:44.889352+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:26.573051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889276+0000","last_change":"2024-09-10T15:51:41.866039+0000","last_active":"2024-09-10T15:51:44.889276+0000","last_peered":"2024-09-10T15:51:44.889276+0000","last_clean":"2024-09-10T15:51:44.889276+0000","last_became_active":"2024-09-10T15:51:41.865716+0000","last_became_peered":"2024-09-10T15:51:41.865716+0000","last_unstale":"2024-09-10T15:51:44.889276+0000","last_undegraded":"2024-09-10T15:51:44.889276+0000","last_fullsized":"2024-09-10T15:51:44.889276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:29.481067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-10T15:51:44.895527+0000","last_change":"2024-09-10T15:51:44.308400+0000","last_active":"2024-09-10T15:51:44.308400+0000","last_peered":"2024-09-10T15:51:34.000546+0000","last_clean":"2024-09-10T15:51:34.000546+0000","last_became_active":"2024-09-10T15:51:30.613434+0000","last_became_peered":"2024-09-10T15:51:30.613434+0000","last_unstale":"2024-09-10T15:51:44.895527+0000","last_undegraded":"2024-09-10T15:51:44.895527+0000","last_fullsized":"2024-09-10T15:51:44.895527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.894150+0000","last_change":"2024-09-10T15:51:41.870267+0000","last_active":"2024-09-10T15:51:44.894150+0000","last_peered":"2024-09-10T15:51:44.894150+0000","last_clean":"2024-09-10T15:51:44.894150+0000","last_became_active":"2024-09-10T15:51:41.868802+0000","last_became_peered":"2024-09-10T15:51:41.868802+0000","last_unstale":"2024-09-10T15:51:44.894150+0000","last_undegraded":"2024-09-10T15:51:44.894150+0000","last_fullsized":"2024-09-10T15:51:44.894150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:14.820633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.889334+0000","last_change":"2024-09-10T15:51:32.633999+0000","last_active":"2024-09-10T15:51:44.889334+0000","last_peered":"2024-09-10T15:51:44.889334+0000","last_clean":"2024-09-10T15:51:44.889334+0000","last_became_active":"2024-09-10T15:51:30.613428+0000","last_became_peered":"2024-09-10T15:51:30.613428+0000","last_unstale":"2024-09-10T15:51:44.889334+0000","last_undegraded":"2024-09-10T15:51:44.889334+0000","last_fullsized":"2024-09-10T15:51:44.889334+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:18.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-10T15:51:43.884058+0000","last_change":"2024-09-10T15:51:43.884058+0000","last_active":"2024-09-10T15:51:43.884058+0000","last_peered":"2024-09-10T15:51:43.883639+0000","last_clean":"2024-09-10T15:51:43.883639+0000","last_became_active":"2024-09-10T15:51:30.614886+0000","last_became_peered":"2024-09-10T15:51:30.614886+0000","last_unstale":"2024-09-10T15:51:43.884058+0000","last_undegraded":"2024-09-10T15:51:43.884058+0000","last_fullsized":"2024-09-10T15:51:43.884058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T15:51:44.952468+0000","last_change":"2024-09-10T15:51:41.869011+0000","last_active":"2024-09-10T15:51:44.952468+0000","last_peered":"2024-09-10T15:51:44.952468+0000","last_clean":"2024-09-10T15:51:44.952468+0000","last_became_active":"2024-09-10T15:51:41.868056+0000","last_became_peered":"2024-09-10T15:51:41.868056+0000","last_unstale":"2024-09-10T15:51:44.952468+0000","last_undegraded":"2024-09-10T15:51:44.952468+0000","last_fullsized":"2024-09-10T15:51:44.952468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:04.162191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672970,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":176128,"data_stored":45312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672969,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":188416,"data_stored":48570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:51:52.249 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:51:52.527 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:52.528 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:51:52.539 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-09-10T15:51:52.419430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54432,"kb_used_data":384,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689248,"statfs":{"total":193273528320,"available":193217789952,"internally_reserved":0,"allocated":393216,"data_stored":102574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001253"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.895537+0000","last_change":"2024-09-10T15:51:41.867143+0000","last_active":"2024-09-10T15:51:45.895537+0000","last_peered":"2024-09-10T15:51:45.895537+0000","last_clean":"2024-09-10T15:51:45.895537+0000","last_became_active":"2024-09-10T15:51:41.866879+0000","last_became_peered":"2024-09-10T15:51:41.866879+0000","last_unstale":"2024-09-10T15:51:45.895537+0000","last_undegraded":"2024-09-10T15:51:45.895537+0000","last_fullsized":"2024-09-10T15:51:45.895537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:09.819843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.899383+0000","last_change":"2024-09-10T15:51:45.899291+0000","last_active":"2024-09-10T15:51:45.899383+0000","last_peered":"2024-09-10T15:51:45.899383+0000","last_clean":"2024-09-10T15:51:45.899383+0000","last_became_active":"2024-09-10T15:51:45.899015+0000","last_became_peered":"2024-09-10T15:51:45.899015+0000","last_unstale":"2024-09-10T15:51:45.899383+0000","last_undegraded":"2024-09-10T15:51:45.899383+0000","last_fullsized":"2024-09-10T15:51:45.899383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:56.323529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.897326+0000","last_change":"2024-09-10T15:51:45.897230+0000","last_active":"2024-09-10T15:51:45.897326+0000","last_peered":"2024-09-10T15:51:45.897326+0000","last_clean":"2024-09-10T15:51:45.897326+0000","last_became_active":"2024-09-10T15:51:45.896955+0000","last_became_peered":"2024-09-10T15:51:45.896955+0000","last_unstale":"2024-09-10T15:51:45.897326+0000","last_undegraded":"2024-09-10T15:51:45.897326+0000","last_fullsized":"2024-09-10T15:51:45.897326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:35.351091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.899491+0000","last_change":"2024-09-10T15:51:45.899291+0000","last_active":"2024-09-10T15:51:45.899491+0000","last_peered":"2024-09-10T15:51:45.899491+0000","last_clean":"2024-09-10T15:51:45.899491+0000","last_became_active":"2024-09-10T15:51:45.899004+0000","last_became_peered":"2024-09-10T15:51:45.899004+0000","last_unstale":"2024-09-10T15:51:45.899491+0000","last_undegraded":"2024-09-10T15:51:45.899491+0000","last_fullsized":"2024-09-10T15:51:45.899491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:35.854005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.900571+0000","last_change":"2024-09-10T15:51:45.900515+0000","last_active":"2024-09-10T15:51:45.900571+0000","last_peered":"2024-09-10T15:51:45.900571+0000","last_clean":"2024-09-10T15:51:45.900571+0000","last_became_active":"2024-09-10T15:51:45.900278+0000","last_became_peered":"2024-09-10T15:51:45.900278+0000","last_unstale":"2024-09-10T15:51:45.900571+0000","last_undegraded":"2024-09-10T15:51:45.900571+0000","last_fullsized":"2024-09-10T15:51:45.900571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:34.376967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.892926+0000","last_change":"2024-09-10T15:51:41.869027+0000","last_active":"2024-09-10T15:51:45.892926+0000","last_peered":"2024-09-10T15:51:45.892926+0000","last_clean":"2024-09-10T15:51:45.892926+0000","last_became_active":"2024-09-10T15:51:41.868747+0000","last_became_peered":"2024-09-10T15:51:41.868747+0000","last_unstale":"2024-09-10T15:51:45.892926+0000","last_undegraded":"2024-09-10T15:51:45.892926+0000","last_fullsized":"2024-09-10T15:51:45.892926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:38.255269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.892766+0000","last_change":"2024-09-10T15:51:41.869751+0000","last_active":"2024-09-10T15:51:45.892766+0000","last_peered":"2024-09-10T15:51:45.892766+0000","last_clean":"2024-09-10T15:51:45.892766+0000","last_became_active":"2024-09-10T15:51:41.869474+0000","last_became_peered":"2024-09-10T15:51:41.869474+0000","last_unstale":"2024-09-10T15:51:45.892766+0000","last_undegraded":"2024-09-10T15:51:45.892766+0000","last_fullsized":"2024-09-10T15:51:45.892766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:54.279052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.898726+0000","last_change":"2024-09-10T15:51:45.898639+0000","last_active":"2024-09-10T15:51:45.898726+0000","last_peered":"2024-09-10T15:51:45.898726+0000","last_clean":"2024-09-10T15:51:45.898726+0000","last_became_active":"2024-09-10T15:51:45.898380+0000","last_became_peered":"2024-09-10T15:51:45.898380+0000","last_unstale":"2024-09-10T15:51:45.898726+0000","last_undegraded":"2024-09-10T15:51:45.898726+0000","last_fullsized":"2024-09-10T15:51:45.898726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:47.182559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.892654+0000","last_change":"2024-09-10T15:51:32.634333+0000","last_active":"2024-09-10T15:51:45.892654+0000","last_peered":"2024-09-10T15:51:45.892654+0000","last_clean":"2024-09-10T15:51:45.892654+0000","last_became_active":"2024-09-10T15:51:30.613634+0000","last_became_peered":"2024-09-10T15:51:30.613634+0000","last_unstale":"2024-09-10T15:51:45.892654+0000","last_undegraded":"2024-09-10T15:51:45.892654+0000","last_fullsized":"2024-09-10T15:51:45.892654+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:50.070951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.898262+0000","last_change":"2024-09-10T15:51:45.898173+0000","last_active":"2024-09-10T15:51:45.898262+0000","last_peered":"2024-09-10T15:51:45.898262+0000","last_clean":"2024-09-10T15:51:45.898262+0000","last_became_active":"2024-09-10T15:51:45.897903+0000","last_became_peered":"2024-09-10T15:51:45.897903+0000","last_unstale":"2024-09-10T15:51:45.898262+0000","last_undegraded":"2024-09-10T15:51:45.898262+0000","last_fullsized":"2024-09-10T15:51:45.898262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:05.015758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.899501+0000","last_change":"2024-09-10T15:51:45.899385+0000","last_active":"2024-09-10T15:51:45.899501+0000","last_peered":"2024-09-10T15:51:45.899501+0000","last_clean":"2024-09-10T15:51:45.899501+0000","last_became_active":"2024-09-10T15:51:45.899073+0000","last_became_peered":"2024-09-10T15:51:45.899073+0000","last_unstale":"2024-09-10T15:51:45.899501+0000","last_undegraded":"2024-09-10T15:51:45.899501+0000","last_fullsized":"2024-09-10T15:51:45.899501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:34.988883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.895290+0000","last_change":"2024-09-10T15:51:41.867323+0000","last_active":"2024-09-10T15:51:45.895290+0000","last_peered":"2024-09-10T15:51:45.895290+0000","last_clean":"2024-09-10T15:51:45.895290+0000","last_became_active":"2024-09-10T15:51:41.867050+0000","last_became_peered":"2024-09-10T15:51:41.867050+0000","last_unstale":"2024-09-10T15:51:45.895290+0000","last_undegraded":"2024-09-10T15:51:45.895290+0000","last_fullsized":"2024-09-10T15:51:45.895290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:58.326451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.895318+0000","last_change":"2024-09-10T15:51:32.629990+0000","last_active":"2024-09-10T15:51:45.895318+0000","last_peered":"2024-09-10T15:51:45.895318+0000","last_clean":"2024-09-10T15:51:45.895318+0000","last_became_active":"2024-09-10T15:51:30.617003+0000","last_became_peered":"2024-09-10T15:51:30.617003+0000","last_unstale":"2024-09-10T15:51:45.895318+0000","last_undegraded":"2024-09-10T15:51:45.895318+0000","last_fullsized":"2024-09-10T15:51:45.895318+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:50.477908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.897354+0000","last_change":"2024-09-10T15:51:45.897273+0000","last_active":"2024-09-10T15:51:45.897354+0000","last_peered":"2024-09-10T15:51:45.897354+0000","last_clean":"2024-09-10T15:51:45.897354+0000","last_became_active":"2024-09-10T15:51:45.897010+0000","last_became_peered":"2024-09-10T15:51:45.897010+0000","last_unstale":"2024-09-10T15:51:45.897354+0000","last_undegraded":"2024-09-10T15:51:45.897354+0000","last_fullsized":"2024-09-10T15:51:45.897354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:35.402970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.902772+0000","last_change":"2024-09-10T15:51:45.902705+0000","last_active":"2024-09-10T15:51:45.902772+0000","last_peered":"2024-09-10T15:51:45.902772+0000","last_clean":"2024-09-10T15:51:45.902772+0000","last_became_active":"2024-09-10T15:51:45.898674+0000","last_became_peered":"2024-09-10T15:51:45.898674+0000","last_unstale":"2024-09-10T15:51:45.902772+0000","last_undegraded":"2024-09-10T15:51:45.902772+0000","last_fullsized":"2024-09-10T15:51:45.902772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:48.066820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.897335+0000","last_change":"2024-09-10T15:51:45.897226+0000","last_active":"2024-09-10T15:51:45.897335+0000","last_peered":"2024-09-10T15:51:45.897335+0000","last_clean":"2024-09-10T15:51:45.897335+0000","last_became_active":"2024-09-10T15:51:45.896952+0000","last_became_peered":"2024-09-10T15:51:45.896952+0000","last_unstale":"2024-09-10T15:51:45.897335+0000","last_undegraded":"2024-09-10T15:51:45.897335+0000","last_fullsized":"2024-09-10T15:51:45.897335+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:34.259547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.893623+0000","last_change":"2024-09-10T15:51:41.869802+0000","last_active":"2024-09-10T15:51:45.893623+0000","last_peered":"2024-09-10T15:51:45.893623+0000","last_clean":"2024-09-10T15:51:45.893623+0000","last_became_active":"2024-09-10T15:51:41.869551+0000","last_became_peered":"2024-09-10T15:51:41.869551+0000","last_unstale":"2024-09-10T15:51:45.893623+0000","last_undegraded":"2024-09-10T15:51:45.893623+0000","last_fullsized":"2024-09-10T15:51:45.893623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:49.796567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.893156+0000","last_change":"2024-09-10T15:51:32.634015+0000","last_active":"2024-09-10T15:51:45.893156+0000","last_peered":"2024-09-10T15:51:45.893156+0000","last_clean":"2024-09-10T15:51:45.893156+0000","last_became_active":"2024-09-10T15:51:30.613543+0000","last_became_peered":"2024-09-10T15:51:30.613543+0000","last_unstale":"2024-09-10T15:51:45.893156+0000","last_undegraded":"2024-09-10T15:51:45.893156+0000","last_fullsized":"2024-09-10T15:51:45.893156+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:26.573051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.892866+0000","last_change":"2024-09-10T15:51:41.866039+0000","last_active":"2024-09-10T15:51:45.892866+0000","last_peered":"2024-09-10T15:51:45.892866+0000","last_clean":"2024-09-10T15:51:45.892866+0000","last_became_active":"2024-09-10T15:51:41.865716+0000","last_became_peered":"2024-09-10T15:51:41.865716+0000","last_unstale":"2024-09-10T15:51:45.892866+0000","last_undegraded":"2024-09-10T15:51:45.892866+0000","last_fullsized":"2024-09-10T15:51:45.892866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:29.481067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.899489+0000","last_change":"2024-09-10T15:51:45.899414+0000","last_active":"2024-09-10T15:51:45.899489+0000","last_peered":"2024-09-10T15:51:45.899489+0000","last_clean":"2024-09-10T15:51:45.899489+0000","last_became_active":"2024-09-10T15:51:45.899147+0000","last_became_peered":"2024-09-10T15:51:45.899147+0000","last_unstale":"2024-09-10T15:51:45.899489+0000","last_undegraded":"2024-09-10T15:51:45.899489+0000","last_fullsized":"2024-09-10T15:51:45.899489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:51.901853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.893533+0000","last_change":"2024-09-10T15:51:41.870267+0000","last_active":"2024-09-10T15:51:45.893533+0000","last_peered":"2024-09-10T15:51:45.893533+0000","last_clean":"2024-09-10T15:51:45.893533+0000","last_became_active":"2024-09-10T15:51:41.868802+0000","last_became_peered":"2024-09-10T15:51:41.868802+0000","last_unstale":"2024-09-10T15:51:45.893533+0000","last_undegraded":"2024-09-10T15:51:45.893533+0000","last_fullsized":"2024-09-10T15:51:45.893533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:14.820633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.892959+0000","last_change":"2024-09-10T15:51:32.633999+0000","last_active":"2024-09-10T15:51:45.892959+0000","last_peered":"2024-09-10T15:51:45.892959+0000","last_clean":"2024-09-10T15:51:45.892959+0000","last_became_active":"2024-09-10T15:51:30.613428+0000","last_became_peered":"2024-09-10T15:51:30.613428+0000","last_unstale":"2024-09-10T15:51:45.892959+0000","last_undegraded":"2024-09-10T15:51:45.892959+0000","last_fullsized":"2024-09-10T15:51:45.892959+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:18.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.897484+0000","last_change":"2024-09-10T15:51:45.897392+0000","last_active":"2024-09-10T15:51:45.897484+0000","last_peered":"2024-09-10T15:51:45.897484+0000","last_clean":"2024-09-10T15:51:45.897484+0000","last_became_active":"2024-09-10T15:51:45.897105+0000","last_became_peered":"2024-09-10T15:51:45.897105+0000","last_unstale":"2024-09-10T15:51:45.897484+0000","last_undegraded":"2024-09-10T15:51:45.897484+0000","last_fullsized":"2024-09-10T15:51:45.897484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:48.907763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.893064+0000","last_change":"2024-09-10T15:51:41.869011+0000","last_active":"2024-09-10T15:51:45.893064+0000","last_peered":"2024-09-10T15:51:45.893064+0000","last_clean":"2024-09-10T15:51:45.893064+0000","last_became_active":"2024-09-10T15:51:41.868056+0000","last_became_peered":"2024-09-10T15:51:41.868056+0000","last_unstale":"2024-09-10T15:51:45.893064+0000","last_undegraded":"2024-09-10T15:51:45.893064+0000","last_fullsized":"2024-09-10T15:51:45.893064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:04.162191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672971,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":51287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":51287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:51:55.542 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:51:55.833 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:51:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:51:55.846 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-10T15:51:54.419699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54432,"kb_used_data":384,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689248,"statfs":{"total":193273528320,"available":193217789952,"internally_reserved":0,"allocated":393216,"data_stored":102574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.531005"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.895537+0000","last_change":"2024-09-10T15:51:41.867143+0000","last_active":"2024-09-10T15:51:45.895537+0000","last_peered":"2024-09-10T15:51:45.895537+0000","last_clean":"2024-09-10T15:51:45.895537+0000","last_became_active":"2024-09-10T15:51:41.866879+0000","last_became_peered":"2024-09-10T15:51:41.866879+0000","last_unstale":"2024-09-10T15:51:45.895537+0000","last_undegraded":"2024-09-10T15:51:45.895537+0000","last_fullsized":"2024-09-10T15:51:45.895537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:09.819843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.899383+0000","last_change":"2024-09-10T15:51:45.899291+0000","last_active":"2024-09-10T15:51:45.899383+0000","last_peered":"2024-09-10T15:51:45.899383+0000","last_clean":"2024-09-10T15:51:45.899383+0000","last_became_active":"2024-09-10T15:51:45.899015+0000","last_became_peered":"2024-09-10T15:51:45.899015+0000","last_unstale":"2024-09-10T15:51:45.899383+0000","last_undegraded":"2024-09-10T15:51:45.899383+0000","last_fullsized":"2024-09-10T15:51:45.899383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:56.323529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.897326+0000","last_change":"2024-09-10T15:51:45.897230+0000","last_active":"2024-09-10T15:51:45.897326+0000","last_peered":"2024-09-10T15:51:45.897326+0000","last_clean":"2024-09-10T15:51:45.897326+0000","last_became_active":"2024-09-10T15:51:45.896955+0000","last_became_peered":"2024-09-10T15:51:45.896955+0000","last_unstale":"2024-09-10T15:51:45.897326+0000","last_undegraded":"2024-09-10T15:51:45.897326+0000","last_fullsized":"2024-09-10T15:51:45.897326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:35.351091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.899491+0000","last_change":"2024-09-10T15:51:45.899291+0000","last_active":"2024-09-10T15:51:45.899491+0000","last_peered":"2024-09-10T15:51:45.899491+0000","last_clean":"2024-09-10T15:51:45.899491+0000","last_became_active":"2024-09-10T15:51:45.899004+0000","last_became_peered":"2024-09-10T15:51:45.899004+0000","last_unstale":"2024-09-10T15:51:45.899491+0000","last_undegraded":"2024-09-10T15:51:45.899491+0000","last_fullsized":"2024-09-10T15:51:45.899491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:35.854005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.900571+0000","last_change":"2024-09-10T15:51:45.900515+0000","last_active":"2024-09-10T15:51:45.900571+0000","last_peered":"2024-09-10T15:51:45.900571+0000","last_clean":"2024-09-10T15:51:45.900571+0000","last_became_active":"2024-09-10T15:51:45.900278+0000","last_became_peered":"2024-09-10T15:51:45.900278+0000","last_unstale":"2024-09-10T15:51:45.900571+0000","last_undegraded":"2024-09-10T15:51:45.900571+0000","last_fullsized":"2024-09-10T15:51:45.900571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:34.376967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.892926+0000","last_change":"2024-09-10T15:51:41.869027+0000","last_active":"2024-09-10T15:51:45.892926+0000","last_peered":"2024-09-10T15:51:45.892926+0000","last_clean":"2024-09-10T15:51:45.892926+0000","last_became_active":"2024-09-10T15:51:41.868747+0000","last_became_peered":"2024-09-10T15:51:41.868747+0000","last_unstale":"2024-09-10T15:51:45.892926+0000","last_undegraded":"2024-09-10T15:51:45.892926+0000","last_fullsized":"2024-09-10T15:51:45.892926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:38.255269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.892766+0000","last_change":"2024-09-10T15:51:41.869751+0000","last_active":"2024-09-10T15:51:45.892766+0000","last_peered":"2024-09-10T15:51:45.892766+0000","last_clean":"2024-09-10T15:51:45.892766+0000","last_became_active":"2024-09-10T15:51:41.869474+0000","last_became_peered":"2024-09-10T15:51:41.869474+0000","last_unstale":"2024-09-10T15:51:45.892766+0000","last_undegraded":"2024-09-10T15:51:45.892766+0000","last_fullsized":"2024-09-10T15:51:45.892766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:54.279052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.898726+0000","last_change":"2024-09-10T15:51:45.898639+0000","last_active":"2024-09-10T15:51:45.898726+0000","last_peered":"2024-09-10T15:51:45.898726+0000","last_clean":"2024-09-10T15:51:45.898726+0000","last_became_active":"2024-09-10T15:51:45.898380+0000","last_became_peered":"2024-09-10T15:51:45.898380+0000","last_unstale":"2024-09-10T15:51:45.898726+0000","last_undegraded":"2024-09-10T15:51:45.898726+0000","last_fullsized":"2024-09-10T15:51:45.898726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:47.182559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.892654+0000","last_change":"2024-09-10T15:51:32.634333+0000","last_active":"2024-09-10T15:51:45.892654+0000","last_peered":"2024-09-10T15:51:45.892654+0000","last_clean":"2024-09-10T15:51:45.892654+0000","last_became_active":"2024-09-10T15:51:30.613634+0000","last_became_peered":"2024-09-10T15:51:30.613634+0000","last_unstale":"2024-09-10T15:51:45.892654+0000","last_undegraded":"2024-09-10T15:51:45.892654+0000","last_fullsized":"2024-09-10T15:51:45.892654+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:50.070951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.898262+0000","last_change":"2024-09-10T15:51:45.898173+0000","last_active":"2024-09-10T15:51:45.898262+0000","last_peered":"2024-09-10T15:51:45.898262+0000","last_clean":"2024-09-10T15:51:45.898262+0000","last_became_active":"2024-09-10T15:51:45.897903+0000","last_became_peered":"2024-09-10T15:51:45.897903+0000","last_unstale":"2024-09-10T15:51:45.898262+0000","last_undegraded":"2024-09-10T15:51:45.898262+0000","last_fullsized":"2024-09-10T15:51:45.898262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:05.015758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.899501+0000","last_change":"2024-09-10T15:51:45.899385+0000","last_active":"2024-09-10T15:51:45.899501+0000","last_peered":"2024-09-10T15:51:45.899501+0000","last_clean":"2024-09-10T15:51:45.899501+0000","last_became_active":"2024-09-10T15:51:45.899073+0000","last_became_peered":"2024-09-10T15:51:45.899073+0000","last_unstale":"2024-09-10T15:51:45.899501+0000","last_undegraded":"2024-09-10T15:51:45.899501+0000","last_fullsized":"2024-09-10T15:51:45.899501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:34.988883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.895290+0000","last_change":"2024-09-10T15:51:41.867323+0000","last_active":"2024-09-10T15:51:45.895290+0000","last_peered":"2024-09-10T15:51:45.895290+0000","last_clean":"2024-09-10T15:51:45.895290+0000","last_became_active":"2024-09-10T15:51:41.867050+0000","last_became_peered":"2024-09-10T15:51:41.867050+0000","last_unstale":"2024-09-10T15:51:45.895290+0000","last_undegraded":"2024-09-10T15:51:45.895290+0000","last_fullsized":"2024-09-10T15:51:45.895290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:58.326451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.895318+0000","last_change":"2024-09-10T15:51:32.629990+0000","last_active":"2024-09-10T15:51:45.895318+0000","last_peered":"2024-09-10T15:51:45.895318+0000","last_clean":"2024-09-10T15:51:45.895318+0000","last_became_active":"2024-09-10T15:51:30.617003+0000","last_became_peered":"2024-09-10T15:51:30.617003+0000","last_unstale":"2024-09-10T15:51:45.895318+0000","last_undegraded":"2024-09-10T15:51:45.895318+0000","last_fullsized":"2024-09-10T15:51:45.895318+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:50.477908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.897354+0000","last_change":"2024-09-10T15:51:45.897273+0000","last_active":"2024-09-10T15:51:45.897354+0000","last_peered":"2024-09-10T15:51:45.897354+0000","last_clean":"2024-09-10T15:51:45.897354+0000","last_became_active":"2024-09-10T15:51:45.897010+0000","last_became_peered":"2024-09-10T15:51:45.897010+0000","last_unstale":"2024-09-10T15:51:45.897354+0000","last_undegraded":"2024-09-10T15:51:45.897354+0000","last_fullsized":"2024-09-10T15:51:45.897354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:35.402970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.902772+0000","last_change":"2024-09-10T15:51:45.902705+0000","last_active":"2024-09-10T15:51:45.902772+0000","last_peered":"2024-09-10T15:51:45.902772+0000","last_clean":"2024-09-10T15:51:45.902772+0000","last_became_active":"2024-09-10T15:51:45.898674+0000","last_became_peered":"2024-09-10T15:51:45.898674+0000","last_unstale":"2024-09-10T15:51:45.902772+0000","last_undegraded":"2024-09-10T15:51:45.902772+0000","last_fullsized":"2024-09-10T15:51:45.902772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:48.066820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.897335+0000","last_change":"2024-09-10T15:51:45.897226+0000","last_active":"2024-09-10T15:51:45.897335+0000","last_peered":"2024-09-10T15:51:45.897335+0000","last_clean":"2024-09-10T15:51:45.897335+0000","last_became_active":"2024-09-10T15:51:45.896952+0000","last_became_peered":"2024-09-10T15:51:45.896952+0000","last_unstale":"2024-09-10T15:51:45.897335+0000","last_undegraded":"2024-09-10T15:51:45.897335+0000","last_fullsized":"2024-09-10T15:51:45.897335+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:34.259547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.893623+0000","last_change":"2024-09-10T15:51:41.869802+0000","last_active":"2024-09-10T15:51:45.893623+0000","last_peered":"2024-09-10T15:51:45.893623+0000","last_clean":"2024-09-10T15:51:45.893623+0000","last_became_active":"2024-09-10T15:51:41.869551+0000","last_became_peered":"2024-09-10T15:51:41.869551+0000","last_unstale":"2024-09-10T15:51:45.893623+0000","last_undegraded":"2024-09-10T15:51:45.893623+0000","last_fullsized":"2024-09-10T15:51:45.893623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:49.796567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.893156+0000","last_change":"2024-09-10T15:51:32.634015+0000","last_active":"2024-09-10T15:51:45.893156+0000","last_peered":"2024-09-10T15:51:45.893156+0000","last_clean":"2024-09-10T15:51:45.893156+0000","last_became_active":"2024-09-10T15:51:30.613543+0000","last_became_peered":"2024-09-10T15:51:30.613543+0000","last_unstale":"2024-09-10T15:51:45.893156+0000","last_undegraded":"2024-09-10T15:51:45.893156+0000","last_fullsized":"2024-09-10T15:51:45.893156+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:26.573051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.892866+0000","last_change":"2024-09-10T15:51:41.866039+0000","last_active":"2024-09-10T15:51:45.892866+0000","last_peered":"2024-09-10T15:51:45.892866+0000","last_clean":"2024-09-10T15:51:45.892866+0000","last_became_active":"2024-09-10T15:51:41.865716+0000","last_became_peered":"2024-09-10T15:51:41.865716+0000","last_unstale":"2024-09-10T15:51:45.892866+0000","last_undegraded":"2024-09-10T15:51:45.892866+0000","last_fullsized":"2024-09-10T15:51:45.892866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:29.481067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.899489+0000","last_change":"2024-09-10T15:51:45.899414+0000","last_active":"2024-09-10T15:51:45.899489+0000","last_peered":"2024-09-10T15:51:45.899489+0000","last_clean":"2024-09-10T15:51:45.899489+0000","last_became_active":"2024-09-10T15:51:45.899147+0000","last_became_peered":"2024-09-10T15:51:45.899147+0000","last_unstale":"2024-09-10T15:51:45.899489+0000","last_undegraded":"2024-09-10T15:51:45.899489+0000","last_fullsized":"2024-09-10T15:51:45.899489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:51.901853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.893533+0000","last_change":"2024-09-10T15:51:41.870267+0000","last_active":"2024-09-10T15:51:45.893533+0000","last_peered":"2024-09-10T15:51:45.893533+0000","last_clean":"2024-09-10T15:51:45.893533+0000","last_became_active":"2024-09-10T15:51:41.868802+0000","last_became_peered":"2024-09-10T15:51:41.868802+0000","last_unstale":"2024-09-10T15:51:45.893533+0000","last_undegraded":"2024-09-10T15:51:45.893533+0000","last_fullsized":"2024-09-10T15:51:45.893533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:14.820633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.892959+0000","last_change":"2024-09-10T15:51:32.633999+0000","last_active":"2024-09-10T15:51:45.892959+0000","last_peered":"2024-09-10T15:51:45.892959+0000","last_clean":"2024-09-10T15:51:45.892959+0000","last_became_active":"2024-09-10T15:51:30.613428+0000","last_became_peered":"2024-09-10T15:51:30.613428+0000","last_unstale":"2024-09-10T15:51:45.892959+0000","last_undegraded":"2024-09-10T15:51:45.892959+0000","last_fullsized":"2024-09-10T15:51:45.892959+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:18.136340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":38,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.897484+0000","last_change":"2024-09-10T15:51:45.897392+0000","last_active":"2024-09-10T15:51:45.897484+0000","last_peered":"2024-09-10T15:51:45.897484+0000","last_clean":"2024-09-10T15:51:45.897484+0000","last_became_active":"2024-09-10T15:51:45.897105+0000","last_became_peered":"2024-09-10T15:51:45.897105+0000","last_unstale":"2024-09-10T15:51:45.897484+0000","last_undegraded":"2024-09-10T15:51:45.897484+0000","last_fullsized":"2024-09-10T15:51:45.897484+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:48.907763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T15:51:45.893064+0000","last_change":"2024-09-10T15:51:41.869011+0000","last_active":"2024-09-10T15:51:45.893064+0000","last_peered":"2024-09-10T15:51:45.893064+0000","last_clean":"2024-09-10T15:51:45.893064+0000","last_became_active":"2024-09-10T15:51:41.868056+0000","last_became_peered":"2024-09-10T15:51:41.868056+0000","last_unstale":"2024-09-10T15:51:45.893064+0000","last_undegraded":"2024-09-10T15:51:45.893064+0000","last_fullsized":"2024-09-10T15:51:45.893064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:04.162191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672971,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":51287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":51287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:51:55.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-10T15:51:55.847 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-10T15:51:55.847 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-10T15:51:55.896 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-10T15:51:55.896 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-10T15:51:55.945 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_1 2024-09-10T15:51:55.945 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_1 2024-09-10T15:51:55.992 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-10T15:51:55.992 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-10T15:51:56.040 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-10T15:51:56.040 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-10T15:51:56.089 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_2 2024-09-10T15:51:56.089 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_2 2024-09-10T15:51:56.136 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-10T15:51:56.136 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-10T15:51:56.184 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-10T15:51:56.184 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-10T15:51:56.232 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_3 2024-09-10T15:51:56.232 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_3 2024-09-10T15:51:56.279 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-10T15:51:56.279 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-10T15:51:56.327 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-10T15:51:56.328 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-10T15:51:56.376 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_4 2024-09-10T15:51:56.376 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_4 2024-09-10T15:51:56.423 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-10T15:51:56.423 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-10T15:51:56.471 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-10T15:51:56.472 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-10T15:51:56.520 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_5 2024-09-10T15:51:56.599 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_5 2024-09-10T15:51:56.647 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-10T15:51:56.647 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-10T15:51:56.696 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-10T15:51:56.696 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-10T15:51:56.744 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_6 2024-09-10T15:51:56.745 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_6 2024-09-10T15:51:56.792 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-10T15:51:56.792 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-10T15:51:56.840 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-10T15:51:56.840 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-10T15:51:56.891 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_7 2024-09-10T15:51:56.891 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_7 2024-09-10T15:51:56.938 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-10T15:51:56.938 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-10T15:51:56.986 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-10T15:51:56.986 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-10T15:51:57.034 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_8 2024-09-10T15:51:57.034 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_8 2024-09-10T15:51:57.081 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-10T15:51:57.081 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-10T15:51:57.129 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-10T15:51:57.129 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-10T15:51:57.177 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_9 2024-09-10T15:51:57.177 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_9 2024-09-10T15:51:57.227 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 injectargs '--osd-recovery-delay-start 1000 --osd-min-pg-log-entries 100000000' 2024-09-10T15:51:57.346 INFO:teuthology.orchestra.run.smithi121.stdout:{} 2024-09-10T15:51:57.346 INFO:teuthology.orchestra.run.smithi121.stderr:osd_min_pg_log_entries = '' (not observed, change may require restart) osd_recovery_delay_start = '' 2024-09-10T15:51:57.356 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T15:51:57.357 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T15:51:57.400 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T15:51:57.400 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-10T15:51:57.690 INFO:tasks.ceph.mon.c.smithi121.stderr:2024-09-10T15:51:57.689+0000 7fb1890bc640 -1 mon.c@2(leader).osd e22 definitely_dead 0 2024-09-10T15:51:57.931 INFO:teuthology.orchestra.run.smithi121.stderr:marked down osd.0. 2024-09-10T15:51:57.945 INFO:tasks.util.rados:rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-10T15:51:57.945 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-10T15:51:58.949 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-10T15:51:58.949 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-10T15:51:59.002 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-10T15:51:59.002 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-10T15:51:59.052 INFO:tasks.util.rados:rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-10T15:51:59.052 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-10T15:51:59.100 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-10T15:51:59.100 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-10T15:51:59.148 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-10T15:51:59.149 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-10T15:51:59.196 INFO:tasks.util.rados:rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-10T15:51:59.197 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-10T15:51:59.245 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-10T15:51:59.245 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-10T15:51:59.293 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-10T15:51:59.293 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-10T15:51:59.340 INFO:tasks.util.rados:rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-10T15:51:59.340 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-10T15:51:59.390 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-10T15:51:59.390 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-10T15:51:59.439 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-10T15:51:59.439 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-10T15:51:59.487 INFO:tasks.util.rados:rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-10T15:51:59.487 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-10T15:51:59.535 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-10T15:51:59.535 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-10T15:51:59.584 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-10T15:51:59.584 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-10T15:51:59.632 INFO:tasks.util.rados:rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-10T15:51:59.632 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-10T15:51:59.679 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-10T15:51:59.679 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-10T15:51:59.727 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-10T15:51:59.727 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-10T15:51:59.775 INFO:tasks.util.rados:rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-10T15:51:59.775 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-10T15:51:59.823 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-10T15:51:59.823 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-10T15:51:59.872 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-10T15:51:59.872 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-10T15:51:59.920 INFO:tasks.util.rados:rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-10T15:51:59.921 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-10T15:51:59.969 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-10T15:51:59.969 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-10T15:52:00.017 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-10T15:52:00.017 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-10T15:52:00.065 INFO:tasks.util.rados:rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-10T15:52:00.066 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-10T15:52:00.114 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-10T15:52:00.114 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-10T15:52:00.162 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-10T15:52:00.162 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-10T15:52:00.210 INFO:tasks.rep_lost_unfound_delete:osd.0 command_args is foo 2024-09-10T15:52:00.210 INFO:tasks.rep_lost_unfound_delete:() 2024-09-10T15:52:00.211 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T15:52:00.211 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 --osd-recovery-delay-start 1000 2024-09-10T15:52:00.213 INFO:tasks.ceph.osd.0:Started 2024-09-10T15:52:00.213 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T15:52:00.312 INFO:teuthology.orchestra.run.smithi121.stderr:no valid command found; 10 closest matches: 2024-09-10T15:52:00.312 INFO:teuthology.orchestra.run.smithi121.stderr:0 2024-09-10T15:52:00.312 INFO:teuthology.orchestra.run.smithi121.stderr:1 2024-09-10T15:52:00.312 INFO:teuthology.orchestra.run.smithi121.stderr:2 2024-09-10T15:52:00.312 INFO:teuthology.orchestra.run.smithi121.stderr:abort 2024-09-10T15:52:00.312 INFO:teuthology.orchestra.run.smithi121.stderr:assert 2024-09-10T15:52:00.312 INFO:teuthology.orchestra.run.smithi121.stderr:config diff 2024-09-10T15:52:00.312 INFO:teuthology.orchestra.run.smithi121.stderr:config diff get 2024-09-10T15:52:00.312 INFO:teuthology.orchestra.run.smithi121.stderr:config get 2024-09-10T15:52:00.312 INFO:teuthology.orchestra.run.smithi121.stderr:config help [] 2024-09-10T15:52:00.312 INFO:teuthology.orchestra.run.smithi121.stderr:config set ... 2024-09-10T15:52:00.313 INFO:teuthology.orchestra.run.smithi121.stderr:admin_socket: invalid command 2024-09-10T15:52:00.314 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T15:52:00.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-10T15:52:01.367 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:01.365+0000 7fc1c7a1f640 -1 Falling back to public interface 2024-09-10T15:52:05.141 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:05.139+0000 7fc1c7a1f640 -1 osd.0 22 log_to_monitors true 2024-09-10T15:52:05.315 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T15:52:05.426 INFO:teuthology.orchestra.run.smithi121.stderr:no valid command found; 10 closest matches: 2024-09-10T15:52:05.426 INFO:teuthology.orchestra.run.smithi121.stderr:0 2024-09-10T15:52:05.426 INFO:teuthology.orchestra.run.smithi121.stderr:1 2024-09-10T15:52:05.426 INFO:teuthology.orchestra.run.smithi121.stderr:2 2024-09-10T15:52:05.426 INFO:teuthology.orchestra.run.smithi121.stderr:abort 2024-09-10T15:52:05.426 INFO:teuthology.orchestra.run.smithi121.stderr:assert 2024-09-10T15:52:05.426 INFO:teuthology.orchestra.run.smithi121.stderr:bluefs debug_inject_read_zeros 2024-09-10T15:52:05.426 INFO:teuthology.orchestra.run.smithi121.stderr:bluefs files list 2024-09-10T15:52:05.426 INFO:teuthology.orchestra.run.smithi121.stderr:bluefs stats 2024-09-10T15:52:05.426 INFO:teuthology.orchestra.run.smithi121.stderr:bluestore allocator dump block 2024-09-10T15:52:05.427 INFO:teuthology.orchestra.run.smithi121.stderr:bluestore allocator fragmentation block 2024-09-10T15:52:05.427 INFO:teuthology.orchestra.run.smithi121.stderr:admin_socket: invalid command 2024-09-10T15:52:05.428 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T15:52:05.429 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-10T15:52:06.004 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:06.002+0000 7fc1bb20b640 -1 osd.0 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:52:10.429 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-10T15:52:10.544 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:10.544 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-09-10T15:52:10.544 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-09-10T15:52:10.545 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:10.553 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-10T15:52:10.843 INFO:tasks.ceph.mon.c.smithi121.stderr:2024-09-10T15:52:10.842+0000 7fb1890bc640 -1 mon.c@2(leader).osd e27 definitely_dead 0 2024-09-10T15:52:10.844 INFO:teuthology.orchestra.run.smithi121.stderr:osd.0 is already in. 2024-09-10T15:52:10.857 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.0 to be up 2024-09-10T15:52:10.857 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:52:11.142 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:11.142 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","created":"2024-09-10T15:51:14.060262+0000","modified":"2024-09-10T15:52:07.997085+0000","last_up_change":"2024-09-10T15:52:06.995612+0000","last_in_change":"2024-09-10T15:51:44.875189+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:51:29.314004+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-10T15:51:40.401037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"53b58107-009b-46ac-921d-2e8d9580081e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":22,"up_from":26,"up_thru":26,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6800","nonce":824549766}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6801","nonce":824549766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6803","nonce":824549766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6802","nonce":824549766}]},"public_addr":"172.21.15.121:6800/824549766","cluster_addr":"172.21.15.121:6801/824549766","heartbeat_back_addr":"172.21.15.121:6803/824549766","heartbeat_front_addr":"172.21.15.121:6802/824549766","state":["exists","up"]},{"osd":1,"uuid":"aced9681-6a62-49a4-a1e2-6ff7d9a544ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6804","nonce":3301493939}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6805","nonce":3301493939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6807","nonce":3301493939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6806","nonce":3301493939}]},"public_addr":"172.21.15.121:6804/3301493939","cluster_addr":"172.21.15.121:6805/3301493939","heartbeat_back_addr":"172.21.15.121:6807/3301493939","heartbeat_front_addr":"172.21.15.121:6806/3301493939","state":["exists","up"]},{"osd":2,"uuid":"0cfba235-9896-429e-8396-69dfeb7a44bb","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6800","nonce":1473565644}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6801","nonce":1473565644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6803","nonce":1473565644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6802","nonce":1473565644}]},"public_addr":"172.21.15.121:6800/1473565644","cluster_addr":"172.21.15.121:6801/1473565644","heartbeat_back_addr":"172.21.15.121:6803/1473565644","heartbeat_front_addr":"172.21.15.121:6802/1473565644","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:22.164346+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.644539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-10T15:51:21.413080+0000","dead_epoch":20}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:52:11.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.0 is up 2024-09-10T15:52:11.155 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:52:11.156 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T15:52:11.287 INFO:teuthology.orchestra.run.smithi121.stdout:111669149699 2024-09-10T15:52:11.288 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:52:11.289 INFO:teuthology.orchestra.run.smithi121.stdout:42949672976 2024-09-10T15:52:11.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:42949672974 2024-09-10T15:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-09-10T15:52:11.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672976 got 42949672974 for osd.1 2024-09-10T15:52:11.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149699 got 1 for osd.0 2024-09-10T15:52:12.651 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:52:12.651 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:52:12.962 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-09-10T15:52:12.962 INFO:teuthology.orchestra.run.smithi121.stdout:42949672974 2024-09-10T15:52:12.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672976 got 42949672974 for osd.1 2024-09-10T15:52:12.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149699 got 1 for osd.0 2024-09-10T15:52:13.980 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T15:52:13.981 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:52:14.283 INFO:teuthology.orchestra.run.smithi121.stdout:111669149699 2024-09-10T15:52:14.288 INFO:teuthology.orchestra.run.smithi121.stdout:42949672976 2024-09-10T15:52:14.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149699 got 111669149699 for osd.0 2024-09-10T15:52:14.297 DEBUG:teuthology.parallel:result is None 2024-09-10T15:52:14.301 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672976 got 42949672976 for osd.1 2024-09-10T15:52:14.302 DEBUG:teuthology.parallel:result is None 2024-09-10T15:52:14.302 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-10T15:52:14.302 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:52:14.625 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:14.625 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:52:14.637 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-09-10T15:52:14.423571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55104,"kb_used_data":608,"kb_used_omap":5,"kb_used_meta":54458,"kb_avail":188688576,"statfs":{"total":193273528320,"available":193217101824,"internally_reserved":0,"allocated":622592,"data_stored":150930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6030,"internal_metadata":55765106},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001984"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T15:52:08.016993+0000","last_change":"2024-09-10T15:52:08.016882+0000","last_active":"2024-09-10T15:52:08.016993+0000","last_peered":"2024-09-10T15:52:08.016993+0000","last_clean":"2024-09-10T15:52:08.016993+0000","last_became_active":"2024-09-10T15:52:08.016458+0000","last_became_peered":"2024-09-10T15:52:08.016458+0000","last_unstale":"2024-09-10T15:52:08.016993+0000","last_undegraded":"2024-09-10T15:52:08.016993+0000","last_fullsized":"2024-09-10T15:52:08.016993+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:51.963644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'4","reported_seq":30,"reported_epoch":27,"state":"active+recovering+degraded","last_fresh":"2024-09-10T15:52:08.016114+0000","last_change":"2024-09-10T15:52:08.018954+0000","last_active":"2024-09-10T15:52:08.016114+0000","last_peered":"2024-09-10T15:52:08.016114+0000","last_clean":"2024-09-10T15:51:56.324820+0000","last_became_active":"2024-09-10T15:52:08.015805+0000","last_became_peered":"2024-09-10T15:52:08.015805+0000","last_unstale":"2024-09-10T15:52:08.016114+0000","last_undegraded":"2024-09-10T15:52:08.005720+0000","last_fullsized":"2024-09-10T15:52:08.016114+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'3","reported_seq":42,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:08.018186+0000","last_change":"2024-09-10T15:52:08.018309+0000","last_active":"2024-09-10T15:52:08.018186+0000","last_peered":"2024-09-10T15:52:08.018186+0000","last_clean":"2024-09-10T15:51:56.935537+0000","last_became_active":"2024-09-10T15:52:08.018132+0000","last_became_peered":"2024-09-10T15:52:08.018132+0000","last_unstale":"2024-09-10T15:52:08.018186+0000","last_undegraded":"2024-09-10T15:52:08.009872+0000","last_fullsized":"2024-09-10T15:52:08.018186+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T15:52:08.016282+0000","last_change":"2024-09-10T15:52:08.015439+0000","last_active":"2024-09-10T15:52:08.016282+0000","last_peered":"2024-09-10T15:52:08.016282+0000","last_clean":"2024-09-10T15:52:08.016282+0000","last_became_active":"2024-09-10T15:52:08.014949+0000","last_became_peered":"2024-09-10T15:52:08.014949+0000","last_unstale":"2024-09-10T15:52:08.016282+0000","last_undegraded":"2024-09-10T15:52:08.016282+0000","last_fullsized":"2024-09-10T15:52:08.016282+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:04.686508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"24'5","reported_seq":36,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:08.016742+0000","last_change":"2024-09-10T15:52:08.016977+0000","last_active":"2024-09-10T15:52:08.016742+0000","last_peered":"2024-09-10T15:52:08.016742+0000","last_clean":"2024-09-10T15:51:56.229374+0000","last_became_active":"2024-09-10T15:52:08.016626+0000","last_became_peered":"2024-09-10T15:52:08.016626+0000","last_unstale":"2024-09-10T15:52:08.016742+0000","last_undegraded":"2024-09-10T15:52:08.005985+0000","last_fullsized":"2024-09-10T15:52:08.016742+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"24'3","reported_seq":38,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:08.015524+0000","last_change":"2024-09-10T15:52:08.015714+0000","last_active":"2024-09-10T15:52:08.015524+0000","last_peered":"2024-09-10T15:52:08.015524+0000","last_clean":"2024-09-10T15:51:57.126582+0000","last_became_active":"2024-09-10T15:52:08.015418+0000","last_became_peered":"2024-09-10T15:52:08.015418+0000","last_unstale":"2024-09-10T15:52:08.015524+0000","last_undegraded":"2024-09-10T15:52:08.008704+0000","last_fullsized":"2024-09-10T15:52:08.015524+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"24'4","reported_seq":39,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:08.013882+0000","last_change":"2024-09-10T15:52:08.014073+0000","last_active":"2024-09-10T15:52:08.013882+0000","last_peered":"2024-09-10T15:52:08.013882+0000","last_clean":"2024-09-10T15:51:56.469065+0000","last_became_active":"2024-09-10T15:52:08.013790+0000","last_became_peered":"2024-09-10T15:52:08.013790+0000","last_unstale":"2024-09-10T15:52:08.013882+0000","last_undegraded":"2024-09-10T15:52:08.007635+0000","last_fullsized":"2024-09-10T15:52:08.013882+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"24'1","reported_seq":33,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:08.016799+0000","last_change":"2024-09-10T15:52:08.017036+0000","last_active":"2024-09-10T15:52:08.016799+0000","last_peered":"2024-09-10T15:52:08.016799+0000","last_clean":"2024-09-10T15:51:42.870214+0000","last_became_active":"2024-09-10T15:52:08.016702+0000","last_became_peered":"2024-09-10T15:52:08.016702+0000","last_unstale":"2024-09-10T15:52:08.016799+0000","last_undegraded":"2024-09-10T15:52:08.005282+0000","last_fullsized":"2024-09-10T15:52:08.016799+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T15:52:08.018181+0000","last_change":"2024-09-10T15:52:08.018087+0000","last_active":"2024-09-10T15:52:08.018181+0000","last_peered":"2024-09-10T15:52:08.018181+0000","last_clean":"2024-09-10T15:52:08.018181+0000","last_became_active":"2024-09-10T15:52:08.017706+0000","last_became_peered":"2024-09-10T15:52:08.017706+0000","last_unstale":"2024-09-10T15:52:08.018181+0000","last_undegraded":"2024-09-10T15:52:08.018181+0000","last_fullsized":"2024-09-10T15:52:08.018181+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:36.539494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"24'5","reported_seq":44,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:08.018319+0000","last_change":"2024-09-10T15:52:08.018562+0000","last_active":"2024-09-10T15:52:08.018319+0000","last_peered":"2024-09-10T15:52:08.018319+0000","last_clean":"2024-09-10T15:51:56.983672+0000","last_became_active":"2024-09-10T15:52:08.018231+0000","last_became_peered":"2024-09-10T15:52:08.018231+0000","last_unstale":"2024-09-10T15:52:08.018319+0000","last_undegraded":"2024-09-10T15:52:08.010265+0000","last_fullsized":"2024-09-10T15:52:08.018319+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T15:52:08.014981+0000","last_change":"2024-09-10T15:52:08.014874+0000","last_active":"2024-09-10T15:52:08.014981+0000","last_peered":"2024-09-10T15:52:08.014981+0000","last_clean":"2024-09-10T15:52:08.014981+0000","last_became_active":"2024-09-10T15:52:08.014558+0000","last_became_peered":"2024-09-10T15:52:08.014558+0000","last_unstale":"2024-09-10T15:52:08.014981+0000","last_undegraded":"2024-09-10T15:52:08.014981+0000","last_fullsized":"2024-09-10T15:52:08.014981+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:03.587354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"24'6","reported_seq":32,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:08.015964+0000","last_change":"2024-09-10T15:52:08.016137+0000","last_active":"2024-09-10T15:52:08.015964+0000","last_peered":"2024-09-10T15:52:08.015964+0000","last_clean":"2024-09-10T15:51:57.174499+0000","last_became_active":"2024-09-10T15:52:08.015866+0000","last_became_peered":"2024-09-10T15:52:08.015866+0000","last_unstale":"2024-09-10T15:52:08.015964+0000","last_undegraded":"2024-09-10T15:52:08.006027+0000","last_fullsized":"2024-09-10T15:52:08.015964+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T15:52:08.011089+0000","last_change":"2024-09-10T15:52:08.010985+0000","last_active":"2024-09-10T15:52:08.011089+0000","last_peered":"2024-09-10T15:52:08.011089+0000","last_clean":"2024-09-10T15:52:08.011089+0000","last_became_active":"2024-09-10T15:52:08.010619+0000","last_became_peered":"2024-09-10T15:52:08.010619+0000","last_unstale":"2024-09-10T15:52:08.011089+0000","last_undegraded":"2024-09-10T15:52:08.011089+0000","last_fullsized":"2024-09-10T15:52:08.011089+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:51.581402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"24'3","reported_seq":38,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:08.016000+0000","last_change":"2024-09-10T15:52:08.016185+0000","last_active":"2024-09-10T15:52:08.016000+0000","last_peered":"2024-09-10T15:52:08.016000+0000","last_clean":"2024-09-10T15:51:56.181288+0000","last_became_active":"2024-09-10T15:52:08.015687+0000","last_became_peered":"2024-09-10T15:52:08.015687+0000","last_unstale":"2024-09-10T15:52:08.016000+0000","last_undegraded":"2024-09-10T15:52:08.009158+0000","last_fullsized":"2024-09-10T15:52:08.016000+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T15:52:08.018721+0000","last_change":"2024-09-10T15:52:08.018617+0000","last_active":"2024-09-10T15:52:08.018721+0000","last_peered":"2024-09-10T15:52:08.018721+0000","last_clean":"2024-09-10T15:52:08.018721+0000","last_became_active":"2024-09-10T15:52:08.017892+0000","last_became_peered":"2024-09-10T15:52:08.017892+0000","last_unstale":"2024-09-10T15:52:08.018721+0000","last_undegraded":"2024-09-10T15:52:08.018721+0000","last_fullsized":"2024-09-10T15:52:08.018721+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:20.958469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"24'4","reported_seq":40,"reported_epoch":27,"state":"active+recovering+degraded","last_fresh":"2024-09-10T15:52:08.011449+0000","last_change":"2024-09-10T15:52:08.012128+0000","last_active":"2024-09-10T15:52:08.011449+0000","last_peered":"2024-09-10T15:52:08.011449+0000","last_clean":"2024-09-10T15:51:56.644471+0000","last_became_active":"2024-09-10T15:52:08.011170+0000","last_became_peered":"2024-09-10T15:52:08.011170+0000","last_unstale":"2024-09-10T15:52:08.011449+0000","last_undegraded":"2024-09-10T15:52:08.002147+0000","last_fullsized":"2024-09-10T15:52:08.011449+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"24'1","reported_seq":35,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:08.015460+0000","last_change":"2024-09-10T15:52:08.015660+0000","last_active":"2024-09-10T15:52:08.015460+0000","last_peered":"2024-09-10T15:52:08.015460+0000","last_clean":"2024-09-10T15:51:45.893657+0000","last_became_active":"2024-09-10T15:52:08.015371+0000","last_became_peered":"2024-09-10T15:52:08.015371+0000","last_unstale":"2024-09-10T15:52:08.015460+0000","last_undegraded":"2024-09-10T15:52:08.009530+0000","last_fullsized":"2024-09-10T15:52:08.015460+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T15:52:08.013909+0000","last_change":"2024-09-10T15:52:08.013820+0000","last_active":"2024-09-10T15:52:08.013909+0000","last_peered":"2024-09-10T15:52:08.013909+0000","last_clean":"2024-09-10T15:52:08.013909+0000","last_became_active":"2024-09-10T15:52:08.013397+0000","last_became_peered":"2024-09-10T15:52:08.013397+0000","last_unstale":"2024-09-10T15:52:08.013909+0000","last_undegraded":"2024-09-10T15:52:08.013909+0000","last_fullsized":"2024-09-10T15:52:08.013909+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:05.852204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"24'5","reported_seq":40,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:08.015280+0000","last_change":"2024-09-10T15:52:08.015813+0000","last_active":"2024-09-10T15:52:08.015280+0000","last_peered":"2024-09-10T15:52:08.015280+0000","last_clean":"2024-09-10T15:51:57.078241+0000","last_became_active":"2024-09-10T15:52:08.015161+0000","last_became_peered":"2024-09-10T15:52:08.015161+0000","last_unstale":"2024-09-10T15:52:08.015280+0000","last_undegraded":"2024-09-10T15:52:08.007830+0000","last_fullsized":"2024-09-10T15:52:08.015280+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T15:52:08.016457+0000","last_change":"2024-09-10T15:52:08.015521+0000","last_active":"2024-09-10T15:52:08.016457+0000","last_peered":"2024-09-10T15:52:08.016457+0000","last_clean":"2024-09-10T15:52:08.016457+0000","last_became_active":"2024-09-10T15:52:08.014927+0000","last_became_peered":"2024-09-10T15:52:08.014927+0000","last_unstale":"2024-09-10T15:52:08.016457+0000","last_undegraded":"2024-09-10T15:52:08.016457+0000","last_fullsized":"2024-09-10T15:52:08.016457+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:37.483150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"24'2","reported_seq":36,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:08.018061+0000","last_change":"2024-09-10T15:52:08.018249+0000","last_active":"2024-09-10T15:52:08.018061+0000","last_peered":"2024-09-10T15:52:08.018061+0000","last_clean":"2024-09-10T15:51:45.893570+0000","last_became_active":"2024-09-10T15:52:08.017944+0000","last_became_peered":"2024-09-10T15:52:08.017944+0000","last_unstale":"2024-09-10T15:52:08.018061+0000","last_undegraded":"2024-09-10T15:52:08.008339+0000","last_fullsized":"2024-09-10T15:52:08.018061+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T15:52:08.014607+0000","last_change":"2024-09-10T15:52:08.014516+0000","last_active":"2024-09-10T15:52:08.014607+0000","last_peered":"2024-09-10T15:52:08.014607+0000","last_clean":"2024-09-10T15:52:08.014607+0000","last_became_active":"2024-09-10T15:52:08.014204+0000","last_became_peered":"2024-09-10T15:52:08.014204+0000","last_unstale":"2024-09-10T15:52:08.014607+0000","last_undegraded":"2024-09-10T15:52:08.014607+0000","last_fullsized":"2024-09-10T15:52:08.014607+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:33.040467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T15:52:08.014202+0000","last_change":"2024-09-10T15:52:08.014097+0000","last_active":"2024-09-10T15:52:08.014202+0000","last_peered":"2024-09-10T15:52:08.014202+0000","last_clean":"2024-09-10T15:52:08.014202+0000","last_became_active":"2024-09-10T15:52:08.013780+0000","last_became_peered":"2024-09-10T15:52:08.013780+0000","last_unstale":"2024-09-10T15:52:08.014202+0000","last_undegraded":"2024-09-10T15:52:08.014202+0000","last_fullsized":"2024-09-10T15:52:08.014202+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:44.526340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"24'9","reported_seq":44,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:08.015516+0000","last_change":"2024-09-10T15:52:08.016476+0000","last_active":"2024-09-10T15:52:08.015516+0000","last_peered":"2024-09-10T15:52:08.015516+0000","last_clean":"2024-09-10T15:51:56.837465+0000","last_became_active":"2024-09-10T15:52:08.015426+0000","last_became_peered":"2024-09-10T15:52:08.015426+0000","last_unstale":"2024-09-10T15:52:08.015516+0000","last_undegraded":"2024-09-10T15:52:08.008451+0000","last_fullsized":"2024-09-10T15:52:08.015516+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"0,1","objects":1},{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672976,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":79812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149699,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":268,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":274432,"data_stored":71118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28110509},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:52:14.638 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:52:14.984 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:14.984 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":60,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":27,"num_osds":3,"num_up_osds":2,"osd_up_since":1725983526,"num_in_osds":2,"osd_in_since":1725983504,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+degraded","count":12},{"state_name":"active+clean","count":10},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56426496,"bytes_avail":193217101824,"bytes_total":193273528320,"degraded_objects":27,"degraded_total":60,"degraded_ratio":0.45000000000000001},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-10T15:51:14.061210+0000","services":{}},"progress_events":{}} 2024-09-10T15:52:14.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '199e91a6-cca4-4c41-a5fa-011e971838b5', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded', 'count': 14}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 60, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 27, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1725983526, 'num_in_osds': 2, 'osd_in_since': 1725983504, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+degraded', 'count': 12}, {'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovering+degraded', 'count': 2}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56426496, 'bytes_avail': 193217101824, 'bytes_total': 193273528320, 'degraded_objects': 27, 'degraded_total': 60, 'degraded_ratio': 0.45}, 'fsmap': {'epoch': 1, 'btime': '2024-09-10T15:51:14:059711+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 1, 'modified': '2024-09-10T15:51:14.061210+0000', 'services': {}}, 'progress_events': {}} 2024-09-10T15:52:14.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-10T15:52:14.997 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T15:52:14.997 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T15:52:15.032 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T15:52:15.032 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-10T15:52:15.365 INFO:tasks.ceph.mon.c.smithi121.stderr:2024-09-10T15:52:15.364+0000 7fb1890bc640 -1 mon.c@2(leader).osd e27 definitely_dead 0 2024-09-10T15:52:16.030 INFO:teuthology.orchestra.run.smithi121.stderr:marked down osd.1. 2024-09-10T15:52:16.048 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-10T15:52:16.340 INFO:tasks.ceph.mon.c.smithi121.stderr:2024-09-10T15:52:16.339+0000 7fb1890bc640 -1 mon.c@2(leader).osd e28 definitely_dead 0 2024-09-10T15:52:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:marked out osd.1. 2024-09-10T15:52:17.050 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd lost 1 --yes-i-really-mean-it 2024-09-10T15:52:18.035 INFO:teuthology.orchestra.run.smithi121.stderr:marked osd lost in epoch 28 2024-09-10T15:52:18.039 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.036+0000 7fc1a59c6640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-10T15:52:18.039 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.036+0000 7fc1a51c5640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-10T15:52:18.039 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.036+0000 7fc1a81cb640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-10T15:52:18.039 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.036+0000 7fc1a99ce640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-10T15:52:18.039 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.036+0000 7fc1a39c2640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T15:52:18.039 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.036+0000 7fc1a89cc640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T15:52:18.039 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.036+0000 7fc1a49c4640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T15:52:18.039 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.036+0000 7fc1a91cd640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T15:52:18.039 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.036+0000 7fc1a81cb640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-10T15:52:18.039 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.037+0000 7fc1a31c1640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-10T15:52:18.040 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.037+0000 7fc1a71c9640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T15:52:18.040 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.037+0000 7fc1a69c8640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-10T15:52:18.040 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.037+0000 7fc1a29c0640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T15:52:18.040 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:18.037+0000 7fc1a61c7640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T15:52:18.050 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T15:52:18.050 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T15:52:18.053 INFO:tasks.ceph.osd.2:Started 2024-09-10T15:52:18.053 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T15:52:18.153 INFO:teuthology.orchestra.run.smithi121.stderr:no valid command found; 10 closest matches: 2024-09-10T15:52:18.153 INFO:teuthology.orchestra.run.smithi121.stderr:0 2024-09-10T15:52:18.153 INFO:teuthology.orchestra.run.smithi121.stderr:1 2024-09-10T15:52:18.153 INFO:teuthology.orchestra.run.smithi121.stderr:2 2024-09-10T15:52:18.153 INFO:teuthology.orchestra.run.smithi121.stderr:abort 2024-09-10T15:52:18.153 INFO:teuthology.orchestra.run.smithi121.stderr:assert 2024-09-10T15:52:18.153 INFO:teuthology.orchestra.run.smithi121.stderr:config diff 2024-09-10T15:52:18.153 INFO:teuthology.orchestra.run.smithi121.stderr:config diff get 2024-09-10T15:52:18.153 INFO:teuthology.orchestra.run.smithi121.stderr:config get 2024-09-10T15:52:18.153 INFO:teuthology.orchestra.run.smithi121.stderr:config help [] 2024-09-10T15:52:18.153 INFO:teuthology.orchestra.run.smithi121.stderr:config set ... 2024-09-10T15:52:18.154 INFO:teuthology.orchestra.run.smithi121.stderr:admin_socket: invalid command 2024-09-10T15:52:18.155 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T15:52:18.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-10T15:52:19.460 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-09-10T15:52:19.458+0000 7f36cc3c5640 -1 Falling back to public interface 2024-09-10T15:52:23.156 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T15:52:23.257 INFO:teuthology.orchestra.run.smithi121.stderr:no valid command found; 10 closest matches: 2024-09-10T15:52:23.258 INFO:teuthology.orchestra.run.smithi121.stderr:0 2024-09-10T15:52:23.258 INFO:teuthology.orchestra.run.smithi121.stderr:1 2024-09-10T15:52:23.258 INFO:teuthology.orchestra.run.smithi121.stderr:2 2024-09-10T15:52:23.258 INFO:teuthology.orchestra.run.smithi121.stderr:abort 2024-09-10T15:52:23.258 INFO:teuthology.orchestra.run.smithi121.stderr:assert 2024-09-10T15:52:23.258 INFO:teuthology.orchestra.run.smithi121.stderr:config diff 2024-09-10T15:52:23.258 INFO:teuthology.orchestra.run.smithi121.stderr:config diff get 2024-09-10T15:52:23.258 INFO:teuthology.orchestra.run.smithi121.stderr:config get 2024-09-10T15:52:23.258 INFO:teuthology.orchestra.run.smithi121.stderr:config help [] 2024-09-10T15:52:23.258 INFO:teuthology.orchestra.run.smithi121.stderr:config set ... 2024-09-10T15:52:23.258 INFO:teuthology.orchestra.run.smithi121.stderr:admin_socket: invalid command 2024-09-10T15:52:23.259 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T15:52:23.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-10T15:52:23.718 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-09-10T15:52:23.716+0000 7f36cc3c5640 -1 osd.2 18 log_to_monitors true 2024-09-10T15:52:24.101 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.099+0000 7fc1a59c6640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-10T15:52:24.102 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.100+0000 7fc1a99ce640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-10T15:52:24.102 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.100+0000 7fc1a91cd640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-10T15:52:24.102 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.100+0000 7fc1a61c7640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T15:52:24.102 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.100+0000 7fc1a91cd640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T15:52:24.103 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.100+0000 7fc1a89cc640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T15:52:24.103 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.100+0000 7fc1a69c8640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-10T15:52:24.103 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.100+0000 7fc1a29c0640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T15:52:24.103 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.100+0000 7fc1a89cc640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T15:52:24.103 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.101+0000 7fc1a81cb640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-10T15:52:24.103 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.101+0000 7fc1a31c1640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-10T15:52:24.103 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.101+0000 7fc1a41c3640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-10T15:52:24.103 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.101+0000 7fc1a39c2640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T15:52:24.103 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:24.101+0000 7fc1a31c1640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T15:52:24.111 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-09-10T15:52:24.110+0000 7f36c417b640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:52:25.111 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.108+0000 7fc1a99ce640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-10T15:52:25.111 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.108+0000 7fc1a59c6640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-10T15:52:25.112 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.108+0000 7fc1a51c5640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-10T15:52:25.112 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.108+0000 7fc1a51c5640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T15:52:25.112 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.108+0000 7fc1a39c2640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T15:52:25.112 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.109+0000 7fc1a21bf640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T15:52:25.112 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.109+0000 7fc1a69c8640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-10T15:52:25.112 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.109+0000 7fc1a49c4640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T15:52:25.112 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.109+0000 7fc1a69c8640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T15:52:25.112 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.109+0000 7fc1a89cc640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T15:52:25.112 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.109+0000 7fc1a81cb640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-10T15:52:25.112 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.109+0000 7fc1a71c9640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-10T15:52:25.112 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.109+0000 7fc1a31c1640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T15:52:25.112 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:25.109+0000 7fc1a81cb640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-10T15:52:26.126 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:26.124+0000 7fc1a89cc640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T15:52:26.126 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:26.124+0000 7fc1a49c4640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T15:52:26.128 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:26.125+0000 7fc1a29c0640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T15:52:26.128 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:26.125+0000 7fc1a21bf640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T15:52:26.128 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:26.125+0000 7fc1a79ca640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T15:52:26.129 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:26.127+0000 7fc1a91cd640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T15:52:26.129 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:26.127+0000 7fc1a31c1640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T15:52:27.131 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:27.129+0000 7fc1a49c4640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T15:52:27.132 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:27.130+0000 7fc1a49c4640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T15:52:27.132 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:27.130+0000 7fc1a79ca640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T15:52:27.132 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:27.130+0000 7fc1a51c5640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T15:52:27.133 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:27.131+0000 7fc1a71c9640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T15:52:27.133 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:27.131+0000 7fc1a21bf640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T15:52:27.133 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:27.132+0000 7fc1a29c0640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T15:52:28.260 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-10T15:52:28.376 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:28.376 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-09-10T15:52:28.377 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-09-10T15:52:28.377 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:28.385 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-10T15:52:28.675 INFO:tasks.ceph.mon.c.smithi121.stderr:2024-09-10T15:52:28.674+0000 7fb1890bc640 -1 mon.c@2(leader).osd e34 definitely_dead 0 2024-09-10T15:52:29.131 INFO:teuthology.orchestra.run.smithi121.stderr:marked in osd.2. 2024-09-10T15:52:29.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.2 to be up 2024-09-10T15:52:29.151 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:52:29.433 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:29.433 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":35,"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","created":"2024-09-10T15:51:14.060262+0000","modified":"2024-09-10T15:52:29.127296+0000","last_up_change":"2024-09-10T15:52:25.105731+0000","last_in_change":"2024-09-10T15:52:29.127296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:51:29.314004+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-10T15:51:40.401037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"53b58107-009b-46ac-921d-2e8d9580081e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":22,"up_from":26,"up_thru":33,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6800","nonce":824549766}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6801","nonce":824549766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6803","nonce":824549766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6802","nonce":824549766}]},"public_addr":"172.21.15.121:6800/824549766","cluster_addr":"172.21.15.121:6801/824549766","heartbeat_back_addr":"172.21.15.121:6803/824549766","heartbeat_front_addr":"172.21.15.121:6802/824549766","state":["exists","up"]},{"osd":1,"uuid":"aced9681-6a62-49a4-a1e2-6ff7d9a544ef","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":26,"down_at":28,"lost_at":28,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6804","nonce":3301493939}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6805","nonce":3301493939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6807","nonce":3301493939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6806","nonce":3301493939}]},"public_addr":"172.21.15.121:6804/3301493939","cluster_addr":"172.21.15.121:6805/3301493939","heartbeat_back_addr":"172.21.15.121:6807/3301493939","heartbeat_front_addr":"172.21.15.121:6806/3301493939","state":["exists"]},{"osd":2,"uuid":"0cfba235-9896-429e-8396-69dfeb7a44bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":18,"up_from":32,"up_thru":17,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6804","nonce":4255745125}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6805","nonce":4255745125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6807","nonce":4255745125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6806","nonce":4255745125}]},"public_addr":"172.21.15.121:6804/4255745125","cluster_addr":"172.21.15.121:6805/4255745125","heartbeat_back_addr":"172.21.15.121:6807/4255745125","heartbeat_front_addr":"172.21.15.121:6806/4255745125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:22.164346+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-10T15:51:21.644539+0000","dead_epoch":28},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.413080+0000","dead_epoch":20}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.1","osds":[0,2]},{"pgid":"1.3","osds":[0,2]},{"pgid":"1.5","osds":[0,2]},{"pgid":"1.7","osds":[0,2]},{"pgid":"2.0","osds":[0,2]},{"pgid":"2.1","osds":[0,2]},{"pgid":"2.3","osds":[0,2]},{"pgid":"2.c","osds":[0,2]},{"pgid":"2.d","osds":[0,2]},{"pgid":"2.e","osds":[0,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:52:29.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.2 is up 2024-09-10T15:52:29.446 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:52:29.446 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T15:52:29.578 INFO:teuthology.orchestra.run.smithi121.stdout:111669149703 2024-09-10T15:52:29.578 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:52:29.579 INFO:teuthology.orchestra.run.smithi121.stdout:137438953475 2024-09-10T15:52:29.579 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:52:29.915 INFO:teuthology.orchestra.run.smithi121.stdout:111669149702 2024-09-10T15:52:29.916 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-09-10T15:52:29.928 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149703 got 111669149702 for osd.0 2024-09-10T15:52:29.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 137438953475 got 1 for osd.2 2024-09-10T15:52:30.929 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:52:30.930 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:52:31.151 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:31.148+0000 7fc1a79ca640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-10T15:52:31.151 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:31.149+0000 7fc1a29c0640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-10T15:52:31.152 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:31.149+0000 7fc1a21bf640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-10T15:52:31.152 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:31.150+0000 7fc1a89cc640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-10T15:52:31.152 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:31.150+0000 7fc1a89cc640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-10T15:52:31.152 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:31.150+0000 7fc1a59c6640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-10T15:52:31.153 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:31.151+0000 7fc1a51c5640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-10T15:52:31.153 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:31.151+0000 7fc1a81cb640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-10T15:52:31.153 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-09-10T15:52:31.151+0000 7fc1a71c9640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-10T15:52:31.244 INFO:teuthology.orchestra.run.smithi121.stdout:111669149703 2024-09-10T15:52:31.244 INFO:teuthology.orchestra.run.smithi121.stdout:137438953475 2024-09-10T15:52:31.258 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149703 got 111669149703 for osd.0 2024-09-10T15:52:31.258 DEBUG:teuthology.parallel:result is None 2024-09-10T15:52:31.258 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 137438953475 got 137438953475 for osd.2 2024-09-10T15:52:31.259 DEBUG:teuthology.parallel:result is None 2024-09-10T15:52:31.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-10T15:52:31.259 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:52:31.537 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:31.538 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:52:31.549 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-10T15:52:31.147813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":18,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55564,"kb_used_data":620,"kb_used_omap":8,"kb_used_meta":54903,"kb_avail":188688116,"statfs":{"total":193273528320,"available":193216630784,"internally_reserved":0,"allocated":634880,"data_stored":186633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":56221015},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-21,"num_objects_misplaced":0,"num_objects_unfound":-9,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.721589"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":49,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:52:30.154448+0000","last_change":"2024-09-10T15:52:30.154389+0000","last_active":"2024-09-10T15:52:30.154448+0000","last_peered":"2024-09-10T15:52:30.154448+0000","last_clean":"2024-09-10T15:52:30.154448+0000","last_became_active":"2024-09-10T15:52:30.154140+0000","last_became_peered":"2024-09-10T15:52:30.154140+0000","last_unstale":"2024-09-10T15:52:30.154448+0000","last_undegraded":"2024-09-10T15:52:30.154448+0000","last_fullsized":"2024-09-10T15:52:30.154448+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:30.595214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'4","reported_seq":50,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-10T15:52:27.138732+0000","last_change":"2024-09-10T15:52:29.132879+0000","last_active":"2024-09-10T15:52:29.132879+0000","last_peered":"2024-09-10T15:52:27.138732+0000","last_clean":"2024-09-10T15:51:56.324820+0000","last_became_active":"2024-09-10T15:52:27.138428+0000","last_became_peered":"2024-09-10T15:52:27.138428+0000","last_unstale":"2024-09-10T15:52:27.138732+0000","last_undegraded":"2024-09-10T15:52:27.133054+0000","last_fullsized":"2024-09-10T15:52:27.138732+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'3","reported_seq":60,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-10T15:52:27.138422+0000","last_change":"2024-09-10T15:52:29.136977+0000","last_active":"2024-09-10T15:52:29.136977+0000","last_peered":"2024-09-10T15:52:27.138422+0000","last_clean":"2024-09-10T15:51:56.935537+0000","last_became_active":"2024-09-10T15:52:27.138139+0000","last_became_peered":"2024-09-10T15:52:27.138139+0000","last_unstale":"2024-09-10T15:52:27.138422+0000","last_undegraded":"2024-09-10T15:52:27.133013+0000","last_fullsized":"2024-09-10T15:52:27.138422+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":54,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-10T15:52:27.136904+0000","last_change":"2024-09-10T15:52:29.134823+0000","last_active":"2024-09-10T15:52:29.134823+0000","last_peered":"2024-09-10T15:52:27.136904+0000","last_clean":"2024-09-10T15:52:27.136904+0000","last_became_active":"2024-09-10T15:52:27.135356+0000","last_became_peered":"2024-09-10T15:52:27.135356+0000","last_unstale":"2024-09-10T15:52:27.136904+0000","last_undegraded":"2024-09-10T15:52:27.136904+0000","last_fullsized":"2024-09-10T15:52:27.136904+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"24'5","reported_seq":63,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.151998+0000","last_change":"2024-09-10T15:52:30.152160+0000","last_active":"2024-09-10T15:52:30.151998+0000","last_peered":"2024-09-10T15:52:30.151998+0000","last_clean":"2024-09-10T15:51:56.229374+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:52:30.151998+0000","last_undegraded":"2024-09-10T15:52:30.147386+0000","last_fullsized":"2024-09-10T15:52:30.151998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"24'3","reported_seq":59,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.152412+0000","last_change":"2024-09-10T15:52:30.152575+0000","last_active":"2024-09-10T15:52:30.152412+0000","last_peered":"2024-09-10T15:52:30.152412+0000","last_clean":"2024-09-10T15:51:57.126582+0000","last_became_active":"2024-09-10T15:52:30.152332+0000","last_became_peered":"2024-09-10T15:52:30.152332+0000","last_unstale":"2024-09-10T15:52:30.152412+0000","last_undegraded":"2024-09-10T15:52:30.147849+0000","last_fullsized":"2024-09-10T15:52:30.152412+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"24'4","reported_seq":60,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.157008+0000","last_change":"2024-09-10T15:52:30.157171+0000","last_active":"2024-09-10T15:52:30.157008+0000","last_peered":"2024-09-10T15:52:30.157008+0000","last_clean":"2024-09-10T15:51:56.469065+0000","last_became_active":"2024-09-10T15:52:30.156955+0000","last_became_peered":"2024-09-10T15:52:30.156955+0000","last_unstale":"2024-09-10T15:52:30.157008+0000","last_undegraded":"2024-09-10T15:52:30.148233+0000","last_fullsized":"2024-09-10T15:52:30.157008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"24'1","reported_seq":60,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.152350+0000","last_change":"2024-09-10T15:52:30.152559+0000","last_active":"2024-09-10T15:52:30.152350+0000","last_peered":"2024-09-10T15:52:30.152350+0000","last_clean":"2024-09-10T15:51:42.870214+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:52:30.152350+0000","last_undegraded":"2024-09-10T15:52:30.147342+0000","last_fullsized":"2024-09-10T15:52:30.152350+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:52:30.152711+0000","last_change":"2024-09-10T15:52:30.152637+0000","last_active":"2024-09-10T15:52:30.152711+0000","last_peered":"2024-09-10T15:52:30.152711+0000","last_clean":"2024-09-10T15:52:30.152711+0000","last_became_active":"2024-09-10T15:52:30.151730+0000","last_became_peered":"2024-09-10T15:52:30.151730+0000","last_unstale":"2024-09-10T15:52:30.152711+0000","last_undegraded":"2024-09-10T15:52:30.152711+0000","last_fullsized":"2024-09-10T15:52:30.152711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:09.577308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"24'5","reported_seq":62,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-10T15:52:27.137206+0000","last_change":"2024-09-10T15:52:29.136362+0000","last_active":"2024-09-10T15:52:29.136362+0000","last_peered":"2024-09-10T15:52:27.137206+0000","last_clean":"2024-09-10T15:51:56.983672+0000","last_became_active":"2024-09-10T15:52:27.136884+0000","last_became_peered":"2024-09-10T15:52:27.136884+0000","last_unstale":"2024-09-10T15:52:27.137206+0000","last_undegraded":"2024-09-10T15:52:27.131775+0000","last_fullsized":"2024-09-10T15:52:27.137206+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":68,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-10T15:52:27.140177+0000","last_change":"2024-09-10T15:52:29.133665+0000","last_active":"2024-09-10T15:52:29.133665+0000","last_peered":"2024-09-10T15:52:27.140177+0000","last_clean":"2024-09-10T15:52:27.140177+0000","last_became_active":"2024-09-10T15:52:27.138284+0000","last_became_peered":"2024-09-10T15:52:27.138284+0000","last_unstale":"2024-09-10T15:52:27.140177+0000","last_undegraded":"2024-09-10T15:52:27.140177+0000","last_fullsized":"2024-09-10T15:52:27.140177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"24'6","reported_seq":55,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.152188+0000","last_change":"2024-09-10T15:52:30.152404+0000","last_active":"2024-09-10T15:52:30.152188+0000","last_peered":"2024-09-10T15:52:30.152188+0000","last_clean":"2024-09-10T15:51:57.174499+0000","last_became_active":"2024-09-10T15:52:30.152112+0000","last_became_peered":"2024-09-10T15:52:30.152112+0000","last_unstale":"2024-09-10T15:52:30.152188+0000","last_undegraded":"2024-09-10T15:52:30.147626+0000","last_fullsized":"2024-09-10T15:52:30.152188+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":62,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.151796+0000","last_change":"2024-09-10T15:52:30.152323+0000","last_active":"2024-09-10T15:52:30.151796+0000","last_peered":"2024-09-10T15:52:30.151796+0000","last_clean":"2024-09-10T15:52:08.013728+0000","last_became_active":"2024-09-10T15:52:30.151717+0000","last_became_peered":"2024-09-10T15:52:30.151717+0000","last_unstale":"2024-09-10T15:52:30.151796+0000","last_undegraded":"2024-09-10T15:52:30.144026+0000","last_fullsized":"2024-09-10T15:52:30.151796+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"24'3","reported_seq":56,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-10T15:52:27.137390+0000","last_change":"2024-09-10T15:52:29.136209+0000","last_active":"2024-09-10T15:52:29.136209+0000","last_peered":"2024-09-10T15:52:27.137390+0000","last_clean":"2024-09-10T15:51:56.181288+0000","last_became_active":"2024-09-10T15:52:27.137042+0000","last_became_peered":"2024-09-10T15:52:27.137042+0000","last_unstale":"2024-09-10T15:52:27.137390+0000","last_undegraded":"2024-09-10T15:52:27.131243+0000","last_fullsized":"2024-09-10T15:52:27.137390+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-10T15:52:27.140202+0000","last_change":"2024-09-10T15:52:29.135638+0000","last_active":"2024-09-10T15:52:29.135638+0000","last_peered":"2024-09-10T15:52:27.140202+0000","last_clean":"2024-09-10T15:52:27.140202+0000","last_became_active":"2024-09-10T15:52:27.138133+0000","last_became_peered":"2024-09-10T15:52:27.138133+0000","last_unstale":"2024-09-10T15:52:27.140202+0000","last_undegraded":"2024-09-10T15:52:27.140202+0000","last_fullsized":"2024-09-10T15:52:27.140202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"24'4","reported_seq":57,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-10T15:52:27.136727+0000","last_change":"2024-09-10T15:52:29.135946+0000","last_active":"2024-09-10T15:52:29.135946+0000","last_peered":"2024-09-10T15:52:27.136727+0000","last_clean":"2024-09-10T15:51:56.644471+0000","last_became_active":"2024-09-10T15:52:27.136415+0000","last_became_peered":"2024-09-10T15:52:27.136415+0000","last_unstale":"2024-09-10T15:52:27.136727+0000","last_undegraded":"2024-09-10T15:52:27.131706+0000","last_fullsized":"2024-09-10T15:52:27.136727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'1","reported_seq":57,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-10T15:52:30.152071+0000","last_change":"2024-09-10T15:52:30.153333+0000","last_active":"2024-09-10T15:52:30.152071+0000","last_peered":"2024-09-10T15:52:30.152071+0000","last_clean":"2024-09-10T15:51:45.893657+0000","last_became_active":"2024-09-10T15:52:30.151806+0000","last_became_peered":"2024-09-10T15:52:30.151806+0000","last_unstale":"2024-09-10T15:52:30.152071+0000","last_undegraded":"2024-09-10T15:52:30.147601+0000","last_fullsized":"2024-09-10T15:52:30.152071+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:52:30.154476+0000","last_change":"2024-09-10T15:52:30.154393+0000","last_active":"2024-09-10T15:52:30.154476+0000","last_peered":"2024-09-10T15:52:30.154476+0000","last_clean":"2024-09-10T15:52:30.154476+0000","last_became_active":"2024-09-10T15:52:30.154111+0000","last_became_peered":"2024-09-10T15:52:30.154111+0000","last_unstale":"2024-09-10T15:52:30.154476+0000","last_undegraded":"2024-09-10T15:52:30.154476+0000","last_fullsized":"2024-09-10T15:52:30.154476+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:53.944111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"24'5","reported_seq":61,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.159727+0000","last_change":"2024-09-10T15:52:30.159885+0000","last_active":"2024-09-10T15:52:30.159727+0000","last_peered":"2024-09-10T15:52:30.159727+0000","last_clean":"2024-09-10T15:51:57.078241+0000","last_became_active":"2024-09-10T15:52:30.159635+0000","last_became_peered":"2024-09-10T15:52:30.159635+0000","last_unstale":"2024-09-10T15:52:30.159727+0000","last_undegraded":"2024-09-10T15:52:30.148722+0000","last_fullsized":"2024-09-10T15:52:30.159727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":68,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-10T15:52:27.139035+0000","last_change":"2024-09-10T15:52:29.135252+0000","last_active":"2024-09-10T15:52:29.135252+0000","last_peered":"2024-09-10T15:52:27.139035+0000","last_clean":"2024-09-10T15:52:27.139035+0000","last_became_active":"2024-09-10T15:52:27.136617+0000","last_became_peered":"2024-09-10T15:52:27.136617+0000","last_unstale":"2024-09-10T15:52:27.139035+0000","last_undegraded":"2024-09-10T15:52:27.139035+0000","last_fullsized":"2024-09-10T15:52:27.139035+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"24'2","reported_seq":57,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.159625+0000","last_change":"2024-09-10T15:52:30.159788+0000","last_active":"2024-09-10T15:52:30.159625+0000","last_peered":"2024-09-10T15:52:30.159625+0000","last_clean":"2024-09-10T15:51:45.893570+0000","last_became_active":"2024-09-10T15:52:30.159560+0000","last_became_peered":"2024-09-10T15:52:30.159560+0000","last_unstale":"2024-09-10T15:52:30.159625+0000","last_undegraded":"2024-09-10T15:52:30.148492+0000","last_fullsized":"2024-09-10T15:52:30.159625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:52:30.149362+0000","last_change":"2024-09-10T15:52:30.149270+0000","last_active":"2024-09-10T15:52:30.149362+0000","last_peered":"2024-09-10T15:52:30.149362+0000","last_clean":"2024-09-10T15:52:30.149362+0000","last_became_active":"2024-09-10T15:52:30.148972+0000","last_became_peered":"2024-09-10T15:52:30.148972+0000","last_unstale":"2024-09-10T15:52:30.149362+0000","last_undegraded":"2024-09-10T15:52:30.149362+0000","last_fullsized":"2024-09-10T15:52:30.149362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:21.166160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-10T15:52:27.141037+0000","last_change":"2024-09-10T15:52:29.135292+0000","last_active":"2024-09-10T15:52:29.135292+0000","last_peered":"2024-09-10T15:52:27.141037+0000","last_clean":"2024-09-10T15:52:27.141037+0000","last_became_active":"2024-09-10T15:52:27.138277+0000","last_became_peered":"2024-09-10T15:52:27.138277+0000","last_unstale":"2024-09-10T15:52:27.141037+0000","last_undegraded":"2024-09-10T15:52:27.141037+0000","last_fullsized":"2024-09-10T15:52:27.141037+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"24'9","reported_seq":66,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.152168+0000","last_change":"2024-09-10T15:52:30.152339+0000","last_active":"2024-09-10T15:52:30.152168+0000","last_peered":"2024-09-10T15:52:30.152168+0000","last_clean":"2024-09-10T15:51:56.837465+0000","last_became_active":"2024-09-10T15:52:30.152069+0000","last_became_peered":"2024-09-10T15:52:30.152069+0000","last_unstale":"2024-09-10T15:52:30.152168+0000","last_undegraded":"2024-09-10T15:52:30.148182+0000","last_fullsized":"2024-09-10T15:52:30.152168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":18,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149704,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":340,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":348160,"data_stored":98872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28110509},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953475,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":280,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":286720,"data_stored":87761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:52:31.550 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:52:31.897 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:31.897 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":77,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":37,"num_osds":3,"num_up_osds":2,"osd_up_since":1725983545,"num_in_osds":2,"osd_in_since":1725983549,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":14},{"state_name":"remapped+peering","count":10}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56889344,"bytes_avail":193216638976,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-10T15:52:20.424864+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:52:31.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '199e91a6-cca4-4c41-a5fa-011e971838b5', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 77, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 37, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1725983545, 'num_in_osds': 2, 'osd_in_since': 1725983549, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 14}, {'state_name': 'remapped+peering', 'count': 10}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56889344, 'bytes_avail': 193216638976, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'btime': '2024-09-10T15:51:14:059711+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 2, 'modified': '2024-09-10T15:52:20.424864+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '1': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-10T15:52:34.911 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:52:35.150 INFO:tasks.ceph.mon.c.smithi121.stderr:2024-09-10T15:52:35.148+0000 7fb18b8c1640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 5 pgs recovery_unfound (PG_DAMAGED) 2024-09-10T15:52:35.207 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:35.208 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:52:35.220 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-10T15:52:34.427545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55588,"kb_used_data":644,"kb_used_omap":8,"kb_used_meta":54903,"kb_avail":188688092,"statfs":{"total":193273528320,"available":193216606208,"internally_reserved":0,"allocated":659456,"data_stored":195604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":56221015},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000900"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":49,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:52:30.154448+0000","last_change":"2024-09-10T15:52:30.154389+0000","last_active":"2024-09-10T15:52:30.154448+0000","last_peered":"2024-09-10T15:52:30.154448+0000","last_clean":"2024-09-10T15:52:30.154448+0000","last_became_active":"2024-09-10T15:52:30.154140+0000","last_became_peered":"2024-09-10T15:52:30.154140+0000","last_unstale":"2024-09-10T15:52:30.154448+0000","last_undegraded":"2024-09-10T15:52:30.154448+0000","last_fullsized":"2024-09-10T15:52:30.154448+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:30.595214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'4","reported_seq":56,"reported_epoch":37,"state":"active+recovery_unfound","last_fresh":"2024-09-10T15:52:31.158585+0000","last_change":"2024-09-10T15:52:31.158841+0000","last_active":"2024-09-10T15:52:31.158585+0000","last_peered":"2024-09-10T15:52:31.158585+0000","last_clean":"2024-09-10T15:51:56.324820+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:52:31.158585+0000","last_undegraded":"2024-09-10T15:52:31.151645+0000","last_fullsized":"2024-09-10T15:52:31.158585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"24'3","reported_seq":67,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:52:31.161688+0000","last_change":"2024-09-10T15:52:34.107356+0000","last_active":"2024-09-10T15:52:31.161688+0000","last_peered":"2024-09-10T15:52:31.161688+0000","last_clean":"2024-09-10T15:51:56.935537+0000","last_became_active":"2024-09-10T15:52:31.157749+0000","last_became_peered":"2024-09-10T15:52:31.157749+0000","last_unstale":"2024-09-10T15:52:31.161688+0000","last_undegraded":"2024-09-10T15:52:31.161688+0000","last_fullsized":"2024-09-10T15:52:31.161688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":60,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.156866+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:52:31.156866+0000","last_peered":"2024-09-10T15:52:31.156866+0000","last_clean":"2024-09-10T15:52:31.156866+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:52:31.156866+0000","last_undegraded":"2024-09-10T15:52:31.156866+0000","last_fullsized":"2024-09-10T15:52:31.156866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"24'5","reported_seq":63,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.151998+0000","last_change":"2024-09-10T15:52:30.152160+0000","last_active":"2024-09-10T15:52:30.151998+0000","last_peered":"2024-09-10T15:52:30.151998+0000","last_clean":"2024-09-10T15:51:56.229374+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:52:30.151998+0000","last_undegraded":"2024-09-10T15:52:30.147386+0000","last_fullsized":"2024-09-10T15:52:30.151998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"24'3","reported_seq":59,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.152412+0000","last_change":"2024-09-10T15:52:30.152575+0000","last_active":"2024-09-10T15:52:30.152412+0000","last_peered":"2024-09-10T15:52:30.152412+0000","last_clean":"2024-09-10T15:51:57.126582+0000","last_became_active":"2024-09-10T15:52:30.152332+0000","last_became_peered":"2024-09-10T15:52:30.152332+0000","last_unstale":"2024-09-10T15:52:30.152412+0000","last_undegraded":"2024-09-10T15:52:30.147849+0000","last_fullsized":"2024-09-10T15:52:30.152412+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"24'4","reported_seq":60,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.157008+0000","last_change":"2024-09-10T15:52:30.157171+0000","last_active":"2024-09-10T15:52:30.157008+0000","last_peered":"2024-09-10T15:52:30.157008+0000","last_clean":"2024-09-10T15:51:56.469065+0000","last_became_active":"2024-09-10T15:52:30.156955+0000","last_became_peered":"2024-09-10T15:52:30.156955+0000","last_unstale":"2024-09-10T15:52:30.157008+0000","last_undegraded":"2024-09-10T15:52:30.148233+0000","last_fullsized":"2024-09-10T15:52:30.157008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"24'1","reported_seq":60,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.152350+0000","last_change":"2024-09-10T15:52:30.152559+0000","last_active":"2024-09-10T15:52:30.152350+0000","last_peered":"2024-09-10T15:52:30.152350+0000","last_clean":"2024-09-10T15:51:42.870214+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:52:30.152350+0000","last_undegraded":"2024-09-10T15:52:30.147342+0000","last_fullsized":"2024-09-10T15:52:30.152350+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:52:30.152711+0000","last_change":"2024-09-10T15:52:30.152637+0000","last_active":"2024-09-10T15:52:30.152711+0000","last_peered":"2024-09-10T15:52:30.152711+0000","last_clean":"2024-09-10T15:52:30.152711+0000","last_became_active":"2024-09-10T15:52:30.151730+0000","last_became_peered":"2024-09-10T15:52:30.151730+0000","last_unstale":"2024-09-10T15:52:30.152711+0000","last_undegraded":"2024-09-10T15:52:30.152711+0000","last_fullsized":"2024-09-10T15:52:30.152711+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:09.577308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"24'5","reported_seq":69,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:52:31.162204+0000","last_change":"2024-09-10T15:52:34.107284+0000","last_active":"2024-09-10T15:52:31.162204+0000","last_peered":"2024-09-10T15:52:31.162204+0000","last_clean":"2024-09-10T15:51:56.983672+0000","last_became_active":"2024-09-10T15:52:31.157754+0000","last_became_peered":"2024-09-10T15:52:31.157754+0000","last_unstale":"2024-09-10T15:52:31.162204+0000","last_undegraded":"2024-09-10T15:52:31.162204+0000","last_fullsized":"2024-09-10T15:52:31.162204+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.160328+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:52:31.160328+0000","last_peered":"2024-09-10T15:52:31.160328+0000","last_clean":"2024-09-10T15:52:31.160328+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:52:31.160328+0000","last_undegraded":"2024-09-10T15:52:31.160328+0000","last_fullsized":"2024-09-10T15:52:31.160328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"24'6","reported_seq":55,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.152188+0000","last_change":"2024-09-10T15:52:30.152404+0000","last_active":"2024-09-10T15:52:30.152188+0000","last_peered":"2024-09-10T15:52:30.152188+0000","last_clean":"2024-09-10T15:51:57.174499+0000","last_became_active":"2024-09-10T15:52:30.152112+0000","last_became_peered":"2024-09-10T15:52:30.152112+0000","last_unstale":"2024-09-10T15:52:30.152188+0000","last_undegraded":"2024-09-10T15:52:30.147626+0000","last_fullsized":"2024-09-10T15:52:30.152188+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":62,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.151796+0000","last_change":"2024-09-10T15:52:30.152323+0000","last_active":"2024-09-10T15:52:30.151796+0000","last_peered":"2024-09-10T15:52:30.151796+0000","last_clean":"2024-09-10T15:52:08.013728+0000","last_became_active":"2024-09-10T15:52:30.151717+0000","last_became_peered":"2024-09-10T15:52:30.151717+0000","last_unstale":"2024-09-10T15:52:30.151796+0000","last_undegraded":"2024-09-10T15:52:30.144026+0000","last_fullsized":"2024-09-10T15:52:30.151796+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"24'3","reported_seq":62,"reported_epoch":37,"state":"active+recovery_unfound","last_fresh":"2024-09-10T15:52:31.160969+0000","last_change":"2024-09-10T15:52:31.161142+0000","last_active":"2024-09-10T15:52:31.160969+0000","last_peered":"2024-09-10T15:52:31.160969+0000","last_clean":"2024-09-10T15:51:56.181288+0000","last_became_active":"2024-09-10T15:52:31.157172+0000","last_became_peered":"2024-09-10T15:52:31.157172+0000","last_unstale":"2024-09-10T15:52:31.160969+0000","last_undegraded":"2024-09-10T15:52:31.152185+0000","last_fullsized":"2024-09-10T15:52:31.160969+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":91,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.161559+0000","last_change":"2024-09-10T15:52:31.159524+0000","last_active":"2024-09-10T15:52:31.161559+0000","last_peered":"2024-09-10T15:52:31.161559+0000","last_clean":"2024-09-10T15:52:31.161559+0000","last_became_active":"2024-09-10T15:52:31.159205+0000","last_became_peered":"2024-09-10T15:52:31.159205+0000","last_unstale":"2024-09-10T15:52:31.161559+0000","last_undegraded":"2024-09-10T15:52:31.161559+0000","last_fullsized":"2024-09-10T15:52:31.161559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:04.415297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"24'4","reported_seq":63,"reported_epoch":37,"state":"active+recovery_unfound","last_fresh":"2024-09-10T15:52:31.160503+0000","last_change":"2024-09-10T15:52:31.160690+0000","last_active":"2024-09-10T15:52:31.160503+0000","last_peered":"2024-09-10T15:52:31.160503+0000","last_clean":"2024-09-10T15:51:56.644471+0000","last_became_active":"2024-09-10T15:52:31.157034+0000","last_became_peered":"2024-09-10T15:52:31.157034+0000","last_unstale":"2024-09-10T15:52:31.160503+0000","last_undegraded":"2024-09-10T15:52:31.149786+0000","last_fullsized":"2024-09-10T15:52:31.160503+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"24'1","reported_seq":57,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-10T15:52:30.152071+0000","last_change":"2024-09-10T15:52:30.153333+0000","last_active":"2024-09-10T15:52:30.152071+0000","last_peered":"2024-09-10T15:52:30.152071+0000","last_clean":"2024-09-10T15:51:45.893657+0000","last_became_active":"2024-09-10T15:52:30.151806+0000","last_became_peered":"2024-09-10T15:52:30.151806+0000","last_unstale":"2024-09-10T15:52:30.152071+0000","last_undegraded":"2024-09-10T15:52:30.147601+0000","last_fullsized":"2024-09-10T15:52:30.152071+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:52:30.154476+0000","last_change":"2024-09-10T15:52:30.154393+0000","last_active":"2024-09-10T15:52:30.154476+0000","last_peered":"2024-09-10T15:52:30.154476+0000","last_clean":"2024-09-10T15:52:30.154476+0000","last_became_active":"2024-09-10T15:52:30.154111+0000","last_became_peered":"2024-09-10T15:52:30.154111+0000","last_unstale":"2024-09-10T15:52:30.154476+0000","last_undegraded":"2024-09-10T15:52:30.154476+0000","last_fullsized":"2024-09-10T15:52:30.154476+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:53.944111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"24'5","reported_seq":61,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.159727+0000","last_change":"2024-09-10T15:52:30.159885+0000","last_active":"2024-09-10T15:52:30.159727+0000","last_peered":"2024-09-10T15:52:30.159727+0000","last_clean":"2024-09-10T15:51:57.078241+0000","last_became_active":"2024-09-10T15:52:30.159635+0000","last_became_peered":"2024-09-10T15:52:30.159635+0000","last_unstale":"2024-09-10T15:52:30.159727+0000","last_undegraded":"2024-09-10T15:52:30.148722+0000","last_fullsized":"2024-09-10T15:52:30.159727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.157409+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:52:31.157409+0000","last_peered":"2024-09-10T15:52:31.157409+0000","last_clean":"2024-09-10T15:52:31.157409+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:52:31.157409+0000","last_undegraded":"2024-09-10T15:52:31.157409+0000","last_fullsized":"2024-09-10T15:52:31.157409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"24'2","reported_seq":57,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.159625+0000","last_change":"2024-09-10T15:52:30.159788+0000","last_active":"2024-09-10T15:52:30.159625+0000","last_peered":"2024-09-10T15:52:30.159625+0000","last_clean":"2024-09-10T15:51:45.893570+0000","last_became_active":"2024-09-10T15:52:30.159560+0000","last_became_peered":"2024-09-10T15:52:30.159560+0000","last_unstale":"2024-09-10T15:52:30.159625+0000","last_undegraded":"2024-09-10T15:52:30.148492+0000","last_fullsized":"2024-09-10T15:52:30.159625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-10T15:52:30.149362+0000","last_change":"2024-09-10T15:52:30.149270+0000","last_active":"2024-09-10T15:52:30.149362+0000","last_peered":"2024-09-10T15:52:30.149362+0000","last_clean":"2024-09-10T15:52:30.149362+0000","last_became_active":"2024-09-10T15:52:30.148972+0000","last_became_peered":"2024-09-10T15:52:30.148972+0000","last_unstale":"2024-09-10T15:52:30.149362+0000","last_undegraded":"2024-09-10T15:52:30.149362+0000","last_fullsized":"2024-09-10T15:52:30.149362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:21.166160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.158991+0000","last_change":"2024-09-10T15:52:31.155379+0000","last_active":"2024-09-10T15:52:31.158991+0000","last_peered":"2024-09-10T15:52:31.158991+0000","last_clean":"2024-09-10T15:52:31.158991+0000","last_became_active":"2024-09-10T15:52:31.155102+0000","last_became_peered":"2024-09-10T15:52:31.155102+0000","last_unstale":"2024-09-10T15:52:31.158991+0000","last_undegraded":"2024-09-10T15:52:31.158991+0000","last_fullsized":"2024-09-10T15:52:31.158991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.976123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"24'9","reported_seq":66,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:30.152168+0000","last_change":"2024-09-10T15:52:30.152339+0000","last_active":"2024-09-10T15:52:30.152168+0000","last_peered":"2024-09-10T15:52:30.152168+0000","last_clean":"2024-09-10T15:51:56.837465+0000","last_became_active":"2024-09-10T15:52:30.152069+0000","last_became_peered":"2024-09-10T15:52:30.152069+0000","last_unstale":"2024-09-10T15:52:30.152168+0000","last_undegraded":"2024-09-10T15:52:30.148182+0000","last_fullsized":"2024-09-10T15:52:30.152168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149704,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":340,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":348160,"data_stored":98872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28110509},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953476,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":304,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":311296,"data_stored":96732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:52:35.221 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:52:35.565 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:35.565 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 54/60 objects degraded (90.000%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":81,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":37,"num_osds":3,"num_up_osds":2,"osd_up_since":1725983545,"num_in_osds":2,"osd_in_since":1725983549,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56922112,"bytes_avail":193216606208,"bytes_total":193273528320,"degraded_objects":54,"degraded_total":60,"degraded_ratio":0.90000000000000002,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-10T15:52:32.427210+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:52:35.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '199e91a6-cca4-4c41-a5fa-011e971838b5', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 54/60 objects degraded (90.000%), 12 pgs degraded', 'count': 12}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 81, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 37, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1725983545, 'num_in_osds': 2, 'osd_in_since': 1725983549, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering+degraded', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56922112, 'bytes_avail': 193216606208, 'bytes_total': 193273528320, 'degraded_objects': 54, 'degraded_total': 60, 'degraded_ratio': 0.9, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-10T15:51:14:059711+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-10T15:52:32.427210+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-10T15:52:35.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-10T15:52:35.579 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:52:35.579 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T15:52:35.711 INFO:teuthology.orchestra.run.smithi121.stdout:111669149705 2024-09-10T15:52:35.712 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:52:35.712 INFO:teuthology.orchestra.run.smithi121.stdout:137438953477 2024-09-10T15:52:35.713 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:52:36.045 INFO:teuthology.orchestra.run.smithi121.stdout:137438953476 2024-09-10T15:52:36.049 INFO:teuthology.orchestra.run.smithi121.stdout:111669149704 2024-09-10T15:52:36.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 137438953477 got 137438953476 for osd.2 2024-09-10T15:52:36.061 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149705 got 111669149704 for osd.0 2024-09-10T15:52:37.059 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:52:37.062 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:52:37.371 INFO:teuthology.orchestra.run.smithi121.stdout:111669149706 2024-09-10T15:52:37.371 INFO:teuthology.orchestra.run.smithi121.stdout:137438953477 2024-09-10T15:52:37.385 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 137438953477 got 137438953477 for osd.2 2024-09-10T15:52:37.385 DEBUG:teuthology.parallel:result is None 2024-09-10T15:52:37.386 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149705 got 111669149706 for osd.0 2024-09-10T15:52:37.386 DEBUG:teuthology.parallel:result is None 2024-09-10T15:52:37.386 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:52:37.732 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:37.733 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":83,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":37,"num_osds":3,"num_up_osds":2,"osd_up_since":1725983545,"num_in_osds":2,"osd_in_since":1725983549,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56930304,"bytes_avail":193216598016,"bytes_total":193273528320,"degraded_objects":54,"degraded_total":60,"degraded_ratio":0.90000000000000002,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-10T15:52:32.427210+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:52:37.745 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '199e91a6-cca4-4c41-a5fa-011e971838b5', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 54/60 objects degraded (90.000%), 11 pgs degraded', 'count': 11}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 83, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 37, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1725983545, 'num_in_osds': 2, 'osd_in_since': 1725983549, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56930304, 'bytes_avail': 193216598016, 'bytes_total': 193273528320, 'degraded_objects': 54, 'degraded_total': 60, 'degraded_ratio': 0.9, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-10T15:51:14:059711+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-10T15:52:32.427210+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-10T15:52:37.745 INFO:tasks.rep_lost_unfound_delete:there are 27 unfound objects 2024-09-10T15:52:37.745 DEBUG:teuthology.orchestra.run.smithi121:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.admin -b 4096 -p unfounddel_pool -t 20 bench 240 write' 2024-09-10T15:52:37.796 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:hints = 1 2024-09-10T15:52:37.796 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Maintaining 20 concurrent writes of 4096 bytes to objects of size 4096 for up to 240 seconds or 0 objects 2024-09-10T15:52:37.796 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Object prefix: benchmark_data_smithi121_43865 2024-09-10T15:52:37.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:52:37.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 0 0 0 0 0 0 - 0 2024-09-10T15:52:38.796 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 1 20 2530 2510 9.80414 9.80469 0.00520811 0.00793997 2024-09-10T15:52:39.796 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 2 20 5654 5634 11.0031 12.2031 0.00306031 0.00708754 2024-09-10T15:52:40.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 3 20 8737 8717 11.3494 12.043 0.00600947 0.00687448 2024-09-10T15:52:41.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 4 20 11163 11143 10.881 9.47656 0.00565677 0.00717126 2024-09-10T15:52:42.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 5 20 12891 12871 10.0547 6.75 0.0064448 0.00776145 2024-09-10T15:52:43.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 6 20 15180 15160 9.86905 8.94141 0.00513568 0.00790809 2024-09-10T15:52:44.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 7 20 17306 17286 9.64548 8.30469 0.0122759 0.00809202 2024-09-10T15:52:45.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 8 20 19845 19825 9.67926 9.91797 0.00706215 0.00806515 2024-09-10T15:52:46.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 9 20 21874 21854 9.48437 7.92578 0.00625231 0.00823047 2024-09-10T15:52:47.749 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:52:47.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 10 20 23836 23816 9.30227 7.66406 0.0139919 0.00839028 2024-09-10T15:52:48.288 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:48.289 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:52:48.309 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-09-10T15:52:46.430205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":76821735,"num_objects":18782,"num_object_clones":0,"num_object_copies":37564,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":18782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18809,"num_write_kb":75056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18810,"ondisk_log_size":18810,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":251016,"kb_used_data":137760,"kb_used_omap":8,"kb_used_meta":113207,"kb_avail":188492664,"statfs":{"total":193273528320,"available":193016487936,"internally_reserved":0,"allocated":141066240,"data_stored":140596905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":115924311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,5,3,0,0,0,0,9],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":76808192,"num_objects":18752,"num_object_clones":0,"num_object_copies":37504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18752,"num_write_kb":75008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002659"},"pg_stats":[{"pgid":"2.f","version":"37'1264","reported_seq":1315,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:46.016810+0000","last_change":"2024-09-10T15:52:30.154389+0000","last_active":"2024-09-10T15:52:46.016810+0000","last_peered":"2024-09-10T15:52:46.016810+0000","last_clean":"2024-09-10T15:52:46.016810+0000","last_became_active":"2024-09-10T15:52:30.154140+0000","last_became_peered":"2024-09-10T15:52:30.154140+0000","last_unstale":"2024-09-10T15:52:46.016810+0000","last_undegraded":"2024-09-10T15:52:46.016810+0000","last_fullsized":"2024-09-10T15:52:46.016810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1264,"log_dups_size":0,"ondisk_log_size":1264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:30.595214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5177344,"num_objects":1264,"num_object_clones":0,"num_object_copies":2528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1264,"num_write_kb":5056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'1031","reported_seq":1083,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:52:44.086014+0000","last_change":"2024-09-10T15:52:37.808552+0000","last_active":"2024-09-10T15:52:44.086014+0000","last_peered":"2024-09-10T15:52:44.086014+0000","last_clean":"2024-09-10T15:51:56.324820+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:52:44.086014+0000","last_undegraded":"2024-09-10T15:52:31.158841+0000","last_fullsized":"2024-09-10T15:52:44.086014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1031,"log_dups_size":0,"ondisk_log_size":1031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4207558,"num_objects":1029,"num_object_clones":0,"num_object_copies":2058,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1031,"num_write_kb":4112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1029}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'980","reported_seq":1043,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:52:44.112384+0000","last_change":"2024-09-10T15:52:34.107356+0000","last_active":"2024-09-10T15:52:44.112384+0000","last_peered":"2024-09-10T15:52:44.112384+0000","last_clean":"2024-09-10T15:51:56.935537+0000","last_became_active":"2024-09-10T15:52:31.157749+0000","last_became_peered":"2024-09-10T15:52:31.157749+0000","last_unstale":"2024-09-10T15:52:44.112384+0000","last_undegraded":"2024-09-10T15:52:31.161688+0000","last_fullsized":"2024-09-10T15:52:44.112384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4002275,"num_objects":978,"num_object_clones":0,"num_object_copies":1956,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":980,"num_write_kb":3910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":978}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'1010","reported_seq":1070,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:44.088420+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:52:44.088420+0000","last_peered":"2024-09-10T15:52:44.088420+0000","last_clean":"2024-09-10T15:52:44.088420+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:52:44.088420+0000","last_undegraded":"2024-09-10T15:52:44.088420+0000","last_fullsized":"2024-09-10T15:52:44.088420+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1010,"log_dups_size":0,"ondisk_log_size":1010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4136960,"num_objects":1010,"num_object_clones":0,"num_object_copies":2020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1010,"num_write_kb":4040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'1248","reported_seq":1308,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:46.025103+0000","last_change":"2024-09-10T15:52:30.152160+0000","last_active":"2024-09-10T15:52:46.025103+0000","last_peered":"2024-09-10T15:52:46.025103+0000","last_clean":"2024-09-10T15:51:56.229374+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:52:46.025103+0000","last_undegraded":"2024-09-10T15:52:30.147386+0000","last_fullsized":"2024-09-10T15:52:46.025103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1248,"log_dups_size":0,"ondisk_log_size":1248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5092294,"num_objects":1245,"num_object_clones":0,"num_object_copies":2490,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1248,"num_write_kb":4976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1245}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'1239","reported_seq":1297,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:46.013134+0000","last_change":"2024-09-10T15:52:30.152575+0000","last_active":"2024-09-10T15:52:46.013134+0000","last_peered":"2024-09-10T15:52:46.013134+0000","last_clean":"2024-09-10T15:51:57.126582+0000","last_became_active":"2024-09-10T15:52:30.152332+0000","last_became_peered":"2024-09-10T15:52:30.152332+0000","last_unstale":"2024-09-10T15:52:46.013134+0000","last_undegraded":"2024-09-10T15:52:30.147849+0000","last_fullsized":"2024-09-10T15:52:46.013134+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1239,"log_dups_size":0,"ondisk_log_size":1239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5063622,"num_objects":1238,"num_object_clones":0,"num_object_copies":2476,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1239,"num_write_kb":4947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1238}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'1277","reported_seq":1335,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:46.012763+0000","last_change":"2024-09-10T15:52:30.157171+0000","last_active":"2024-09-10T15:52:46.012763+0000","last_peered":"2024-09-10T15:52:46.012763+0000","last_clean":"2024-09-10T15:51:56.469065+0000","last_became_active":"2024-09-10T15:52:30.156955+0000","last_became_peered":"2024-09-10T15:52:30.156955+0000","last_unstale":"2024-09-10T15:52:46.012763+0000","last_undegraded":"2024-09-10T15:52:30.148233+0000","last_fullsized":"2024-09-10T15:52:46.012763+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5215174,"num_objects":1275,"num_object_clones":0,"num_object_copies":2550,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1277,"num_write_kb":5096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1275}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'1277","reported_seq":1337,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:46.024952+0000","last_change":"2024-09-10T15:52:30.152559+0000","last_active":"2024-09-10T15:52:46.024952+0000","last_peered":"2024-09-10T15:52:46.024952+0000","last_clean":"2024-09-10T15:51:42.870214+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:52:46.024952+0000","last_undegraded":"2024-09-10T15:52:30.147342+0000","last_fullsized":"2024-09-10T15:52:46.024952+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5226979,"num_objects":1277,"num_object_clones":0,"num_object_copies":2554,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1277,"num_write_kb":5105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1277}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.151621+0000","last_change":"2024-09-10T15:52:30.152637+0000","last_active":"2024-09-10T15:52:31.151621+0000","last_peered":"2024-09-10T15:52:31.151621+0000","last_clean":"2024-09-10T15:52:31.151621+0000","last_became_active":"2024-09-10T15:52:30.151730+0000","last_became_peered":"2024-09-10T15:52:30.151730+0000","last_unstale":"2024-09-10T15:52:31.151621+0000","last_undegraded":"2024-09-10T15:52:31.151621+0000","last_fullsized":"2024-09-10T15:52:31.151621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:09.577308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'1045","reported_seq":1108,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:52:44.087135+0000","last_change":"2024-09-10T15:52:34.107284+0000","last_active":"2024-09-10T15:52:44.087135+0000","last_peered":"2024-09-10T15:52:44.087135+0000","last_clean":"2024-09-10T15:51:56.983672+0000","last_became_active":"2024-09-10T15:52:31.157754+0000","last_became_peered":"2024-09-10T15:52:31.157754+0000","last_unstale":"2024-09-10T15:52:44.087135+0000","last_undegraded":"2024-09-10T15:52:31.162204+0000","last_fullsized":"2024-09-10T15:52:44.087135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1045,"log_dups_size":0,"ondisk_log_size":1045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4260806,"num_objects":1042,"num_object_clones":0,"num_object_copies":2084,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1045,"num_write_kb":4164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1042}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.160328+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:52:31.160328+0000","last_peered":"2024-09-10T15:52:31.160328+0000","last_clean":"2024-09-10T15:52:31.160328+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:52:31.160328+0000","last_undegraded":"2024-09-10T15:52:31.160328+0000","last_fullsized":"2024-09-10T15:52:31.160328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'1288","reported_seq":1337,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:46.017276+0000","last_change":"2024-09-10T15:52:30.152404+0000","last_active":"2024-09-10T15:52:46.017276+0000","last_peered":"2024-09-10T15:52:46.017276+0000","last_clean":"2024-09-10T15:51:57.174499+0000","last_became_active":"2024-09-10T15:52:30.152112+0000","last_became_peered":"2024-09-10T15:52:30.152112+0000","last_unstale":"2024-09-10T15:52:46.017276+0000","last_undegraded":"2024-09-10T15:52:30.147626+0000","last_fullsized":"2024-09-10T15:52:46.017276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1288,"log_dups_size":0,"ondisk_log_size":1288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5252038,"num_objects":1284,"num_object_clones":0,"num_object_copies":2568,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1288,"num_write_kb":5132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1284}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:31.152685+0000","last_change":"2024-09-10T15:52:30.152323+0000","last_active":"2024-09-10T15:52:31.152685+0000","last_peered":"2024-09-10T15:52:31.152685+0000","last_clean":"2024-09-10T15:52:08.013728+0000","last_became_active":"2024-09-10T15:52:30.151717+0000","last_became_peered":"2024-09-10T15:52:30.151717+0000","last_unstale":"2024-09-10T15:52:31.152685+0000","last_undegraded":"2024-09-10T15:52:30.144026+0000","last_fullsized":"2024-09-10T15:52:31.152685+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'990","reported_seq":1049,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:52:44.085851+0000","last_change":"2024-09-10T15:52:37.803135+0000","last_active":"2024-09-10T15:52:44.085851+0000","last_peered":"2024-09-10T15:52:44.085851+0000","last_clean":"2024-09-10T15:51:56.181288+0000","last_became_active":"2024-09-10T15:52:31.157172+0000","last_became_peered":"2024-09-10T15:52:31.157172+0000","last_unstale":"2024-09-10T15:52:44.085851+0000","last_undegraded":"2024-09-10T15:52:31.161142+0000","last_fullsized":"2024-09-10T15:52:44.085851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4043718,"num_objects":989,"num_object_clones":0,"num_object_copies":1978,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":990,"num_write_kb":3951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":989}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":91,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.161559+0000","last_change":"2024-09-10T15:52:31.159524+0000","last_active":"2024-09-10T15:52:31.161559+0000","last_peered":"2024-09-10T15:52:31.161559+0000","last_clean":"2024-09-10T15:52:31.161559+0000","last_became_active":"2024-09-10T15:52:31.159205+0000","last_became_peered":"2024-09-10T15:52:31.159205+0000","last_unstale":"2024-09-10T15:52:31.161559+0000","last_undegraded":"2024-09-10T15:52:31.161559+0000","last_fullsized":"2024-09-10T15:52:31.161559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:04.415297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'1009","reported_seq":1065,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-10T15:52:44.107931+0000","last_change":"2024-09-10T15:52:37.800480+0000","last_active":"2024-09-10T15:52:44.107931+0000","last_peered":"2024-09-10T15:52:44.107931+0000","last_clean":"2024-09-10T15:51:56.644471+0000","last_became_active":"2024-09-10T15:52:31.157034+0000","last_became_peered":"2024-09-10T15:52:31.157034+0000","last_unstale":"2024-09-10T15:52:44.107931+0000","last_undegraded":"2024-09-10T15:52:31.160690+0000","last_fullsized":"2024-09-10T15:52:44.107931+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4117446,"num_objects":1007,"num_object_clones":0,"num_object_copies":2014,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1009,"num_write_kb":4023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1007}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'1278","reported_seq":1336,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-10T15:52:46.025045+0000","last_change":"2024-09-10T15:52:31.150765+0000","last_active":"2024-09-10T15:52:46.025045+0000","last_peered":"2024-09-10T15:52:46.025045+0000","last_clean":"2024-09-10T15:51:45.893657+0000","last_became_active":"2024-09-10T15:52:30.151806+0000","last_became_peered":"2024-09-10T15:52:30.151806+0000","last_unstale":"2024-09-10T15:52:46.025045+0000","last_undegraded":"2024-09-10T15:52:46.025045+0000","last_fullsized":"2024-09-10T15:52:46.025045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1278,"log_dups_size":0,"ondisk_log_size":1278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5231075,"num_objects":1278,"num_object_clones":0,"num_object_copies":2556,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1278,"num_write_kb":5109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1278}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.149171+0000","last_change":"2024-09-10T15:52:30.154393+0000","last_active":"2024-09-10T15:52:31.149171+0000","last_peered":"2024-09-10T15:52:31.149171+0000","last_clean":"2024-09-10T15:52:31.149171+0000","last_became_active":"2024-09-10T15:52:30.154111+0000","last_became_peered":"2024-09-10T15:52:30.154111+0000","last_unstale":"2024-09-10T15:52:31.149171+0000","last_undegraded":"2024-09-10T15:52:31.149171+0000","last_fullsized":"2024-09-10T15:52:31.149171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:53.944111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'1327","reported_seq":1385,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:46.009112+0000","last_change":"2024-09-10T15:52:30.159885+0000","last_active":"2024-09-10T15:52:46.009112+0000","last_peered":"2024-09-10T15:52:46.009112+0000","last_clean":"2024-09-10T15:51:57.078241+0000","last_became_active":"2024-09-10T15:52:30.159635+0000","last_became_peered":"2024-09-10T15:52:30.159635+0000","last_unstale":"2024-09-10T15:52:46.009112+0000","last_undegraded":"2024-09-10T15:52:30.148722+0000","last_fullsized":"2024-09-10T15:52:46.009112+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1327,"log_dups_size":0,"ondisk_log_size":1327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5416361,"num_objects":1325,"num_object_clones":0,"num_object_copies":2650,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1327,"num_write_kb":5292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1325}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.157409+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:52:31.157409+0000","last_peered":"2024-09-10T15:52:31.157409+0000","last_clean":"2024-09-10T15:52:31.157409+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:52:31.157409+0000","last_undegraded":"2024-09-10T15:52:31.157409+0000","last_fullsized":"2024-09-10T15:52:31.157409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'1249","reported_seq":1305,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:46.024961+0000","last_change":"2024-09-10T15:52:30.159788+0000","last_active":"2024-09-10T15:52:46.024961+0000","last_peered":"2024-09-10T15:52:46.024961+0000","last_clean":"2024-09-10T15:51:45.893570+0000","last_became_active":"2024-09-10T15:52:30.159560+0000","last_became_peered":"2024-09-10T15:52:30.159560+0000","last_unstale":"2024-09-10T15:52:46.024961+0000","last_undegraded":"2024-09-10T15:52:30.148492+0000","last_fullsized":"2024-09-10T15:52:46.024961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1249,"log_dups_size":0,"ondisk_log_size":1249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5108678,"num_objects":1249,"num_object_clones":0,"num_object_copies":2498,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1249,"num_write_kb":4990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1249}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.152595+0000","last_change":"2024-09-10T15:52:30.149270+0000","last_active":"2024-09-10T15:52:31.152595+0000","last_peered":"2024-09-10T15:52:31.152595+0000","last_clean":"2024-09-10T15:52:31.152595+0000","last_became_active":"2024-09-10T15:52:30.148972+0000","last_became_peered":"2024-09-10T15:52:30.148972+0000","last_unstale":"2024-09-10T15:52:31.152595+0000","last_undegraded":"2024-09-10T15:52:31.152595+0000","last_fullsized":"2024-09-10T15:52:31.152595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:21.166160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.158991+0000","last_change":"2024-09-10T15:52:31.155379+0000","last_active":"2024-09-10T15:52:31.158991+0000","last_peered":"2024-09-10T15:52:31.158991+0000","last_clean":"2024-09-10T15:52:31.158991+0000","last_became_active":"2024-09-10T15:52:31.155102+0000","last_became_peered":"2024-09-10T15:52:31.155102+0000","last_unstale":"2024-09-10T15:52:31.158991+0000","last_undegraded":"2024-09-10T15:52:31.158991+0000","last_fullsized":"2024-09-10T15:52:31.158991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.976123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'1295","reported_seq":1354,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:46.025182+0000","last_change":"2024-09-10T15:52:30.152339+0000","last_active":"2024-09-10T15:52:46.025182+0000","last_peered":"2024-09-10T15:52:46.025182+0000","last_clean":"2024-09-10T15:51:56.837465+0000","last_became_active":"2024-09-10T15:52:30.152069+0000","last_became_peered":"2024-09-10T15:52:30.152069+0000","last_unstale":"2024-09-10T15:52:46.025182+0000","last_undegraded":"2024-09-10T15:52:30.148182+0000","last_fullsized":"2024-09-10T15:52:46.025182+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5269388,"num_objects":1290,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1295,"num_write_kb":5151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1289}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":76821716,"num_objects":18780,"num_object_clones":0,"num_object_copies":37560,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":18780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18807,"num_write_kb":75054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":149532672,"data_stored":149395378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18807,"ondisk_log_size":18807,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149709,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":143024,"kb_used_data":75292,"kb_used_omap":4,"kb_used_meta":67707,"kb_avail":94228816,"statfs":{"total":96636764160,"available":96490307584,"internally_reserved":0,"allocated":77099008,"data_stored":76844237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":69332653},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,9],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953479,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107992,"kb_used_data":62468,"kb_used_omap":4,"kb_used_meta":45499,"kb_avail":94263848,"statfs":{"total":96636764160,"available":96526180352,"internally_reserved":0,"allocated":63967232,"data_stored":63752668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":46591658},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,5,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":83771392,"data_stored":83735262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65646592,"data_stored":65646592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:52:48.310 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.f 2024-09-10T15:52:48.310 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.e state active+recovery_unfound+degraded 2024-09-10T15:52:48.310 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.e list_unfound '{}' 2024-09-10T15:52:48.520 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:48.520 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:48.520 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:48.520 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:48.520 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:48.520 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:48.520 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:48.520 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 2, 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 2, 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existing_4", 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 1224592654, 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:48.521 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'3", 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "22'1", 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existing_6", 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:48.522 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 3101443614, 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'4", 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "22'2", 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "NotRecovering", 2024-09-10T15:52:48.523 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:48.524 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:48.524 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:48.524 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:48.524 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:48.524 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:48.524 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:48.524 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:48.524 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:48.541 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.e on osd.2 2024-09-10T15:52:48.541 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.e mark_unfound_lost delete 2024-09-10T15:52:48.771 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:52:48.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 11 20 25866 25846 9.17743 7.92969 0.00837051 0.00850534 2024-09-10T15:52:48.800 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.d state active+recovery_unfound+degraded 2024-09-10T15:52:48.800 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.d list_unfound '{}' 2024-09-10T15:52:48.957 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:48.958 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:48.958 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:48.958 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:48.958 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:48.958 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:48.958 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:48.958 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:48.958 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:48.958 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:48.958 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 1, 2024-09-10T15:52:48.958 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 1, 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existed_7", 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 2358579165, 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'3", 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:48.959 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "NotRecovering", 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:48.960 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:48.961 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:48.968 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.d on osd.2 2024-09-10T15:52:48.968 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.d mark_unfound_lost delete 2024-09-10T15:52:49.112 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-10T15:52:49.134 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.c 2024-09-10T15:52:49.134 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.b state active+recovery_wait+degraded 2024-09-10T15:52:49.135 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.b list_unfound '{}' 2024-09-10T15:52:49.337 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:49.337 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:49.337 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:49.337 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:49.337 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 2, 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 2, 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existed_3", 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:49.338 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:49.339 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 943960171, 2024-09-10T15:52:49.339 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:49.339 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:49.339 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:49.339 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:49.339 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'5", 2024-09-10T15:52:49.339 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:49.339 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:49.339 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:49.339 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:49.339 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:49.339 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existing_1", 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 2885884443, 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'4", 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "22'1", 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:52:49.340 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:49.341 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:49.341 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:49.341 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "Active", 2024-09-10T15:52:49.341 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:49.341 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:49.341 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:49.341 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:49.341 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:49.341 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:49.341 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:49.341 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:49.342 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:49.347 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.b on osd.0 2024-09-10T15:52:49.347 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.b mark_unfound_lost delete 2024-09-10T15:52:49.535 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:52:49.562 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.a state active+recovery_wait+degraded 2024-09-10T15:52:49.562 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.a list_unfound '{}' 2024-09-10T15:52:49.775 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:49.776 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:49.776 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:49.776 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:49.776 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:49.776 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:49.776 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:49.776 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:49.776 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:49.776 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:49.776 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 2, 2024-09-10T15:52:49.776 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 2, 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existing_9", 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 1217469130, 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'3", 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "22'1", 2024-09-10T15:52:49.777 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "new_9", 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 170713514, 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:49.778 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'2", 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "Active", 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:49.780 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:49.780 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:49.780 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:49.793 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.a on osd.0 2024-09-10T15:52:49.793 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.a mark_unfound_lost delete 2024-09-10T15:52:49.798 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 12 20 26952 26932 8.76614 4.24219 0.0276725 0.00890386 2024-09-10T15:52:50.028 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:52:50.052 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.9 state active+recovery_wait+degraded 2024-09-10T15:52:50.053 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.9 list_unfound '{}' 2024-09-10T15:52:50.272 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:50.272 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:50.272 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:50.272 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:50.272 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 2, 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 2, 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existing_5", 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 2998549577, 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'4", 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "22'2", 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:50.274 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existing_2", 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 4032093161, 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'3", 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "22'1", 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:52:50.275 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:50.276 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:50.276 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:50.276 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "Active", 2024-09-10T15:52:50.276 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:50.276 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:50.276 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:50.276 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:50.276 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:50.276 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:50.276 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:50.276 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:50.276 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:50.293 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.9 on osd.0 2024-09-10T15:52:50.293 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.9 mark_unfound_lost delete 2024-09-10T15:52:50.534 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:52:50.554 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.8 state active+recovery_wait+degraded 2024-09-10T15:52:50.554 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.8 list_unfound '{}' 2024-09-10T15:52:50.768 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:50.768 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:50.768 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:50.768 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:50.768 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:50.768 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 1, 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 1, 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "new_8", 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:50.769 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 2730327960, 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'1", 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "Active", 2024-09-10T15:52:50.770 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:50.771 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:50.771 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:50.771 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:50.771 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:50.771 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:50.771 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:50.771 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:50.771 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:50.785 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.8 on osd.0 2024-09-10T15:52:50.786 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.8 mark_unfound_lost delete 2024-09-10T15:52:50.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 13 20 28776 28756 8.63985 7.125 0.00843775 0.00903487 2024-09-10T15:52:51.013 INFO:teuthology.orchestra.run.smithi121.stderr:pg has no unfound objects 2024-09-10T15:52:51.034 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.0 2024-09-10T15:52:51.034 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.3 state active+recovery_unfound+degraded 2024-09-10T15:52:51.034 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.3 list_unfound '{}' 2024-09-10T15:52:51.251 INFO:tasks.ceph.mon.c.smithi121.stderr:2024-09-10T15:52:51.249+0000 7fb18b8c1640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 3 pgs recovery_unfound (PG_DAMAGED) 2024-09-10T15:52:51.306 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:51.306 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:51.306 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:51.306 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:51.306 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:51.306 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:51.306 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:51.306 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:51.306 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 2, 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 2, 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existing_8", 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 1901787523, 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:51.307 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'5", 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "22'3", 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existed_1", 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:51.308 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 3122379363, 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'4", 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "NotRecovering", 2024-09-10T15:52:51.309 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:51.310 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:51.310 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:51.310 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:51.310 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:51.310 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:51.310 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:51.310 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:51.310 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:51.328 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.3 on osd.2 2024-09-10T15:52:51.329 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.3 mark_unfound_lost delete 2024-09-10T15:52:51.561 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:52:51.585 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.1 2024-09-10T15:52:51.585 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.2 state active+recovery_wait+degraded 2024-09-10T15:52:51.585 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.2 list_unfound '{}' 2024-09-10T15:52:51.747 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:51.747 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:51.747 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:51.747 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:51.747 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 2, 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 2, 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existed_6", 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:51.748 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 1708673474, 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'5", 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:51.749 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existed_9", 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 831402034, 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'6", 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:51.750 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:51.751 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:51.751 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "Active", 2024-09-10T15:52:51.751 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:51.751 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:51.751 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:51.751 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:51.751 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:51.751 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:51.751 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:51.751 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:51.751 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:51.757 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.2 on osd.0 2024-09-10T15:52:51.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.2 mark_unfound_lost delete 2024-09-10T15:52:51.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 14 20 30453 30433 8.49059 6.55078 0.00570618 0.00912448 2024-09-10T15:52:51.908 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:52:51.927 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.2 2024-09-10T15:52:51.927 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.1 state active+recovery_unfound+degraded 2024-09-10T15:52:51.928 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.1 list_unfound '{}' 2024-09-10T15:52:52.190 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:52.191 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:52.191 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:52.191 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:52.191 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:52.191 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:52.191 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:52.191 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:52.191 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:52.191 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:52.191 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 2, 2024-09-10T15:52:52.191 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 2, 2024-09-10T15:52:52.191 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existing_3", 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 459768081, 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'3", 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "22'1", 2024-09-10T15:52:52.192 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "new_3", 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 480221105, 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:52.193 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'2", 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "NotRecovering", 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:52.194 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:52.195 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:52.195 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:52.212 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.1 on osd.2 2024-09-10T15:52:52.212 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.1 mark_unfound_lost delete 2024-09-10T15:52:52.449 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:52:52.468 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.3 2024-09-10T15:52:52.468 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.0 state active+recovery_unfound+degraded 2024-09-10T15:52:52.468 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.0 list_unfound '{}' 2024-09-10T15:52:52.686 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:52.686 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:52.686 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:52.686 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:52.686 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:52.687 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:52.687 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:52.687 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:52.687 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:52.687 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:52.687 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 2, 2024-09-10T15:52:52.688 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 2, 2024-09-10T15:52:52.688 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:52.688 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:52.688 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:52.688 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "new_5", 2024-09-10T15:52:52.688 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:52.688 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:52.688 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 4248945696, 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'3", 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:52.689 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existed_5", 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 4212789968, 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'4", 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:52.690 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:52.691 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:52.691 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "NotRecovering", 2024-09-10T15:52:52.691 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:52.691 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:52.691 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:52.691 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:52.691 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:52.692 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:52.692 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:52.692 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:52.692 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:52.708 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.0 on osd.2 2024-09-10T15:52:52.708 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.0 mark_unfound_lost delete 2024-09-10T15:52:52.797 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 15 20 32013 31993 8.33077 6.09375 0.011341 0.00937044 2024-09-10T15:52:52.939 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:52:52.958 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.7 state active+recovering 2024-09-10T15:52:52.958 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.7 list_unfound '{}' 2024-09-10T15:52:53.183 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 1, 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 1, 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:53.184 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "new_7", 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 3761017527, 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'1", 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:53.185 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:53.186 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:53.186 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "Active", 2024-09-10T15:52:53.186 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:53.186 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:53.186 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:53.186 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:53.186 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:53.186 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:53.186 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:53.186 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:53.186 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:53.205 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.7 on osd.0 2024-09-10T15:52:53.205 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.7 mark_unfound_lost delete 2024-09-10T15:52:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-10T15:52:53.470 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.4 2024-09-10T15:52:53.470 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.6 state active+recovery_wait+degraded 2024-09-10T15:52:53.470 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.6 list_unfound '{}' 2024-09-10T15:52:53.695 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:53.695 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:53.695 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:53.695 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:53.695 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:53.695 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:53.695 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:53.695 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:53.695 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:53.695 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:53.695 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 3, 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 3, 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existed_8", 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 1861424966, 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'5", 2024-09-10T15:52:53.696 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "new_6", 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 3910131350, 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:53.697 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'4", 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "new_4", 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 2787839990, 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:53.698 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'3", 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "Active", 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:53.699 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:53.700 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:53.700 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:53.700 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:53.700 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:53.718 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.6 on osd.0 2024-09-10T15:52:53.719 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.6 mark_unfound_lost delete 2024-09-10T15:52:53.798 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 16 20 33674 33654 8.21558 6.48828 0.0121377 0.0095026 2024-09-10T15:52:53.955 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-10T15:52:53.976 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.5 2024-09-10T15:52:53.977 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.5 state active+recovery_wait+degraded 2024-09-10T15:52:53.977 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.5 list_unfound '{}' 2024-09-10T15:52:54.190 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:54.190 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:54.190 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 2, 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 2, 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "new_1", 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:54.191 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 26281365, 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'1", 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:54.192 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "new_2", 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 3196340981, 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'2", 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:54.193 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:54.194 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:54.194 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "Active", 2024-09-10T15:52:54.194 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:54.194 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:54.194 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:54.194 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:54.194 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:54.194 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:54.194 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:54.194 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:54.194 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:54.217 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.5 on osd.0 2024-09-10T15:52:54.217 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.5 mark_unfound_lost delete 2024-09-10T15:52:54.463 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-10T15:52:54.487 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.6 2024-09-10T15:52:54.487 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.7 2024-09-10T15:52:54.487 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.4 state active+recovery_wait+degraded 2024-09-10T15:52:54.487 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.4 list_unfound '{}' 2024-09-10T15:52:54.645 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:52:54.645 INFO:teuthology.orchestra.run.smithi121.stdout: "offset": { 2024-09-10T15:52:54.645 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "", 2024-09-10T15:52:54.645 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:54.645 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": 0, 2024-09-10T15:52:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 0, 2024-09-10T15:52:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": -9223372036854775808, 2024-09-10T15:52:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: "num_missing": 4, 2024-09-10T15:52:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: "num_unfound": 3, 2024-09-10T15:52:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: "objects": [ 2024-09-10T15:52:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existed_4", 2024-09-10T15:52:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 873891716, 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'8", 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:54.647 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:54.648 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existing_7", 2024-09-10T15:52:54.648 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:54.648 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:54.648 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 3996155460, 2024-09-10T15:52:54.648 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:54.648 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:54.648 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:54.648 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:54.648 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'9", 2024-09-10T15:52:54.648 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "22'6", 2024-09-10T15:52:54.648 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:54.648 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-10T15:52:54.649 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:54.649 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:54.649 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:54.649 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": { 2024-09-10T15:52:54.649 INFO:teuthology.orchestra.run.smithi121.stdout: "oid": "existed_2", 2024-09-10T15:52:54.649 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "", 2024-09-10T15:52:54.649 INFO:teuthology.orchestra.run.smithi121.stdout: "snapid": -2, 2024-09-10T15:52:54.649 INFO:teuthology.orchestra.run.smithi121.stdout: "hash": 1767667508, 2024-09-10T15:52:54.649 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 0, 2024-09-10T15:52:54.649 INFO:teuthology.orchestra.run.smithi121.stdout: "pool": 2, 2024-09-10T15:52:54.650 INFO:teuthology.orchestra.run.smithi121.stdout: "namespace": "" 2024-09-10T15:52:54.650 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:52:54.650 INFO:teuthology.orchestra.run.smithi121.stdout: "need": "24'7", 2024-09-10T15:52:54.650 INFO:teuthology.orchestra.run.smithi121.stdout: "have": "0'0", 2024-09-10T15:52:54.650 INFO:teuthology.orchestra.run.smithi121.stdout: "flags": "none", 2024-09-10T15:52:54.650 INFO:teuthology.orchestra.run.smithi121.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-10T15:52:54.650 INFO:teuthology.orchestra.run.smithi121.stdout: "locations": [] 2024-09-10T15:52:54.650 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:54.650 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:54.651 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "Active", 2024-09-10T15:52:54.651 INFO:teuthology.orchestra.run.smithi121.stdout: "available_might_have_unfound": true, 2024-09-10T15:52:54.651 INFO:teuthology.orchestra.run.smithi121.stdout: "might_have_unfound": [ 2024-09-10T15:52:54.651 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:52:54.651 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "1", 2024-09-10T15:52:54.651 INFO:teuthology.orchestra.run.smithi121.stdout: "status": "osd is down" 2024-09-10T15:52:54.651 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:52:54.651 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:52:54.651 INFO:teuthology.orchestra.run.smithi121.stdout: "more": false 2024-09-10T15:52:54.651 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:52:54.655 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.4 on osd.0 2024-09-10T15:52:54.655 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.4 mark_unfound_lost delete 2024-09-10T15:52:54.785 INFO:teuthology.orchestra.run.smithi121.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-10T15:52:54.798 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 17 20 34963 34943 8.02848 5.03516 0.019215 0.00972406 2024-09-10T15:52:54.813 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 5 2024-09-10T15:52:55.061 INFO:teuthology.orchestra.run.smithi121.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-10T15:52:55.087 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 5 2024-09-10T15:52:55.314 INFO:teuthology.orchestra.run.smithi121.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-10T15:52:55.334 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T15:52:55.334 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T15:52:55.584 INFO:teuthology.orchestra.run.smithi121.stdout:111669149712 2024-09-10T15:52:55.584 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:52:55.593 INFO:teuthology.orchestra.run.smithi121.stdout:137438953482 2024-09-10T15:52:55.593 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:52:55.798 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 18 20 36519 36499 7.91986 6.07812 0.0235172 0.00985927 2024-09-10T15:52:56.131 INFO:teuthology.orchestra.run.smithi121.stdout:111669149710 2024-09-10T15:52:56.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149712 got 111669149710 for osd.0 2024-09-10T15:52:56.182 INFO:teuthology.orchestra.run.smithi121.stdout:137438953481 2024-09-10T15:52:56.202 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 137438953482 got 137438953481 for osd.2 2024-09-10T15:52:56.798 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 19 20 38215 38195 7.85168 6.625 0.00902332 0.00994391 2024-09-10T15:52:57.151 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T15:52:57.204 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T15:52:57.701 INFO:teuthology.orchestra.run.smithi121.stdout:111669149713 2024-09-10T15:52:57.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149712 got 111669149713 for osd.0 2024-09-10T15:52:57.719 DEBUG:teuthology.parallel:result is None 2024-09-10T15:52:57.737 INFO:teuthology.orchestra.run.smithi121.stdout:137438953482 2024-09-10T15:52:57.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 137438953482 got 137438953482 for osd.2 2024-09-10T15:52:57.750 DEBUG:teuthology.parallel:result is None 2024-09-10T15:52:57.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-10T15:52:57.750 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:52:57.798 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:2024-09-10T15:52:57.798373+0000 min lat: 0.00198913 max lat: 0.223652 avg lat: 0.0100368 2024-09-10T15:52:57.798 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:52:57.799 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 20 20 39650 39630 7.73935 5.60547 0.00198913 0.0100368 2024-09-10T15:52:58.275 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:58.276 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:52:58.294 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-10T15:52:56.432769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":150008812,"num_objects":36642,"num_object_clones":0,"num_object_copies":73284,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27111,"num_write_kb":108324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36706,"ondisk_log_size":36706,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":466864,"kb_used_data":285580,"kb_used_omap":8,"kb_used_meta":181239,"kb_avail":188276816,"statfs":{"total":193273528320,"available":192795459584,"internally_reserved":0,"allocated":292433920,"data_stored":291979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":185589079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,7,1,6,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":101035781,"num_objects":24659,"num_object_clones":0,"num_object_copies":49318,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":24659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15101,"num_write_kb":60464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002945"},"pg_stats":[{"pgid":"2.f","version":"37'2383","reported_seq":2433,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:56.055712+0000","last_change":"2024-09-10T15:52:30.154389+0000","last_active":"2024-09-10T15:52:56.055712+0000","last_peered":"2024-09-10T15:52:56.055712+0000","last_clean":"2024-09-10T15:52:56.055712+0000","last_became_active":"2024-09-10T15:52:30.154140+0000","last_became_peered":"2024-09-10T15:52:30.154140+0000","last_unstale":"2024-09-10T15:52:56.055712+0000","last_undegraded":"2024-09-10T15:52:56.055712+0000","last_fullsized":"2024-09-10T15:52:56.055712+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2383,"log_dups_size":0,"ondisk_log_size":2383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:30.595214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9760768,"num_objects":2383,"num_object_clones":0,"num_object_copies":4766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2383,"num_write_kb":9532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'2305","reported_seq":2392,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.553675+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:52:55.553675+0000","last_peered":"2024-09-10T15:52:55.553675+0000","last_clean":"2024-09-10T15:52:55.553675+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:52:55.553675+0000","last_undegraded":"2024-09-10T15:52:55.553675+0000","last_fullsized":"2024-09-10T15:52:55.553675+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":2305,"log_dups_size":0,"ondisk_log_size":2305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9416704,"num_objects":2299,"num_object_clones":0,"num_object_copies":4598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":641,"num_write_kb":2564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'2232","reported_seq":2330,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.522714+0000","last_change":"2024-09-10T15:52:50.491262+0000","last_active":"2024-09-10T15:52:55.522714+0000","last_peered":"2024-09-10T15:52:55.522714+0000","last_clean":"2024-09-10T15:52:55.522714+0000","last_became_active":"2024-09-10T15:52:31.157749+0000","last_became_peered":"2024-09-10T15:52:31.157749+0000","last_unstale":"2024-09-10T15:52:55.522714+0000","last_undegraded":"2024-09-10T15:52:55.522714+0000","last_fullsized":"2024-09-10T15:52:55.522714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":2232,"log_dups_size":0,"ondisk_log_size":2232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:11.778645+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9125888,"num_objects":2228,"num_object_clones":0,"num_object_copies":4456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":500,"num_write_kb":2000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'2274","reported_seq":2333,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.553657+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:52:55.553657+0000","last_peered":"2024-09-10T15:52:55.553657+0000","last_clean":"2024-09-10T15:52:55.553657+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:52:55.553657+0000","last_undegraded":"2024-09-10T15:52:55.553657+0000","last_fullsized":"2024-09-10T15:52:55.553657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2274,"log_dups_size":0,"ondisk_log_size":2274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9314304,"num_objects":2274,"num_object_clones":0,"num_object_copies":4548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2274,"num_write_kb":9096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'2268","reported_seq":2327,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.020037+0000","last_change":"2024-09-10T15:52:30.152160+0000","last_active":"2024-09-10T15:52:56.020037+0000","last_peered":"2024-09-10T15:52:56.020037+0000","last_clean":"2024-09-10T15:51:56.229374+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:52:56.020037+0000","last_undegraded":"2024-09-10T15:52:30.147386+0000","last_fullsized":"2024-09-10T15:52:56.020037+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9262022,"num_objects":2263,"num_object_clones":0,"num_object_copies":4526,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2266,"num_write_kb":9048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2263}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'2272","reported_seq":2329,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.032243+0000","last_change":"2024-09-10T15:52:30.152575+0000","last_active":"2024-09-10T15:52:56.032243+0000","last_peered":"2024-09-10T15:52:56.032243+0000","last_clean":"2024-09-10T15:51:57.126582+0000","last_became_active":"2024-09-10T15:52:30.152332+0000","last_became_peered":"2024-09-10T15:52:30.152332+0000","last_unstale":"2024-09-10T15:52:56.032243+0000","last_undegraded":"2024-09-10T15:52:30.147849+0000","last_fullsized":"2024-09-10T15:52:56.032243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2272,"log_dups_size":0,"ondisk_log_size":2272,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9286598,"num_objects":2269,"num_object_clones":0,"num_object_copies":4538,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2270,"num_write_kb":9071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2269}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'2311","reported_seq":2368,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.020044+0000","last_change":"2024-09-10T15:52:30.157171+0000","last_active":"2024-09-10T15:52:56.020044+0000","last_peered":"2024-09-10T15:52:56.020044+0000","last_clean":"2024-09-10T15:51:56.469065+0000","last_became_active":"2024-09-10T15:52:30.156955+0000","last_became_peered":"2024-09-10T15:52:30.156955+0000","last_unstale":"2024-09-10T15:52:56.020044+0000","last_undegraded":"2024-09-10T15:52:30.148233+0000","last_fullsized":"2024-09-10T15:52:56.020044+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2311,"log_dups_size":0,"ondisk_log_size":2311,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9442246,"num_objects":2307,"num_object_clones":0,"num_object_copies":4614,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2309,"num_write_kb":9224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2307}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'2342","reported_seq":2403,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.043371+0000","last_change":"2024-09-10T15:52:30.152559+0000","last_active":"2024-09-10T15:52:56.043371+0000","last_peered":"2024-09-10T15:52:56.043371+0000","last_clean":"2024-09-10T15:51:42.870214+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:52:56.043371+0000","last_undegraded":"2024-09-10T15:52:30.147342+0000","last_fullsized":"2024-09-10T15:52:56.043371+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2342,"log_dups_size":0,"ondisk_log_size":2342,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9585123,"num_objects":2341,"num_object_clones":0,"num_object_copies":4682,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2341,"num_write_kb":9361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.151621+0000","last_change":"2024-09-10T15:52:30.152637+0000","last_active":"2024-09-10T15:52:31.151621+0000","last_peered":"2024-09-10T15:52:31.151621+0000","last_clean":"2024-09-10T15:52:31.151621+0000","last_became_active":"2024-09-10T15:52:30.151730+0000","last_became_peered":"2024-09-10T15:52:30.151730+0000","last_unstale":"2024-09-10T15:52:31.151621+0000","last_undegraded":"2024-09-10T15:52:31.151621+0000","last_fullsized":"2024-09-10T15:52:31.151621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:09.577308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'2358","reported_seq":2462,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.538822+0000","last_change":"2024-09-10T15:52:52.645800+0000","last_active":"2024-09-10T15:52:55.538822+0000","last_peered":"2024-09-10T15:52:55.538822+0000","last_clean":"2024-09-10T15:52:55.538822+0000","last_became_active":"2024-09-10T15:52:31.157754+0000","last_became_peered":"2024-09-10T15:52:31.157754+0000","last_unstale":"2024-09-10T15:52:55.538822+0000","last_undegraded":"2024-09-10T15:52:55.538822+0000","last_fullsized":"2024-09-10T15:52:55.538822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":2358,"log_dups_size":0,"ondisk_log_size":2358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:37.611240+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9629696,"num_objects":2351,"num_object_clones":0,"num_object_copies":4702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":335,"num_write_kb":1340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.160328+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:52:31.160328+0000","last_peered":"2024-09-10T15:52:31.160328+0000","last_clean":"2024-09-10T15:52:31.160328+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:52:31.160328+0000","last_undegraded":"2024-09-10T15:52:31.160328+0000","last_fullsized":"2024-09-10T15:52:31.160328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'2273","reported_seq":2323,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.005533+0000","last_change":"2024-09-10T15:52:30.152404+0000","last_active":"2024-09-10T15:52:56.005533+0000","last_peered":"2024-09-10T15:52:56.005533+0000","last_clean":"2024-09-10T15:51:57.174499+0000","last_became_active":"2024-09-10T15:52:30.152112+0000","last_became_peered":"2024-09-10T15:52:30.152112+0000","last_unstale":"2024-09-10T15:52:56.005533+0000","last_undegraded":"2024-09-10T15:52:30.147626+0000","last_fullsized":"2024-09-10T15:52:56.005533+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2273,"log_dups_size":0,"ondisk_log_size":2273,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9278406,"num_objects":2267,"num_object_clones":0,"num_object_copies":4534,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2271,"num_write_kb":9064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2267}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:31.152685+0000","last_change":"2024-09-10T15:52:30.152323+0000","last_active":"2024-09-10T15:52:31.152685+0000","last_peered":"2024-09-10T15:52:31.152685+0000","last_clean":"2024-09-10T15:52:08.013728+0000","last_became_active":"2024-09-10T15:52:30.151717+0000","last_became_peered":"2024-09-10T15:52:30.151717+0000","last_unstale":"2024-09-10T15:52:31.152685+0000","last_undegraded":"2024-09-10T15:52:30.144026+0000","last_fullsized":"2024-09-10T15:52:31.152685+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'2209","reported_seq":2309,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.461790+0000","last_change":"2024-09-10T15:52:53.666934+0000","last_active":"2024-09-10T15:52:55.461790+0000","last_peered":"2024-09-10T15:52:55.461790+0000","last_clean":"2024-09-10T15:52:55.461790+0000","last_became_active":"2024-09-10T15:52:31.157172+0000","last_became_peered":"2024-09-10T15:52:31.157172+0000","last_unstale":"2024-09-10T15:52:55.461790+0000","last_undegraded":"2024-09-10T15:52:55.461790+0000","last_fullsized":"2024-09-10T15:52:55.461790+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":2209,"log_dups_size":0,"ondisk_log_size":2209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:39.074879+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9027584,"num_objects":2204,"num_object_clones":0,"num_object_copies":4408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":91,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.161559+0000","last_change":"2024-09-10T15:52:31.159524+0000","last_active":"2024-09-10T15:52:31.161559+0000","last_peered":"2024-09-10T15:52:31.161559+0000","last_clean":"2024-09-10T15:52:31.161559+0000","last_became_active":"2024-09-10T15:52:31.159205+0000","last_became_peered":"2024-09-10T15:52:31.159205+0000","last_unstale":"2024-09-10T15:52:31.161559+0000","last_undegraded":"2024-09-10T15:52:31.161559+0000","last_fullsized":"2024-09-10T15:52:31.161559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:04.415297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'2222","reported_seq":2323,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.514826+0000","last_change":"2024-09-10T15:52:54.903563+0000","last_active":"2024-09-10T15:52:55.514826+0000","last_peered":"2024-09-10T15:52:55.514826+0000","last_clean":"2024-09-10T15:52:55.514826+0000","last_became_active":"2024-09-10T15:52:31.157034+0000","last_became_peered":"2024-09-10T15:52:31.157034+0000","last_unstale":"2024-09-10T15:52:55.514826+0000","last_undegraded":"2024-09-10T15:52:55.514826+0000","last_fullsized":"2024-09-10T15:52:55.514826+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":2222,"log_dups_size":0,"ondisk_log_size":2222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:09.119902+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9076736,"num_objects":2216,"num_object_clones":0,"num_object_copies":4432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'2277","reported_seq":2334,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-10T15:52:56.050984+0000","last_change":"2024-09-10T15:52:31.150765+0000","last_active":"2024-09-10T15:52:56.050984+0000","last_peered":"2024-09-10T15:52:56.050984+0000","last_clean":"2024-09-10T15:51:45.893657+0000","last_became_active":"2024-09-10T15:52:30.151806+0000","last_became_peered":"2024-09-10T15:52:30.151806+0000","last_unstale":"2024-09-10T15:52:56.050984+0000","last_undegraded":"2024-09-10T15:52:56.050984+0000","last_fullsized":"2024-09-10T15:52:56.050984+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2277,"log_dups_size":0,"ondisk_log_size":2277,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9318883,"num_objects":2276,"num_object_clones":0,"num_object_copies":4552,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2276,"num_write_kb":9101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2276}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.149171+0000","last_change":"2024-09-10T15:52:30.154393+0000","last_active":"2024-09-10T15:52:31.149171+0000","last_peered":"2024-09-10T15:52:31.149171+0000","last_clean":"2024-09-10T15:52:31.149171+0000","last_became_active":"2024-09-10T15:52:30.154111+0000","last_became_peered":"2024-09-10T15:52:30.154111+0000","last_unstale":"2024-09-10T15:52:31.149171+0000","last_undegraded":"2024-09-10T15:52:31.149171+0000","last_fullsized":"2024-09-10T15:52:31.149171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:53.944111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'2337","reported_seq":2390,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.055456+0000","last_change":"2024-09-10T15:52:30.159885+0000","last_active":"2024-09-10T15:52:56.055456+0000","last_peered":"2024-09-10T15:52:56.055456+0000","last_clean":"2024-09-10T15:51:57.078241+0000","last_became_active":"2024-09-10T15:52:30.159635+0000","last_became_peered":"2024-09-10T15:52:30.159635+0000","last_unstale":"2024-09-10T15:52:56.055456+0000","last_undegraded":"2024-09-10T15:52:30.148722+0000","last_fullsized":"2024-09-10T15:52:56.055456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2337,"log_dups_size":0,"ondisk_log_size":2337,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9541033,"num_objects":2332,"num_object_clones":0,"num_object_copies":4664,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2334,"num_write_kb":9320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2332}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.157409+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:52:31.157409+0000","last_peered":"2024-09-10T15:52:31.157409+0000","last_clean":"2024-09-10T15:52:31.157409+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:52:31.157409+0000","last_undegraded":"2024-09-10T15:52:31.157409+0000","last_fullsized":"2024-09-10T15:52:31.157409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'2268","reported_seq":2324,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.041474+0000","last_change":"2024-09-10T15:52:30.159788+0000","last_active":"2024-09-10T15:52:56.041474+0000","last_peered":"2024-09-10T15:52:56.041474+0000","last_clean":"2024-09-10T15:51:45.893570+0000","last_became_active":"2024-09-10T15:52:30.159560+0000","last_became_peered":"2024-09-10T15:52:30.159560+0000","last_unstale":"2024-09-10T15:52:56.041474+0000","last_undegraded":"2024-09-10T15:52:30.148492+0000","last_fullsized":"2024-09-10T15:52:56.041474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9274310,"num_objects":2266,"num_object_clones":0,"num_object_copies":4532,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2266,"num_write_kb":9058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2266}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.152595+0000","last_change":"2024-09-10T15:52:30.149270+0000","last_active":"2024-09-10T15:52:31.152595+0000","last_peered":"2024-09-10T15:52:31.152595+0000","last_clean":"2024-09-10T15:52:31.152595+0000","last_became_active":"2024-09-10T15:52:30.148972+0000","last_became_peered":"2024-09-10T15:52:30.148972+0000","last_unstale":"2024-09-10T15:52:31.152595+0000","last_undegraded":"2024-09-10T15:52:31.152595+0000","last_fullsized":"2024-09-10T15:52:31.152595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:21.166160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.158991+0000","last_change":"2024-09-10T15:52:31.155379+0000","last_active":"2024-09-10T15:52:31.158991+0000","last_peered":"2024-09-10T15:52:31.158991+0000","last_clean":"2024-09-10T15:52:31.158991+0000","last_became_active":"2024-09-10T15:52:31.155102+0000","last_became_peered":"2024-09-10T15:52:31.155102+0000","last_unstale":"2024-09-10T15:52:31.158991+0000","last_undegraded":"2024-09-10T15:52:31.158991+0000","last_fullsized":"2024-09-10T15:52:31.158991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.976123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'2372","reported_seq":2429,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.040215+0000","last_change":"2024-09-10T15:52:30.152339+0000","last_active":"2024-09-10T15:52:56.040215+0000","last_peered":"2024-09-10T15:52:56.040215+0000","last_clean":"2024-09-10T15:51:56.837465+0000","last_became_active":"2024-09-10T15:52:30.152069+0000","last_became_peered":"2024-09-10T15:52:30.152069+0000","last_unstale":"2024-09-10T15:52:56.040215+0000","last_undegraded":"2024-09-10T15:52:30.148182+0000","last_fullsized":"2024-09-10T15:52:56.040215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2372,"log_dups_size":0,"ondisk_log_size":2372,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9668492,"num_objects":2364,"num_object_clones":0,"num_object_copies":4728,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2369,"num_write_kb":9447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2363}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":150008793,"num_objects":36640,"num_object_clones":0,"num_object_copies":73280,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27109,"num_write_kb":108322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":299257856,"data_stored":299135014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36703,"ondisk_log_size":36703,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149713,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":244308,"kb_used_data":144640,"kb_used_omap":4,"kb_used_meta":99643,"kb_avail":94127532,"statfs":{"total":96636764160,"available":96386592768,"internally_reserved":0,"allocated":148111360,"data_stored":147871041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":102035117},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953482,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":222556,"kb_used_data":140940,"kb_used_omap":4,"kb_used_meta":81595,"kb_avail":94149284,"statfs":{"total":96636764160,"available":96408866816,"internally_reserved":0,"allocated":144322560,"data_stored":144107996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":83553962},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3,1,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151052288,"data_stored":151030610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":148090880,"data_stored":148090880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:52:58.296 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:52:58.798 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 21 20 41115 41095 7.6433 5.72266 0.00817556 0.0102151 2024-09-10T15:52:58.879 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:58.881 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:52:58.900 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-10T15:52:58.433232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":150008812,"num_objects":36642,"num_object_clones":0,"num_object_copies":73284,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27111,"num_write_kb":108324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36706,"ondisk_log_size":36706,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":466864,"kb_used_data":285580,"kb_used_omap":8,"kb_used_meta":181239,"kb_avail":188276816,"statfs":{"total":193273528320,"available":192795459584,"internally_reserved":0,"allocated":292433920,"data_stored":291979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":185589079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,7,1,6,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":73187077,"num_objects":17860,"num_object_clones":0,"num_object_copies":35720,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":17860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8302,"num_write_kb":33268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003027"},"pg_stats":[{"pgid":"2.f","version":"37'2383","reported_seq":2433,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:56.055712+0000","last_change":"2024-09-10T15:52:30.154389+0000","last_active":"2024-09-10T15:52:56.055712+0000","last_peered":"2024-09-10T15:52:56.055712+0000","last_clean":"2024-09-10T15:52:56.055712+0000","last_became_active":"2024-09-10T15:52:30.154140+0000","last_became_peered":"2024-09-10T15:52:30.154140+0000","last_unstale":"2024-09-10T15:52:56.055712+0000","last_undegraded":"2024-09-10T15:52:56.055712+0000","last_fullsized":"2024-09-10T15:52:56.055712+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2383,"log_dups_size":0,"ondisk_log_size":2383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:30.595214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9760768,"num_objects":2383,"num_object_clones":0,"num_object_copies":4766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2383,"num_write_kb":9532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'2305","reported_seq":2392,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.553675+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:52:55.553675+0000","last_peered":"2024-09-10T15:52:55.553675+0000","last_clean":"2024-09-10T15:52:55.553675+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:52:55.553675+0000","last_undegraded":"2024-09-10T15:52:55.553675+0000","last_fullsized":"2024-09-10T15:52:55.553675+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":2305,"log_dups_size":0,"ondisk_log_size":2305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9416704,"num_objects":2299,"num_object_clones":0,"num_object_copies":4598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":641,"num_write_kb":2564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'2232","reported_seq":2330,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.522714+0000","last_change":"2024-09-10T15:52:50.491262+0000","last_active":"2024-09-10T15:52:55.522714+0000","last_peered":"2024-09-10T15:52:55.522714+0000","last_clean":"2024-09-10T15:52:55.522714+0000","last_became_active":"2024-09-10T15:52:31.157749+0000","last_became_peered":"2024-09-10T15:52:31.157749+0000","last_unstale":"2024-09-10T15:52:55.522714+0000","last_undegraded":"2024-09-10T15:52:55.522714+0000","last_fullsized":"2024-09-10T15:52:55.522714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":2232,"log_dups_size":0,"ondisk_log_size":2232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:11.778645+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9125888,"num_objects":2228,"num_object_clones":0,"num_object_copies":4456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":500,"num_write_kb":2000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'2274","reported_seq":2333,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.553657+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:52:55.553657+0000","last_peered":"2024-09-10T15:52:55.553657+0000","last_clean":"2024-09-10T15:52:55.553657+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:52:55.553657+0000","last_undegraded":"2024-09-10T15:52:55.553657+0000","last_fullsized":"2024-09-10T15:52:55.553657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2274,"log_dups_size":0,"ondisk_log_size":2274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9314304,"num_objects":2274,"num_object_clones":0,"num_object_copies":4548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2274,"num_write_kb":9096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'2268","reported_seq":2327,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.020037+0000","last_change":"2024-09-10T15:52:30.152160+0000","last_active":"2024-09-10T15:52:56.020037+0000","last_peered":"2024-09-10T15:52:56.020037+0000","last_clean":"2024-09-10T15:51:56.229374+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:52:56.020037+0000","last_undegraded":"2024-09-10T15:52:30.147386+0000","last_fullsized":"2024-09-10T15:52:56.020037+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9262022,"num_objects":2263,"num_object_clones":0,"num_object_copies":4526,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2266,"num_write_kb":9048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2263}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'2272","reported_seq":2329,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.032243+0000","last_change":"2024-09-10T15:52:30.152575+0000","last_active":"2024-09-10T15:52:56.032243+0000","last_peered":"2024-09-10T15:52:56.032243+0000","last_clean":"2024-09-10T15:51:57.126582+0000","last_became_active":"2024-09-10T15:52:30.152332+0000","last_became_peered":"2024-09-10T15:52:30.152332+0000","last_unstale":"2024-09-10T15:52:56.032243+0000","last_undegraded":"2024-09-10T15:52:30.147849+0000","last_fullsized":"2024-09-10T15:52:56.032243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2272,"log_dups_size":0,"ondisk_log_size":2272,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9286598,"num_objects":2269,"num_object_clones":0,"num_object_copies":4538,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2270,"num_write_kb":9071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2269}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'2311","reported_seq":2368,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.020044+0000","last_change":"2024-09-10T15:52:30.157171+0000","last_active":"2024-09-10T15:52:56.020044+0000","last_peered":"2024-09-10T15:52:56.020044+0000","last_clean":"2024-09-10T15:51:56.469065+0000","last_became_active":"2024-09-10T15:52:30.156955+0000","last_became_peered":"2024-09-10T15:52:30.156955+0000","last_unstale":"2024-09-10T15:52:56.020044+0000","last_undegraded":"2024-09-10T15:52:30.148233+0000","last_fullsized":"2024-09-10T15:52:56.020044+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2311,"log_dups_size":0,"ondisk_log_size":2311,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9442246,"num_objects":2307,"num_object_clones":0,"num_object_copies":4614,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2309,"num_write_kb":9224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2307}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'2342","reported_seq":2403,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.043371+0000","last_change":"2024-09-10T15:52:30.152559+0000","last_active":"2024-09-10T15:52:56.043371+0000","last_peered":"2024-09-10T15:52:56.043371+0000","last_clean":"2024-09-10T15:51:42.870214+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:52:56.043371+0000","last_undegraded":"2024-09-10T15:52:30.147342+0000","last_fullsized":"2024-09-10T15:52:56.043371+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2342,"log_dups_size":0,"ondisk_log_size":2342,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9585123,"num_objects":2341,"num_object_clones":0,"num_object_copies":4682,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2341,"num_write_kb":9361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.151621+0000","last_change":"2024-09-10T15:52:30.152637+0000","last_active":"2024-09-10T15:52:31.151621+0000","last_peered":"2024-09-10T15:52:31.151621+0000","last_clean":"2024-09-10T15:52:31.151621+0000","last_became_active":"2024-09-10T15:52:30.151730+0000","last_became_peered":"2024-09-10T15:52:30.151730+0000","last_unstale":"2024-09-10T15:52:31.151621+0000","last_undegraded":"2024-09-10T15:52:31.151621+0000","last_fullsized":"2024-09-10T15:52:31.151621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:09.577308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'2358","reported_seq":2462,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.538822+0000","last_change":"2024-09-10T15:52:52.645800+0000","last_active":"2024-09-10T15:52:55.538822+0000","last_peered":"2024-09-10T15:52:55.538822+0000","last_clean":"2024-09-10T15:52:55.538822+0000","last_became_active":"2024-09-10T15:52:31.157754+0000","last_became_peered":"2024-09-10T15:52:31.157754+0000","last_unstale":"2024-09-10T15:52:55.538822+0000","last_undegraded":"2024-09-10T15:52:55.538822+0000","last_fullsized":"2024-09-10T15:52:55.538822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":2358,"log_dups_size":0,"ondisk_log_size":2358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:37.611240+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9629696,"num_objects":2351,"num_object_clones":0,"num_object_copies":4702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":335,"num_write_kb":1340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.160328+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:52:31.160328+0000","last_peered":"2024-09-10T15:52:31.160328+0000","last_clean":"2024-09-10T15:52:31.160328+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:52:31.160328+0000","last_undegraded":"2024-09-10T15:52:31.160328+0000","last_fullsized":"2024-09-10T15:52:31.160328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'2273","reported_seq":2323,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.005533+0000","last_change":"2024-09-10T15:52:30.152404+0000","last_active":"2024-09-10T15:52:56.005533+0000","last_peered":"2024-09-10T15:52:56.005533+0000","last_clean":"2024-09-10T15:51:57.174499+0000","last_became_active":"2024-09-10T15:52:30.152112+0000","last_became_peered":"2024-09-10T15:52:30.152112+0000","last_unstale":"2024-09-10T15:52:56.005533+0000","last_undegraded":"2024-09-10T15:52:30.147626+0000","last_fullsized":"2024-09-10T15:52:56.005533+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2273,"log_dups_size":0,"ondisk_log_size":2273,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9278406,"num_objects":2267,"num_object_clones":0,"num_object_copies":4534,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2271,"num_write_kb":9064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2267}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:31.152685+0000","last_change":"2024-09-10T15:52:30.152323+0000","last_active":"2024-09-10T15:52:31.152685+0000","last_peered":"2024-09-10T15:52:31.152685+0000","last_clean":"2024-09-10T15:52:08.013728+0000","last_became_active":"2024-09-10T15:52:30.151717+0000","last_became_peered":"2024-09-10T15:52:30.151717+0000","last_unstale":"2024-09-10T15:52:31.152685+0000","last_undegraded":"2024-09-10T15:52:30.144026+0000","last_fullsized":"2024-09-10T15:52:31.152685+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'2209","reported_seq":2309,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.461790+0000","last_change":"2024-09-10T15:52:53.666934+0000","last_active":"2024-09-10T15:52:55.461790+0000","last_peered":"2024-09-10T15:52:55.461790+0000","last_clean":"2024-09-10T15:52:55.461790+0000","last_became_active":"2024-09-10T15:52:31.157172+0000","last_became_peered":"2024-09-10T15:52:31.157172+0000","last_unstale":"2024-09-10T15:52:55.461790+0000","last_undegraded":"2024-09-10T15:52:55.461790+0000","last_fullsized":"2024-09-10T15:52:55.461790+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":2209,"log_dups_size":0,"ondisk_log_size":2209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:39.074879+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9027584,"num_objects":2204,"num_object_clones":0,"num_object_copies":4408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":91,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.161559+0000","last_change":"2024-09-10T15:52:31.159524+0000","last_active":"2024-09-10T15:52:31.161559+0000","last_peered":"2024-09-10T15:52:31.161559+0000","last_clean":"2024-09-10T15:52:31.161559+0000","last_became_active":"2024-09-10T15:52:31.159205+0000","last_became_peered":"2024-09-10T15:52:31.159205+0000","last_unstale":"2024-09-10T15:52:31.161559+0000","last_undegraded":"2024-09-10T15:52:31.161559+0000","last_fullsized":"2024-09-10T15:52:31.161559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:04.415297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'2222","reported_seq":2323,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.514826+0000","last_change":"2024-09-10T15:52:54.903563+0000","last_active":"2024-09-10T15:52:55.514826+0000","last_peered":"2024-09-10T15:52:55.514826+0000","last_clean":"2024-09-10T15:52:55.514826+0000","last_became_active":"2024-09-10T15:52:31.157034+0000","last_became_peered":"2024-09-10T15:52:31.157034+0000","last_unstale":"2024-09-10T15:52:55.514826+0000","last_undegraded":"2024-09-10T15:52:55.514826+0000","last_fullsized":"2024-09-10T15:52:55.514826+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":2222,"log_dups_size":0,"ondisk_log_size":2222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:09.119902+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9076736,"num_objects":2216,"num_object_clones":0,"num_object_copies":4432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'2277","reported_seq":2334,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-10T15:52:56.050984+0000","last_change":"2024-09-10T15:52:31.150765+0000","last_active":"2024-09-10T15:52:56.050984+0000","last_peered":"2024-09-10T15:52:56.050984+0000","last_clean":"2024-09-10T15:51:45.893657+0000","last_became_active":"2024-09-10T15:52:30.151806+0000","last_became_peered":"2024-09-10T15:52:30.151806+0000","last_unstale":"2024-09-10T15:52:56.050984+0000","last_undegraded":"2024-09-10T15:52:56.050984+0000","last_fullsized":"2024-09-10T15:52:56.050984+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2277,"log_dups_size":0,"ondisk_log_size":2277,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9318883,"num_objects":2276,"num_object_clones":0,"num_object_copies":4552,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2276,"num_write_kb":9101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2276}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.149171+0000","last_change":"2024-09-10T15:52:30.154393+0000","last_active":"2024-09-10T15:52:31.149171+0000","last_peered":"2024-09-10T15:52:31.149171+0000","last_clean":"2024-09-10T15:52:31.149171+0000","last_became_active":"2024-09-10T15:52:30.154111+0000","last_became_peered":"2024-09-10T15:52:30.154111+0000","last_unstale":"2024-09-10T15:52:31.149171+0000","last_undegraded":"2024-09-10T15:52:31.149171+0000","last_fullsized":"2024-09-10T15:52:31.149171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:53.944111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'2337","reported_seq":2390,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.055456+0000","last_change":"2024-09-10T15:52:30.159885+0000","last_active":"2024-09-10T15:52:56.055456+0000","last_peered":"2024-09-10T15:52:56.055456+0000","last_clean":"2024-09-10T15:51:57.078241+0000","last_became_active":"2024-09-10T15:52:30.159635+0000","last_became_peered":"2024-09-10T15:52:30.159635+0000","last_unstale":"2024-09-10T15:52:56.055456+0000","last_undegraded":"2024-09-10T15:52:30.148722+0000","last_fullsized":"2024-09-10T15:52:56.055456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2337,"log_dups_size":0,"ondisk_log_size":2337,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9541033,"num_objects":2332,"num_object_clones":0,"num_object_copies":4664,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2334,"num_write_kb":9320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2332}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.157409+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:52:31.157409+0000","last_peered":"2024-09-10T15:52:31.157409+0000","last_clean":"2024-09-10T15:52:31.157409+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:52:31.157409+0000","last_undegraded":"2024-09-10T15:52:31.157409+0000","last_fullsized":"2024-09-10T15:52:31.157409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'2268","reported_seq":2324,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.041474+0000","last_change":"2024-09-10T15:52:30.159788+0000","last_active":"2024-09-10T15:52:56.041474+0000","last_peered":"2024-09-10T15:52:56.041474+0000","last_clean":"2024-09-10T15:51:45.893570+0000","last_became_active":"2024-09-10T15:52:30.159560+0000","last_became_peered":"2024-09-10T15:52:30.159560+0000","last_unstale":"2024-09-10T15:52:56.041474+0000","last_undegraded":"2024-09-10T15:52:30.148492+0000","last_fullsized":"2024-09-10T15:52:56.041474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9274310,"num_objects":2266,"num_object_clones":0,"num_object_copies":4532,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2266,"num_write_kb":9058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2266}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.152595+0000","last_change":"2024-09-10T15:52:30.149270+0000","last_active":"2024-09-10T15:52:31.152595+0000","last_peered":"2024-09-10T15:52:31.152595+0000","last_clean":"2024-09-10T15:52:31.152595+0000","last_became_active":"2024-09-10T15:52:30.148972+0000","last_became_peered":"2024-09-10T15:52:30.148972+0000","last_unstale":"2024-09-10T15:52:31.152595+0000","last_undegraded":"2024-09-10T15:52:31.152595+0000","last_fullsized":"2024-09-10T15:52:31.152595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:21.166160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.158991+0000","last_change":"2024-09-10T15:52:31.155379+0000","last_active":"2024-09-10T15:52:31.158991+0000","last_peered":"2024-09-10T15:52:31.158991+0000","last_clean":"2024-09-10T15:52:31.158991+0000","last_became_active":"2024-09-10T15:52:31.155102+0000","last_became_peered":"2024-09-10T15:52:31.155102+0000","last_unstale":"2024-09-10T15:52:31.158991+0000","last_undegraded":"2024-09-10T15:52:31.158991+0000","last_fullsized":"2024-09-10T15:52:31.158991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.976123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'2372","reported_seq":2429,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.040215+0000","last_change":"2024-09-10T15:52:30.152339+0000","last_active":"2024-09-10T15:52:56.040215+0000","last_peered":"2024-09-10T15:52:56.040215+0000","last_clean":"2024-09-10T15:51:56.837465+0000","last_became_active":"2024-09-10T15:52:30.152069+0000","last_became_peered":"2024-09-10T15:52:30.152069+0000","last_unstale":"2024-09-10T15:52:56.040215+0000","last_undegraded":"2024-09-10T15:52:30.148182+0000","last_fullsized":"2024-09-10T15:52:56.040215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2372,"log_dups_size":0,"ondisk_log_size":2372,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9668492,"num_objects":2364,"num_object_clones":0,"num_object_copies":4728,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2369,"num_write_kb":9447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2363}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":150008793,"num_objects":36640,"num_object_clones":0,"num_object_copies":73280,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27109,"num_write_kb":108322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":299257856,"data_stored":299135014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36703,"ondisk_log_size":36703,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149713,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":244308,"kb_used_data":144640,"kb_used_omap":4,"kb_used_meta":99643,"kb_avail":94127532,"statfs":{"total":96636764160,"available":96386592768,"internally_reserved":0,"allocated":148111360,"data_stored":147871041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":102035117},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953482,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":222556,"kb_used_data":140940,"kb_used_omap":4,"kb_used_meta":81595,"kb_avail":94149284,"statfs":{"total":96636764160,"available":96408866816,"internally_reserved":0,"allocated":144322560,"data_stored":144107996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":83553962},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3,1,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151052288,"data_stored":151030610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":148090880,"data_stored":148090880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:52:58.902 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:52:59.443 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:52:59.443 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:52:59.464 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-10T15:52:58.433232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":150008812,"num_objects":36642,"num_object_clones":0,"num_object_copies":73284,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27111,"num_write_kb":108324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36706,"ondisk_log_size":36706,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":466864,"kb_used_data":285580,"kb_used_omap":8,"kb_used_meta":181239,"kb_avail":188276816,"statfs":{"total":193273528320,"available":192795459584,"internally_reserved":0,"allocated":292433920,"data_stored":291979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":185589079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,7,1,6,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":73187077,"num_objects":17860,"num_object_clones":0,"num_object_copies":35720,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":17860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8302,"num_write_kb":33268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003027"},"pg_stats":[{"pgid":"2.f","version":"37'2383","reported_seq":2433,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:56.055712+0000","last_change":"2024-09-10T15:52:30.154389+0000","last_active":"2024-09-10T15:52:56.055712+0000","last_peered":"2024-09-10T15:52:56.055712+0000","last_clean":"2024-09-10T15:52:56.055712+0000","last_became_active":"2024-09-10T15:52:30.154140+0000","last_became_peered":"2024-09-10T15:52:30.154140+0000","last_unstale":"2024-09-10T15:52:56.055712+0000","last_undegraded":"2024-09-10T15:52:56.055712+0000","last_fullsized":"2024-09-10T15:52:56.055712+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2383,"log_dups_size":0,"ondisk_log_size":2383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:30.595214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9760768,"num_objects":2383,"num_object_clones":0,"num_object_copies":4766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2383,"num_write_kb":9532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'2305","reported_seq":2392,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.553675+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:52:55.553675+0000","last_peered":"2024-09-10T15:52:55.553675+0000","last_clean":"2024-09-10T15:52:55.553675+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:52:55.553675+0000","last_undegraded":"2024-09-10T15:52:55.553675+0000","last_fullsized":"2024-09-10T15:52:55.553675+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":2305,"log_dups_size":0,"ondisk_log_size":2305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9416704,"num_objects":2299,"num_object_clones":0,"num_object_copies":4598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":641,"num_write_kb":2564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'2232","reported_seq":2330,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.522714+0000","last_change":"2024-09-10T15:52:50.491262+0000","last_active":"2024-09-10T15:52:55.522714+0000","last_peered":"2024-09-10T15:52:55.522714+0000","last_clean":"2024-09-10T15:52:55.522714+0000","last_became_active":"2024-09-10T15:52:31.157749+0000","last_became_peered":"2024-09-10T15:52:31.157749+0000","last_unstale":"2024-09-10T15:52:55.522714+0000","last_undegraded":"2024-09-10T15:52:55.522714+0000","last_fullsized":"2024-09-10T15:52:55.522714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":2232,"log_dups_size":0,"ondisk_log_size":2232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:11.778645+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9125888,"num_objects":2228,"num_object_clones":0,"num_object_copies":4456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":500,"num_write_kb":2000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'2274","reported_seq":2333,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.553657+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:52:55.553657+0000","last_peered":"2024-09-10T15:52:55.553657+0000","last_clean":"2024-09-10T15:52:55.553657+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:52:55.553657+0000","last_undegraded":"2024-09-10T15:52:55.553657+0000","last_fullsized":"2024-09-10T15:52:55.553657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2274,"log_dups_size":0,"ondisk_log_size":2274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9314304,"num_objects":2274,"num_object_clones":0,"num_object_copies":4548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2274,"num_write_kb":9096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'2268","reported_seq":2327,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.020037+0000","last_change":"2024-09-10T15:52:30.152160+0000","last_active":"2024-09-10T15:52:56.020037+0000","last_peered":"2024-09-10T15:52:56.020037+0000","last_clean":"2024-09-10T15:51:56.229374+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:52:56.020037+0000","last_undegraded":"2024-09-10T15:52:30.147386+0000","last_fullsized":"2024-09-10T15:52:56.020037+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9262022,"num_objects":2263,"num_object_clones":0,"num_object_copies":4526,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2266,"num_write_kb":9048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2263}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'2272","reported_seq":2329,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.032243+0000","last_change":"2024-09-10T15:52:30.152575+0000","last_active":"2024-09-10T15:52:56.032243+0000","last_peered":"2024-09-10T15:52:56.032243+0000","last_clean":"2024-09-10T15:51:57.126582+0000","last_became_active":"2024-09-10T15:52:30.152332+0000","last_became_peered":"2024-09-10T15:52:30.152332+0000","last_unstale":"2024-09-10T15:52:56.032243+0000","last_undegraded":"2024-09-10T15:52:30.147849+0000","last_fullsized":"2024-09-10T15:52:56.032243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2272,"log_dups_size":0,"ondisk_log_size":2272,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9286598,"num_objects":2269,"num_object_clones":0,"num_object_copies":4538,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2270,"num_write_kb":9071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2269}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'2311","reported_seq":2368,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.020044+0000","last_change":"2024-09-10T15:52:30.157171+0000","last_active":"2024-09-10T15:52:56.020044+0000","last_peered":"2024-09-10T15:52:56.020044+0000","last_clean":"2024-09-10T15:51:56.469065+0000","last_became_active":"2024-09-10T15:52:30.156955+0000","last_became_peered":"2024-09-10T15:52:30.156955+0000","last_unstale":"2024-09-10T15:52:56.020044+0000","last_undegraded":"2024-09-10T15:52:30.148233+0000","last_fullsized":"2024-09-10T15:52:56.020044+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2311,"log_dups_size":0,"ondisk_log_size":2311,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9442246,"num_objects":2307,"num_object_clones":0,"num_object_copies":4614,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2309,"num_write_kb":9224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2307}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'2342","reported_seq":2403,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.043371+0000","last_change":"2024-09-10T15:52:30.152559+0000","last_active":"2024-09-10T15:52:56.043371+0000","last_peered":"2024-09-10T15:52:56.043371+0000","last_clean":"2024-09-10T15:51:42.870214+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:52:56.043371+0000","last_undegraded":"2024-09-10T15:52:30.147342+0000","last_fullsized":"2024-09-10T15:52:56.043371+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2342,"log_dups_size":0,"ondisk_log_size":2342,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9585123,"num_objects":2341,"num_object_clones":0,"num_object_copies":4682,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2341,"num_write_kb":9361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.151621+0000","last_change":"2024-09-10T15:52:30.152637+0000","last_active":"2024-09-10T15:52:31.151621+0000","last_peered":"2024-09-10T15:52:31.151621+0000","last_clean":"2024-09-10T15:52:31.151621+0000","last_became_active":"2024-09-10T15:52:30.151730+0000","last_became_peered":"2024-09-10T15:52:30.151730+0000","last_unstale":"2024-09-10T15:52:31.151621+0000","last_undegraded":"2024-09-10T15:52:31.151621+0000","last_fullsized":"2024-09-10T15:52:31.151621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:09.577308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'2358","reported_seq":2462,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.538822+0000","last_change":"2024-09-10T15:52:52.645800+0000","last_active":"2024-09-10T15:52:55.538822+0000","last_peered":"2024-09-10T15:52:55.538822+0000","last_clean":"2024-09-10T15:52:55.538822+0000","last_became_active":"2024-09-10T15:52:31.157754+0000","last_became_peered":"2024-09-10T15:52:31.157754+0000","last_unstale":"2024-09-10T15:52:55.538822+0000","last_undegraded":"2024-09-10T15:52:55.538822+0000","last_fullsized":"2024-09-10T15:52:55.538822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":2358,"log_dups_size":0,"ondisk_log_size":2358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:37.611240+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9629696,"num_objects":2351,"num_object_clones":0,"num_object_copies":4702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":335,"num_write_kb":1340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.160328+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:52:31.160328+0000","last_peered":"2024-09-10T15:52:31.160328+0000","last_clean":"2024-09-10T15:52:31.160328+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:52:31.160328+0000","last_undegraded":"2024-09-10T15:52:31.160328+0000","last_fullsized":"2024-09-10T15:52:31.160328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'2273","reported_seq":2323,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.005533+0000","last_change":"2024-09-10T15:52:30.152404+0000","last_active":"2024-09-10T15:52:56.005533+0000","last_peered":"2024-09-10T15:52:56.005533+0000","last_clean":"2024-09-10T15:51:57.174499+0000","last_became_active":"2024-09-10T15:52:30.152112+0000","last_became_peered":"2024-09-10T15:52:30.152112+0000","last_unstale":"2024-09-10T15:52:56.005533+0000","last_undegraded":"2024-09-10T15:52:30.147626+0000","last_fullsized":"2024-09-10T15:52:56.005533+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2273,"log_dups_size":0,"ondisk_log_size":2273,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9278406,"num_objects":2267,"num_object_clones":0,"num_object_copies":4534,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2271,"num_write_kb":9064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2267}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:31.152685+0000","last_change":"2024-09-10T15:52:30.152323+0000","last_active":"2024-09-10T15:52:31.152685+0000","last_peered":"2024-09-10T15:52:31.152685+0000","last_clean":"2024-09-10T15:52:08.013728+0000","last_became_active":"2024-09-10T15:52:30.151717+0000","last_became_peered":"2024-09-10T15:52:30.151717+0000","last_unstale":"2024-09-10T15:52:31.152685+0000","last_undegraded":"2024-09-10T15:52:30.144026+0000","last_fullsized":"2024-09-10T15:52:31.152685+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'2209","reported_seq":2309,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.461790+0000","last_change":"2024-09-10T15:52:53.666934+0000","last_active":"2024-09-10T15:52:55.461790+0000","last_peered":"2024-09-10T15:52:55.461790+0000","last_clean":"2024-09-10T15:52:55.461790+0000","last_became_active":"2024-09-10T15:52:31.157172+0000","last_became_peered":"2024-09-10T15:52:31.157172+0000","last_unstale":"2024-09-10T15:52:55.461790+0000","last_undegraded":"2024-09-10T15:52:55.461790+0000","last_fullsized":"2024-09-10T15:52:55.461790+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":2209,"log_dups_size":0,"ondisk_log_size":2209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:39.074879+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9027584,"num_objects":2204,"num_object_clones":0,"num_object_copies":4408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":91,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.161559+0000","last_change":"2024-09-10T15:52:31.159524+0000","last_active":"2024-09-10T15:52:31.161559+0000","last_peered":"2024-09-10T15:52:31.161559+0000","last_clean":"2024-09-10T15:52:31.161559+0000","last_became_active":"2024-09-10T15:52:31.159205+0000","last_became_peered":"2024-09-10T15:52:31.159205+0000","last_unstale":"2024-09-10T15:52:31.161559+0000","last_undegraded":"2024-09-10T15:52:31.161559+0000","last_fullsized":"2024-09-10T15:52:31.161559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:04.415297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'2222","reported_seq":2323,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:55.514826+0000","last_change":"2024-09-10T15:52:54.903563+0000","last_active":"2024-09-10T15:52:55.514826+0000","last_peered":"2024-09-10T15:52:55.514826+0000","last_clean":"2024-09-10T15:52:55.514826+0000","last_became_active":"2024-09-10T15:52:31.157034+0000","last_became_peered":"2024-09-10T15:52:31.157034+0000","last_unstale":"2024-09-10T15:52:55.514826+0000","last_undegraded":"2024-09-10T15:52:55.514826+0000","last_fullsized":"2024-09-10T15:52:55.514826+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":2222,"log_dups_size":0,"ondisk_log_size":2222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:09.119902+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9076736,"num_objects":2216,"num_object_clones":0,"num_object_copies":4432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'2277","reported_seq":2334,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-10T15:52:56.050984+0000","last_change":"2024-09-10T15:52:31.150765+0000","last_active":"2024-09-10T15:52:56.050984+0000","last_peered":"2024-09-10T15:52:56.050984+0000","last_clean":"2024-09-10T15:51:45.893657+0000","last_became_active":"2024-09-10T15:52:30.151806+0000","last_became_peered":"2024-09-10T15:52:30.151806+0000","last_unstale":"2024-09-10T15:52:56.050984+0000","last_undegraded":"2024-09-10T15:52:56.050984+0000","last_fullsized":"2024-09-10T15:52:56.050984+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2277,"log_dups_size":0,"ondisk_log_size":2277,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9318883,"num_objects":2276,"num_object_clones":0,"num_object_copies":4552,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2276,"num_write_kb":9101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2276}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.149171+0000","last_change":"2024-09-10T15:52:30.154393+0000","last_active":"2024-09-10T15:52:31.149171+0000","last_peered":"2024-09-10T15:52:31.149171+0000","last_clean":"2024-09-10T15:52:31.149171+0000","last_became_active":"2024-09-10T15:52:30.154111+0000","last_became_peered":"2024-09-10T15:52:30.154111+0000","last_unstale":"2024-09-10T15:52:31.149171+0000","last_undegraded":"2024-09-10T15:52:31.149171+0000","last_fullsized":"2024-09-10T15:52:31.149171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:53.944111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'2337","reported_seq":2390,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.055456+0000","last_change":"2024-09-10T15:52:30.159885+0000","last_active":"2024-09-10T15:52:56.055456+0000","last_peered":"2024-09-10T15:52:56.055456+0000","last_clean":"2024-09-10T15:51:57.078241+0000","last_became_active":"2024-09-10T15:52:30.159635+0000","last_became_peered":"2024-09-10T15:52:30.159635+0000","last_unstale":"2024-09-10T15:52:56.055456+0000","last_undegraded":"2024-09-10T15:52:30.148722+0000","last_fullsized":"2024-09-10T15:52:56.055456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2337,"log_dups_size":0,"ondisk_log_size":2337,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9541033,"num_objects":2332,"num_object_clones":0,"num_object_copies":4664,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2334,"num_write_kb":9320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2332}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.157409+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:52:31.157409+0000","last_peered":"2024-09-10T15:52:31.157409+0000","last_clean":"2024-09-10T15:52:31.157409+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:52:31.157409+0000","last_undegraded":"2024-09-10T15:52:31.157409+0000","last_fullsized":"2024-09-10T15:52:31.157409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'2268","reported_seq":2324,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.041474+0000","last_change":"2024-09-10T15:52:30.159788+0000","last_active":"2024-09-10T15:52:56.041474+0000","last_peered":"2024-09-10T15:52:56.041474+0000","last_clean":"2024-09-10T15:51:45.893570+0000","last_became_active":"2024-09-10T15:52:30.159560+0000","last_became_peered":"2024-09-10T15:52:30.159560+0000","last_unstale":"2024-09-10T15:52:56.041474+0000","last_undegraded":"2024-09-10T15:52:30.148492+0000","last_fullsized":"2024-09-10T15:52:56.041474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9274310,"num_objects":2266,"num_object_clones":0,"num_object_copies":4532,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2266,"num_write_kb":9058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2266}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.152595+0000","last_change":"2024-09-10T15:52:30.149270+0000","last_active":"2024-09-10T15:52:31.152595+0000","last_peered":"2024-09-10T15:52:31.152595+0000","last_clean":"2024-09-10T15:52:31.152595+0000","last_became_active":"2024-09-10T15:52:30.148972+0000","last_became_peered":"2024-09-10T15:52:30.148972+0000","last_unstale":"2024-09-10T15:52:31.152595+0000","last_undegraded":"2024-09-10T15:52:31.152595+0000","last_fullsized":"2024-09-10T15:52:31.152595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:21.166160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.158991+0000","last_change":"2024-09-10T15:52:31.155379+0000","last_active":"2024-09-10T15:52:31.158991+0000","last_peered":"2024-09-10T15:52:31.158991+0000","last_clean":"2024-09-10T15:52:31.158991+0000","last_became_active":"2024-09-10T15:52:31.155102+0000","last_became_peered":"2024-09-10T15:52:31.155102+0000","last_unstale":"2024-09-10T15:52:31.158991+0000","last_undegraded":"2024-09-10T15:52:31.158991+0000","last_fullsized":"2024-09-10T15:52:31.158991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.976123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'2372","reported_seq":2429,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:56.040215+0000","last_change":"2024-09-10T15:52:30.152339+0000","last_active":"2024-09-10T15:52:56.040215+0000","last_peered":"2024-09-10T15:52:56.040215+0000","last_clean":"2024-09-10T15:51:56.837465+0000","last_became_active":"2024-09-10T15:52:30.152069+0000","last_became_peered":"2024-09-10T15:52:30.152069+0000","last_unstale":"2024-09-10T15:52:56.040215+0000","last_undegraded":"2024-09-10T15:52:30.148182+0000","last_fullsized":"2024-09-10T15:52:56.040215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2372,"log_dups_size":0,"ondisk_log_size":2372,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9668492,"num_objects":2364,"num_object_clones":0,"num_object_copies":4728,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2369,"num_write_kb":9447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2363}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":150008793,"num_objects":36640,"num_object_clones":0,"num_object_copies":73280,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27109,"num_write_kb":108322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":299257856,"data_stored":299135014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36703,"ondisk_log_size":36703,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149713,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":244308,"kb_used_data":144640,"kb_used_omap":4,"kb_used_meta":99643,"kb_avail":94127532,"statfs":{"total":96636764160,"available":96386592768,"internally_reserved":0,"allocated":148111360,"data_stored":147871041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":102035117},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953482,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":222556,"kb_used_data":140940,"kb_used_omap":4,"kb_used_meta":81595,"kb_avail":94149284,"statfs":{"total":96636764160,"available":96408866816,"internally_reserved":0,"allocated":144322560,"data_stored":144107996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":83553962},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3,1,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":151052288,"data_stored":151030610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":148090880,"data_stored":148090880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:52:59.799 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 22 20 42739 42719 7.58421 6.34375 0.00885508 0.0102948 2024-09-10T15:53:00.800 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 23 20 43802 43782 7.43454 4.15234 0.0116566 0.0105031 2024-09-10T15:53:01.800 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 24 20 45498 45478 7.40079 6.625 0.0223426 0.0105503 2024-09-10T15:53:02.467 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:02.800 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 25 20 47018 46998 7.34224 5.9375 0.0187651 0.010634 2024-09-10T15:53:02.963 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:02.963 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:02.974 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-10T15:53:02.434083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":177210348,"num_objects":43283,"num_object_clones":0,"num_object_copies":86566,"num_objects_missing_on_primary":6,"num_objects_missing":6,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33752,"num_write_kb":134888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43347,"ondisk_log_size":43347,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":552540,"kb_used_data":340732,"kb_used_omap":8,"kb_used_meta":211767,"kb_avail":188191140,"statfs":{"total":193273528320,"available":192707727360,"internally_reserved":0,"allocated":348909568,"data_stored":348454685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":216849751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3,1,10,9,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":84909829,"num_objects":20722,"num_object_clones":0,"num_object_copies":41444,"num_objects_missing_on_primary":-18,"num_objects_missing":-18,"num_objects_degraded":-37,"num_objects_misplaced":0,"num_objects_unfound":-24,"num_objects_dirty":20722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11164,"num_write_kb":44716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002889"},"pg_stats":[{"pgid":"2.f","version":"37'2851","reported_seq":2902,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.040948+0000","last_change":"2024-09-10T15:52:30.154389+0000","last_active":"2024-09-10T15:53:01.040948+0000","last_peered":"2024-09-10T15:53:01.040948+0000","last_clean":"2024-09-10T15:53:01.040948+0000","last_became_active":"2024-09-10T15:52:30.154140+0000","last_became_peered":"2024-09-10T15:52:30.154140+0000","last_unstale":"2024-09-10T15:53:01.040948+0000","last_undegraded":"2024-09-10T15:53:01.040948+0000","last_fullsized":"2024-09-10T15:53:01.040948+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2851,"log_dups_size":0,"ondisk_log_size":2851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:30.595214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11677696,"num_objects":2851,"num_object_clones":0,"num_object_copies":5702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2851,"num_write_kb":11404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'2659","reported_seq":2746,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:59.155003+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:52:59.155003+0000","last_peered":"2024-09-10T15:52:59.155003+0000","last_clean":"2024-09-10T15:52:59.155003+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:52:59.155003+0000","last_undegraded":"2024-09-10T15:52:59.155003+0000","last_fullsized":"2024-09-10T15:52:59.155003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":2659,"log_dups_size":0,"ondisk_log_size":2659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10866688,"num_objects":2653,"num_object_clones":0,"num_object_copies":5306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":995,"num_write_kb":3980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'2572","reported_seq":2670,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:59.164350+0000","last_change":"2024-09-10T15:52:50.491262+0000","last_active":"2024-09-10T15:52:59.164350+0000","last_peered":"2024-09-10T15:52:59.164350+0000","last_clean":"2024-09-10T15:52:59.164350+0000","last_became_active":"2024-09-10T15:52:31.157749+0000","last_became_peered":"2024-09-10T15:52:31.157749+0000","last_unstale":"2024-09-10T15:52:59.164350+0000","last_undegraded":"2024-09-10T15:52:59.164350+0000","last_fullsized":"2024-09-10T15:52:59.164350+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":2572,"log_dups_size":0,"ondisk_log_size":2572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:11.778645+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10518528,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":3360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'2623","reported_seq":2683,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:59.159418+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:52:59.159418+0000","last_peered":"2024-09-10T15:52:59.159418+0000","last_clean":"2024-09-10T15:52:59.159418+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:52:59.159418+0000","last_undegraded":"2024-09-10T15:52:59.159418+0000","last_fullsized":"2024-09-10T15:52:59.159418+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2623,"log_dups_size":0,"ondisk_log_size":2623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10743808,"num_objects":2623,"num_object_clones":0,"num_object_copies":5246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2623,"num_write_kb":10492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'2708","reported_seq":2778,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.037006+0000","last_change":"2024-09-10T15:53:00.817069+0000","last_active":"2024-09-10T15:53:01.037006+0000","last_peered":"2024-09-10T15:53:01.037006+0000","last_clean":"2024-09-10T15:53:01.037006+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:01.037006+0000","last_undegraded":"2024-09-10T15:53:01.037006+0000","last_fullsized":"2024-09-10T15:53:01.037006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2708,"log_dups_size":0,"ondisk_log_size":2708,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11064262,"num_objects":2703,"num_object_clones":0,"num_object_copies":5406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2706,"num_write_kb":10808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'2728","reported_seq":2796,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.043527+0000","last_change":"2024-09-10T15:53:01.031975+0000","last_active":"2024-09-10T15:53:01.043527+0000","last_peered":"2024-09-10T15:53:01.043527+0000","last_clean":"2024-09-10T15:53:01.043527+0000","last_became_active":"2024-09-10T15:52:30.152332+0000","last_became_peered":"2024-09-10T15:52:30.152332+0000","last_unstale":"2024-09-10T15:53:01.043527+0000","last_undegraded":"2024-09-10T15:53:01.043527+0000","last_fullsized":"2024-09-10T15:53:01.043527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2728,"log_dups_size":0,"ondisk_log_size":2728,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11154374,"num_objects":2725,"num_object_clones":0,"num_object_copies":5450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2726,"num_write_kb":10895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'2756","reported_seq":2817,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-10T15:53:01.059087+0000","last_change":"2024-09-10T15:53:01.045435+0000","last_active":"2024-09-10T15:53:01.059087+0000","last_peered":"2024-09-10T15:53:01.059087+0000","last_clean":"2024-09-10T15:51:56.469065+0000","last_became_active":"2024-09-10T15:52:30.156955+0000","last_became_peered":"2024-09-10T15:52:30.156955+0000","last_unstale":"2024-09-10T15:53:01.059087+0000","last_undegraded":"2024-09-10T15:53:01.059087+0000","last_fullsized":"2024-09-10T15:53:01.059087+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2756,"log_dups_size":0,"ondisk_log_size":2756,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11264966,"num_objects":2752,"num_object_clones":0,"num_object_copies":5504,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2754,"num_write_kb":11004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2752}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'2779","reported_seq":2848,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.015193+0000","last_change":"2024-09-10T15:53:00.993088+0000","last_active":"2024-09-10T15:53:01.015193+0000","last_peered":"2024-09-10T15:53:01.015193+0000","last_clean":"2024-09-10T15:53:01.015193+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:01.015193+0000","last_undegraded":"2024-09-10T15:53:01.015193+0000","last_fullsized":"2024-09-10T15:53:01.015193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2779,"log_dups_size":0,"ondisk_log_size":2779,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11375075,"num_objects":2778,"num_object_clones":0,"num_object_copies":5556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2778,"num_write_kb":11109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.151621+0000","last_change":"2024-09-10T15:52:30.152637+0000","last_active":"2024-09-10T15:52:31.151621+0000","last_peered":"2024-09-10T15:52:31.151621+0000","last_clean":"2024-09-10T15:52:31.151621+0000","last_became_active":"2024-09-10T15:52:30.151730+0000","last_became_peered":"2024-09-10T15:52:30.151730+0000","last_unstale":"2024-09-10T15:52:31.151621+0000","last_undegraded":"2024-09-10T15:52:31.151621+0000","last_fullsized":"2024-09-10T15:52:31.151621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:09.577308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'2702","reported_seq":2806,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:59.159466+0000","last_change":"2024-09-10T15:52:52.645800+0000","last_active":"2024-09-10T15:52:59.159466+0000","last_peered":"2024-09-10T15:52:59.159466+0000","last_clean":"2024-09-10T15:52:59.159466+0000","last_became_active":"2024-09-10T15:52:31.157754+0000","last_became_peered":"2024-09-10T15:52:31.157754+0000","last_unstale":"2024-09-10T15:52:59.159466+0000","last_undegraded":"2024-09-10T15:52:59.159466+0000","last_fullsized":"2024-09-10T15:52:59.159466+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":2702,"log_dups_size":0,"ondisk_log_size":2702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:37.611240+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11038720,"num_objects":2695,"num_object_clones":0,"num_object_copies":5390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":679,"num_write_kb":2716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.160328+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:52:31.160328+0000","last_peered":"2024-09-10T15:52:31.160328+0000","last_clean":"2024-09-10T15:52:31.160328+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:52:31.160328+0000","last_undegraded":"2024-09-10T15:52:31.160328+0000","last_fullsized":"2024-09-10T15:52:31.160328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'2736","reported_seq":2797,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:00.998389+0000","last_change":"2024-09-10T15:53:00.964850+0000","last_active":"2024-09-10T15:53:00.998389+0000","last_peered":"2024-09-10T15:53:00.998389+0000","last_clean":"2024-09-10T15:53:00.998389+0000","last_became_active":"2024-09-10T15:52:30.152112+0000","last_became_peered":"2024-09-10T15:52:30.152112+0000","last_unstale":"2024-09-10T15:53:00.998389+0000","last_undegraded":"2024-09-10T15:53:00.998389+0000","last_fullsized":"2024-09-10T15:53:00.998389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2736,"log_dups_size":0,"ondisk_log_size":2736,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11174854,"num_objects":2730,"num_object_clones":0,"num_object_copies":5460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2734,"num_write_kb":10916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:31.152685+0000","last_change":"2024-09-10T15:52:30.152323+0000","last_active":"2024-09-10T15:52:31.152685+0000","last_peered":"2024-09-10T15:52:31.152685+0000","last_clean":"2024-09-10T15:52:08.013728+0000","last_became_active":"2024-09-10T15:52:30.151717+0000","last_became_peered":"2024-09-10T15:52:30.151717+0000","last_unstale":"2024-09-10T15:52:31.152685+0000","last_undegraded":"2024-09-10T15:52:30.144026+0000","last_fullsized":"2024-09-10T15:52:31.152685+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'2543","reported_seq":2643,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:59.141106+0000","last_change":"2024-09-10T15:52:53.666934+0000","last_active":"2024-09-10T15:52:59.141106+0000","last_peered":"2024-09-10T15:52:59.141106+0000","last_clean":"2024-09-10T15:52:59.141106+0000","last_became_active":"2024-09-10T15:52:31.157172+0000","last_became_peered":"2024-09-10T15:52:31.157172+0000","last_unstale":"2024-09-10T15:52:59.141106+0000","last_undegraded":"2024-09-10T15:52:59.141106+0000","last_fullsized":"2024-09-10T15:52:59.141106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":2543,"log_dups_size":0,"ondisk_log_size":2543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:39.074879+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10395648,"num_objects":2538,"num_object_clones":0,"num_object_copies":5076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":517,"num_write_kb":2068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":91,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.161559+0000","last_change":"2024-09-10T15:52:31.159524+0000","last_active":"2024-09-10T15:52:31.161559+0000","last_peered":"2024-09-10T15:52:31.161559+0000","last_clean":"2024-09-10T15:52:31.161559+0000","last_became_active":"2024-09-10T15:52:31.159205+0000","last_became_peered":"2024-09-10T15:52:31.159205+0000","last_unstale":"2024-09-10T15:52:31.161559+0000","last_undegraded":"2024-09-10T15:52:31.161559+0000","last_fullsized":"2024-09-10T15:52:31.161559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:04.415297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'2571","reported_seq":2672,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:59.148815+0000","last_change":"2024-09-10T15:52:54.903563+0000","last_active":"2024-09-10T15:52:59.148815+0000","last_peered":"2024-09-10T15:52:59.148815+0000","last_clean":"2024-09-10T15:52:59.148815+0000","last_became_active":"2024-09-10T15:52:31.157034+0000","last_became_peered":"2024-09-10T15:52:31.157034+0000","last_unstale":"2024-09-10T15:52:59.148815+0000","last_undegraded":"2024-09-10T15:52:59.148815+0000","last_fullsized":"2024-09-10T15:52:59.148815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":2571,"log_dups_size":0,"ondisk_log_size":2571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:09.119902+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506240,"num_objects":2565,"num_object_clones":0,"num_object_copies":5130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'2710","reported_seq":2775,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.011270+0000","last_change":"2024-09-10T15:53:00.778712+0000","last_active":"2024-09-10T15:53:01.011270+0000","last_peered":"2024-09-10T15:53:01.011270+0000","last_clean":"2024-09-10T15:53:01.011270+0000","last_became_active":"2024-09-10T15:52:30.151806+0000","last_became_peered":"2024-09-10T15:52:30.151806+0000","last_unstale":"2024-09-10T15:53:01.011270+0000","last_undegraded":"2024-09-10T15:53:01.011270+0000","last_fullsized":"2024-09-10T15:53:01.011270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2710,"log_dups_size":0,"ondisk_log_size":2710,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11092451,"num_objects":2709,"num_object_clones":0,"num_object_copies":5418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2709,"num_write_kb":10833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.149171+0000","last_change":"2024-09-10T15:52:30.154393+0000","last_active":"2024-09-10T15:52:31.149171+0000","last_peered":"2024-09-10T15:52:31.149171+0000","last_clean":"2024-09-10T15:52:31.149171+0000","last_became_active":"2024-09-10T15:52:30.154111+0000","last_became_peered":"2024-09-10T15:52:30.154111+0000","last_unstale":"2024-09-10T15:52:31.149171+0000","last_undegraded":"2024-09-10T15:52:31.149171+0000","last_fullsized":"2024-09-10T15:52:31.149171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:53.944111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'2803","reported_seq":2859,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:53:01.061345+0000","last_change":"2024-09-10T15:52:30.159885+0000","last_active":"2024-09-10T15:53:01.061345+0000","last_peered":"2024-09-10T15:53:01.061345+0000","last_clean":"2024-09-10T15:51:57.078241+0000","last_became_active":"2024-09-10T15:52:30.159635+0000","last_became_peered":"2024-09-10T15:52:30.159635+0000","last_unstale":"2024-09-10T15:53:01.061345+0000","last_undegraded":"2024-09-10T15:52:30.148722+0000","last_fullsized":"2024-09-10T15:53:01.061345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2803,"log_dups_size":0,"ondisk_log_size":2803,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11449769,"num_objects":2798,"num_object_clones":0,"num_object_copies":5596,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2800,"num_write_kb":11184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.157409+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:52:31.157409+0000","last_peered":"2024-09-10T15:52:31.157409+0000","last_clean":"2024-09-10T15:52:31.157409+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:52:31.157409+0000","last_undegraded":"2024-09-10T15:52:31.157409+0000","last_fullsized":"2024-09-10T15:52:31.157409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'2719","reported_seq":2775,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:53:01.045561+0000","last_change":"2024-09-10T15:52:30.159788+0000","last_active":"2024-09-10T15:53:01.045561+0000","last_peered":"2024-09-10T15:53:01.045561+0000","last_clean":"2024-09-10T15:51:45.893570+0000","last_became_active":"2024-09-10T15:52:30.159560+0000","last_became_peered":"2024-09-10T15:52:30.159560+0000","last_unstale":"2024-09-10T15:53:01.045561+0000","last_undegraded":"2024-09-10T15:52:30.148492+0000","last_fullsized":"2024-09-10T15:53:01.045561+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2719,"log_dups_size":0,"ondisk_log_size":2719,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11121606,"num_objects":2717,"num_object_clones":0,"num_object_copies":5434,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2717,"num_write_kb":10862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2717}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.152595+0000","last_change":"2024-09-10T15:52:30.149270+0000","last_active":"2024-09-10T15:52:31.152595+0000","last_peered":"2024-09-10T15:52:31.152595+0000","last_clean":"2024-09-10T15:52:31.152595+0000","last_became_active":"2024-09-10T15:52:30.148972+0000","last_became_peered":"2024-09-10T15:52:30.148972+0000","last_unstale":"2024-09-10T15:52:31.152595+0000","last_undegraded":"2024-09-10T15:52:31.152595+0000","last_fullsized":"2024-09-10T15:52:31.152595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:21.166160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.158991+0000","last_change":"2024-09-10T15:52:31.155379+0000","last_active":"2024-09-10T15:52:31.158991+0000","last_peered":"2024-09-10T15:52:31.158991+0000","last_clean":"2024-09-10T15:52:31.158991+0000","last_became_active":"2024-09-10T15:52:31.155102+0000","last_became_peered":"2024-09-10T15:52:31.155102+0000","last_unstale":"2024-09-10T15:52:31.158991+0000","last_undegraded":"2024-09-10T15:52:31.158991+0000","last_fullsized":"2024-09-10T15:52:31.158991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.976123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'2884","reported_seq":2957,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.067039+0000","last_change":"2024-09-10T15:53:00.912097+0000","last_active":"2024-09-10T15:53:01.067039+0000","last_peered":"2024-09-10T15:53:01.067039+0000","last_clean":"2024-09-10T15:53:01.067039+0000","last_became_active":"2024-09-10T15:52:30.152069+0000","last_became_peered":"2024-09-10T15:52:30.152069+0000","last_unstale":"2024-09-10T15:53:01.067039+0000","last_undegraded":"2024-09-10T15:53:01.067039+0000","last_fullsized":"2024-09-10T15:53:01.067039+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2884,"log_dups_size":0,"ondisk_log_size":2884,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11765644,"num_objects":2876,"num_object_clones":0,"num_object_copies":5752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2881,"num_write_kb":11495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":177210329,"num_objects":43281,"num_object_clones":0,"num_object_copies":86562,"num_objects_missing_on_primary":6,"num_objects_missing":6,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33750,"num_write_kb":134886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":352055296,"data_stored":351950519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43344,"ondisk_log_size":43344,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149714,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":274904,"kb_used_data":175240,"kb_used_omap":4,"kb_used_meta":99643,"kb_avail":94096936,"statfs":{"total":96636764160,"available":96355262464,"internally_reserved":0,"allocated":179445760,"data_stored":179205441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":102035117},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,6,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953483,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":277636,"kb_used_data":165492,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94094204,"statfs":{"total":96636764160,"available":96352464896,"internally_reserved":0,"allocated":169463808,"data_stored":169249244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114814634},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3,1,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":181227520,"data_stored":181223907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":170713088,"data_stored":170713088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:02.976 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:03.508 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:03.509 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:03.532 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-10T15:53:02.434083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":177210348,"num_objects":43283,"num_object_clones":0,"num_object_copies":86566,"num_objects_missing_on_primary":6,"num_objects_missing":6,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33752,"num_write_kb":134888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43347,"ondisk_log_size":43347,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":552540,"kb_used_data":340732,"kb_used_omap":8,"kb_used_meta":211767,"kb_avail":188191140,"statfs":{"total":193273528320,"available":192707727360,"internally_reserved":0,"allocated":348909568,"data_stored":348454685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":216849751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3,1,10,9,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":84909829,"num_objects":20722,"num_object_clones":0,"num_object_copies":41444,"num_objects_missing_on_primary":-18,"num_objects_missing":-18,"num_objects_degraded":-37,"num_objects_misplaced":0,"num_objects_unfound":-24,"num_objects_dirty":20722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11164,"num_write_kb":44716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002889"},"pg_stats":[{"pgid":"2.f","version":"37'2851","reported_seq":2902,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.040948+0000","last_change":"2024-09-10T15:52:30.154389+0000","last_active":"2024-09-10T15:53:01.040948+0000","last_peered":"2024-09-10T15:53:01.040948+0000","last_clean":"2024-09-10T15:53:01.040948+0000","last_became_active":"2024-09-10T15:52:30.154140+0000","last_became_peered":"2024-09-10T15:52:30.154140+0000","last_unstale":"2024-09-10T15:53:01.040948+0000","last_undegraded":"2024-09-10T15:53:01.040948+0000","last_fullsized":"2024-09-10T15:53:01.040948+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2851,"log_dups_size":0,"ondisk_log_size":2851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:30.595214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11677696,"num_objects":2851,"num_object_clones":0,"num_object_copies":5702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2851,"num_write_kb":11404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'2659","reported_seq":2746,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:59.155003+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:52:59.155003+0000","last_peered":"2024-09-10T15:52:59.155003+0000","last_clean":"2024-09-10T15:52:59.155003+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:52:59.155003+0000","last_undegraded":"2024-09-10T15:52:59.155003+0000","last_fullsized":"2024-09-10T15:52:59.155003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":2659,"log_dups_size":0,"ondisk_log_size":2659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10866688,"num_objects":2653,"num_object_clones":0,"num_object_copies":5306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":995,"num_write_kb":3980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'2572","reported_seq":2670,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:59.164350+0000","last_change":"2024-09-10T15:52:50.491262+0000","last_active":"2024-09-10T15:52:59.164350+0000","last_peered":"2024-09-10T15:52:59.164350+0000","last_clean":"2024-09-10T15:52:59.164350+0000","last_became_active":"2024-09-10T15:52:31.157749+0000","last_became_peered":"2024-09-10T15:52:31.157749+0000","last_unstale":"2024-09-10T15:52:59.164350+0000","last_undegraded":"2024-09-10T15:52:59.164350+0000","last_fullsized":"2024-09-10T15:52:59.164350+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":2572,"log_dups_size":0,"ondisk_log_size":2572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:11.778645+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10518528,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":840,"num_write_kb":3360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'2623","reported_seq":2683,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:59.159418+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:52:59.159418+0000","last_peered":"2024-09-10T15:52:59.159418+0000","last_clean":"2024-09-10T15:52:59.159418+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:52:59.159418+0000","last_undegraded":"2024-09-10T15:52:59.159418+0000","last_fullsized":"2024-09-10T15:52:59.159418+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2623,"log_dups_size":0,"ondisk_log_size":2623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10743808,"num_objects":2623,"num_object_clones":0,"num_object_copies":5246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2623,"num_write_kb":10492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'2708","reported_seq":2778,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.037006+0000","last_change":"2024-09-10T15:53:00.817069+0000","last_active":"2024-09-10T15:53:01.037006+0000","last_peered":"2024-09-10T15:53:01.037006+0000","last_clean":"2024-09-10T15:53:01.037006+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:01.037006+0000","last_undegraded":"2024-09-10T15:53:01.037006+0000","last_fullsized":"2024-09-10T15:53:01.037006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2708,"log_dups_size":0,"ondisk_log_size":2708,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11064262,"num_objects":2703,"num_object_clones":0,"num_object_copies":5406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2706,"num_write_kb":10808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'2728","reported_seq":2796,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.043527+0000","last_change":"2024-09-10T15:53:01.031975+0000","last_active":"2024-09-10T15:53:01.043527+0000","last_peered":"2024-09-10T15:53:01.043527+0000","last_clean":"2024-09-10T15:53:01.043527+0000","last_became_active":"2024-09-10T15:52:30.152332+0000","last_became_peered":"2024-09-10T15:52:30.152332+0000","last_unstale":"2024-09-10T15:53:01.043527+0000","last_undegraded":"2024-09-10T15:53:01.043527+0000","last_fullsized":"2024-09-10T15:53:01.043527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2728,"log_dups_size":0,"ondisk_log_size":2728,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11154374,"num_objects":2725,"num_object_clones":0,"num_object_copies":5450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2726,"num_write_kb":10895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'2756","reported_seq":2817,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-10T15:53:01.059087+0000","last_change":"2024-09-10T15:53:01.045435+0000","last_active":"2024-09-10T15:53:01.059087+0000","last_peered":"2024-09-10T15:53:01.059087+0000","last_clean":"2024-09-10T15:51:56.469065+0000","last_became_active":"2024-09-10T15:52:30.156955+0000","last_became_peered":"2024-09-10T15:52:30.156955+0000","last_unstale":"2024-09-10T15:53:01.059087+0000","last_undegraded":"2024-09-10T15:53:01.059087+0000","last_fullsized":"2024-09-10T15:53:01.059087+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2756,"log_dups_size":0,"ondisk_log_size":2756,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11264966,"num_objects":2752,"num_object_clones":0,"num_object_copies":5504,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2754,"num_write_kb":11004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2752}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'2779","reported_seq":2848,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.015193+0000","last_change":"2024-09-10T15:53:00.993088+0000","last_active":"2024-09-10T15:53:01.015193+0000","last_peered":"2024-09-10T15:53:01.015193+0000","last_clean":"2024-09-10T15:53:01.015193+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:01.015193+0000","last_undegraded":"2024-09-10T15:53:01.015193+0000","last_fullsized":"2024-09-10T15:53:01.015193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2779,"log_dups_size":0,"ondisk_log_size":2779,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11375075,"num_objects":2778,"num_object_clones":0,"num_object_copies":5556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2778,"num_write_kb":11109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.151621+0000","last_change":"2024-09-10T15:52:30.152637+0000","last_active":"2024-09-10T15:52:31.151621+0000","last_peered":"2024-09-10T15:52:31.151621+0000","last_clean":"2024-09-10T15:52:31.151621+0000","last_became_active":"2024-09-10T15:52:30.151730+0000","last_became_peered":"2024-09-10T15:52:30.151730+0000","last_unstale":"2024-09-10T15:52:31.151621+0000","last_undegraded":"2024-09-10T15:52:31.151621+0000","last_fullsized":"2024-09-10T15:52:31.151621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:09.577308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'2702","reported_seq":2806,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:59.159466+0000","last_change":"2024-09-10T15:52:52.645800+0000","last_active":"2024-09-10T15:52:59.159466+0000","last_peered":"2024-09-10T15:52:59.159466+0000","last_clean":"2024-09-10T15:52:59.159466+0000","last_became_active":"2024-09-10T15:52:31.157754+0000","last_became_peered":"2024-09-10T15:52:31.157754+0000","last_unstale":"2024-09-10T15:52:59.159466+0000","last_undegraded":"2024-09-10T15:52:59.159466+0000","last_fullsized":"2024-09-10T15:52:59.159466+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":2702,"log_dups_size":0,"ondisk_log_size":2702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:37.611240+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11038720,"num_objects":2695,"num_object_clones":0,"num_object_copies":5390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":679,"num_write_kb":2716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.160328+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:52:31.160328+0000","last_peered":"2024-09-10T15:52:31.160328+0000","last_clean":"2024-09-10T15:52:31.160328+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:52:31.160328+0000","last_undegraded":"2024-09-10T15:52:31.160328+0000","last_fullsized":"2024-09-10T15:52:31.160328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'2736","reported_seq":2797,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:00.998389+0000","last_change":"2024-09-10T15:53:00.964850+0000","last_active":"2024-09-10T15:53:00.998389+0000","last_peered":"2024-09-10T15:53:00.998389+0000","last_clean":"2024-09-10T15:53:00.998389+0000","last_became_active":"2024-09-10T15:52:30.152112+0000","last_became_peered":"2024-09-10T15:52:30.152112+0000","last_unstale":"2024-09-10T15:53:00.998389+0000","last_undegraded":"2024-09-10T15:53:00.998389+0000","last_fullsized":"2024-09-10T15:53:00.998389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2736,"log_dups_size":0,"ondisk_log_size":2736,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11174854,"num_objects":2730,"num_object_clones":0,"num_object_copies":5460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2734,"num_write_kb":10916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:52:31.152685+0000","last_change":"2024-09-10T15:52:30.152323+0000","last_active":"2024-09-10T15:52:31.152685+0000","last_peered":"2024-09-10T15:52:31.152685+0000","last_clean":"2024-09-10T15:52:08.013728+0000","last_became_active":"2024-09-10T15:52:30.151717+0000","last_became_peered":"2024-09-10T15:52:30.151717+0000","last_unstale":"2024-09-10T15:52:31.152685+0000","last_undegraded":"2024-09-10T15:52:30.144026+0000","last_fullsized":"2024-09-10T15:52:31.152685+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'2543","reported_seq":2643,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:59.141106+0000","last_change":"2024-09-10T15:52:53.666934+0000","last_active":"2024-09-10T15:52:59.141106+0000","last_peered":"2024-09-10T15:52:59.141106+0000","last_clean":"2024-09-10T15:52:59.141106+0000","last_became_active":"2024-09-10T15:52:31.157172+0000","last_became_peered":"2024-09-10T15:52:31.157172+0000","last_unstale":"2024-09-10T15:52:59.141106+0000","last_undegraded":"2024-09-10T15:52:59.141106+0000","last_fullsized":"2024-09-10T15:52:59.141106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":2543,"log_dups_size":0,"ondisk_log_size":2543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:39.074879+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10395648,"num_objects":2538,"num_object_clones":0,"num_object_copies":5076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":517,"num_write_kb":2068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":91,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.161559+0000","last_change":"2024-09-10T15:52:31.159524+0000","last_active":"2024-09-10T15:52:31.161559+0000","last_peered":"2024-09-10T15:52:31.161559+0000","last_clean":"2024-09-10T15:52:31.161559+0000","last_became_active":"2024-09-10T15:52:31.159205+0000","last_became_peered":"2024-09-10T15:52:31.159205+0000","last_unstale":"2024-09-10T15:52:31.161559+0000","last_undegraded":"2024-09-10T15:52:31.161559+0000","last_fullsized":"2024-09-10T15:52:31.161559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:04.415297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'2571","reported_seq":2672,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:59.148815+0000","last_change":"2024-09-10T15:52:54.903563+0000","last_active":"2024-09-10T15:52:59.148815+0000","last_peered":"2024-09-10T15:52:59.148815+0000","last_clean":"2024-09-10T15:52:59.148815+0000","last_became_active":"2024-09-10T15:52:31.157034+0000","last_became_peered":"2024-09-10T15:52:31.157034+0000","last_unstale":"2024-09-10T15:52:59.148815+0000","last_undegraded":"2024-09-10T15:52:59.148815+0000","last_fullsized":"2024-09-10T15:52:59.148815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":2571,"log_dups_size":0,"ondisk_log_size":2571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:09.119902+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506240,"num_objects":2565,"num_object_clones":0,"num_object_copies":5130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":440,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'2710","reported_seq":2775,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.011270+0000","last_change":"2024-09-10T15:53:00.778712+0000","last_active":"2024-09-10T15:53:01.011270+0000","last_peered":"2024-09-10T15:53:01.011270+0000","last_clean":"2024-09-10T15:53:01.011270+0000","last_became_active":"2024-09-10T15:52:30.151806+0000","last_became_peered":"2024-09-10T15:52:30.151806+0000","last_unstale":"2024-09-10T15:53:01.011270+0000","last_undegraded":"2024-09-10T15:53:01.011270+0000","last_fullsized":"2024-09-10T15:53:01.011270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2710,"log_dups_size":0,"ondisk_log_size":2710,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11092451,"num_objects":2709,"num_object_clones":0,"num_object_copies":5418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2709,"num_write_kb":10833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.149171+0000","last_change":"2024-09-10T15:52:30.154393+0000","last_active":"2024-09-10T15:52:31.149171+0000","last_peered":"2024-09-10T15:52:31.149171+0000","last_clean":"2024-09-10T15:52:31.149171+0000","last_became_active":"2024-09-10T15:52:30.154111+0000","last_became_peered":"2024-09-10T15:52:30.154111+0000","last_unstale":"2024-09-10T15:52:31.149171+0000","last_undegraded":"2024-09-10T15:52:31.149171+0000","last_fullsized":"2024-09-10T15:52:31.149171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:53.944111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'2803","reported_seq":2859,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:53:01.061345+0000","last_change":"2024-09-10T15:52:30.159885+0000","last_active":"2024-09-10T15:53:01.061345+0000","last_peered":"2024-09-10T15:53:01.061345+0000","last_clean":"2024-09-10T15:51:57.078241+0000","last_became_active":"2024-09-10T15:52:30.159635+0000","last_became_peered":"2024-09-10T15:52:30.159635+0000","last_unstale":"2024-09-10T15:53:01.061345+0000","last_undegraded":"2024-09-10T15:52:30.148722+0000","last_fullsized":"2024-09-10T15:53:01.061345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2803,"log_dups_size":0,"ondisk_log_size":2803,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11449769,"num_objects":2798,"num_object_clones":0,"num_object_copies":5596,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2800,"num_write_kb":11184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.157409+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:52:31.157409+0000","last_peered":"2024-09-10T15:52:31.157409+0000","last_clean":"2024-09-10T15:52:31.157409+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:52:31.157409+0000","last_undegraded":"2024-09-10T15:52:31.157409+0000","last_fullsized":"2024-09-10T15:52:31.157409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'2719","reported_seq":2775,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-10T15:53:01.045561+0000","last_change":"2024-09-10T15:52:30.159788+0000","last_active":"2024-09-10T15:53:01.045561+0000","last_peered":"2024-09-10T15:53:01.045561+0000","last_clean":"2024-09-10T15:51:45.893570+0000","last_became_active":"2024-09-10T15:52:30.159560+0000","last_became_peered":"2024-09-10T15:52:30.159560+0000","last_unstale":"2024-09-10T15:53:01.045561+0000","last_undegraded":"2024-09-10T15:52:30.148492+0000","last_fullsized":"2024-09-10T15:53:01.045561+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2719,"log_dups_size":0,"ondisk_log_size":2719,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11121606,"num_objects":2717,"num_object_clones":0,"num_object_copies":5434,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2717,"num_write_kb":10862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2717}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.152595+0000","last_change":"2024-09-10T15:52:30.149270+0000","last_active":"2024-09-10T15:52:31.152595+0000","last_peered":"2024-09-10T15:52:31.152595+0000","last_clean":"2024-09-10T15:52:31.152595+0000","last_became_active":"2024-09-10T15:52:30.148972+0000","last_became_peered":"2024-09-10T15:52:30.148972+0000","last_unstale":"2024-09-10T15:52:31.152595+0000","last_undegraded":"2024-09-10T15:52:31.152595+0000","last_fullsized":"2024-09-10T15:52:31.152595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:21.166160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.158991+0000","last_change":"2024-09-10T15:52:31.155379+0000","last_active":"2024-09-10T15:52:31.158991+0000","last_peered":"2024-09-10T15:52:31.158991+0000","last_clean":"2024-09-10T15:52:31.158991+0000","last_became_active":"2024-09-10T15:52:31.155102+0000","last_became_peered":"2024-09-10T15:52:31.155102+0000","last_unstale":"2024-09-10T15:52:31.158991+0000","last_undegraded":"2024-09-10T15:52:31.158991+0000","last_fullsized":"2024-09-10T15:52:31.158991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.976123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'2884","reported_seq":2957,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.067039+0000","last_change":"2024-09-10T15:53:00.912097+0000","last_active":"2024-09-10T15:53:01.067039+0000","last_peered":"2024-09-10T15:53:01.067039+0000","last_clean":"2024-09-10T15:53:01.067039+0000","last_became_active":"2024-09-10T15:52:30.152069+0000","last_became_peered":"2024-09-10T15:52:30.152069+0000","last_unstale":"2024-09-10T15:53:01.067039+0000","last_undegraded":"2024-09-10T15:53:01.067039+0000","last_fullsized":"2024-09-10T15:53:01.067039+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":2884,"log_dups_size":0,"ondisk_log_size":2884,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11765644,"num_objects":2876,"num_object_clones":0,"num_object_copies":5752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2881,"num_write_kb":11495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":177210329,"num_objects":43281,"num_object_clones":0,"num_object_copies":86562,"num_objects_missing_on_primary":6,"num_objects_missing":6,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33750,"num_write_kb":134886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":352055296,"data_stored":351950519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43344,"ondisk_log_size":43344,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149714,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":274904,"kb_used_data":175240,"kb_used_omap":4,"kb_used_meta":99643,"kb_avail":94096936,"statfs":{"total":96636764160,"available":96355262464,"internally_reserved":0,"allocated":179445760,"data_stored":179205441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":102035117},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,6,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953483,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":277636,"kb_used_data":165492,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94094204,"statfs":{"total":96636764160,"available":96352464896,"internally_reserved":0,"allocated":169463808,"data_stored":169249244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114814634},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3,1,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":181227520,"data_stored":181223907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":170713088,"data_stored":170713088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:03.800 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 26 20 48188 48168 7.23562 4.57031 0.0118121 0.010751 2024-09-10T15:53:04.800 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 27 20 49506 49486 7.15831 5.14844 0.00727108 0.0109076 2024-09-10T15:53:05.800 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 28 20 50957 50937 7.10507 5.66797 0.0179298 0.0109898 2024-09-10T15:53:06.535 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:06.801 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 29 20 52366 52346 7.04985 5.50391 0.0222924 0.0110771 2024-09-10T15:53:07.076 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:07.077 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:07.098 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-09-10T15:53:06.434860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":206300372,"num_objects":50378,"num_object_clones":0,"num_object_copies":100756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28809,"num_write_kb":115184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50450,"ondisk_log_size":50450,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":607828,"kb_used_data":396020,"kb_used_omap":8,"kb_used_meta":211767,"kb_avail":188135852,"statfs":{"total":193273528320,"available":192651112448,"internally_reserved":0,"allocated":405524480,"data_stored":405079972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":216849751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,6,7,6,10,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":78843170,"num_objects":19240,"num_object_clones":0,"num_object_copies":38480,"num_objects_missing_on_primary":-18,"num_objects_missing":-18,"num_objects_degraded":-36,"num_objects_misplaced":0,"num_objects_unfound":-11,"num_objects_dirty":19240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5075,"num_write_kb":20381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002518"},"pg_stats":[{"pgid":"2.f","version":"37'3273","reported_seq":3324,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:06.097448+0000","last_change":"2024-09-10T15:52:30.154389+0000","last_active":"2024-09-10T15:53:06.097448+0000","last_peered":"2024-09-10T15:53:06.097448+0000","last_clean":"2024-09-10T15:53:06.097448+0000","last_became_active":"2024-09-10T15:52:30.154140+0000","last_became_peered":"2024-09-10T15:52:30.154140+0000","last_unstale":"2024-09-10T15:53:06.097448+0000","last_undegraded":"2024-09-10T15:53:06.097448+0000","last_fullsized":"2024-09-10T15:53:06.097448+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3273,"log_dups_size":0,"ondisk_log_size":3273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:30.595214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13406208,"num_objects":3273,"num_object_clones":0,"num_object_copies":6546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3273,"num_write_kb":13092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'3108","reported_seq":3196,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:04.125124+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:04.125124+0000","last_peered":"2024-09-10T15:53:04.125124+0000","last_clean":"2024-09-10T15:53:04.125124+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:04.125124+0000","last_undegraded":"2024-09-10T15:53:04.125124+0000","last_fullsized":"2024-09-10T15:53:04.125124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":3108,"log_dups_size":0,"ondisk_log_size":3108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12705792,"num_objects":3102,"num_object_clones":0,"num_object_copies":6204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1444,"num_write_kb":5776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'2985","reported_seq":3082,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:04.164482+0000","last_change":"2024-09-10T15:52:50.491262+0000","last_active":"2024-09-10T15:53:04.164482+0000","last_peered":"2024-09-10T15:53:04.164482+0000","last_clean":"2024-09-10T15:53:04.164482+0000","last_became_active":"2024-09-10T15:52:31.157749+0000","last_became_peered":"2024-09-10T15:52:31.157749+0000","last_unstale":"2024-09-10T15:53:04.164482+0000","last_undegraded":"2024-09-10T15:53:04.164482+0000","last_fullsized":"2024-09-10T15:53:04.164482+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":2985,"log_dups_size":0,"ondisk_log_size":2985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:11.778645+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12210176,"num_objects":2981,"num_object_clones":0,"num_object_copies":5962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1253,"num_write_kb":5012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'3050","reported_seq":3110,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:04.176488+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:04.176488+0000","last_peered":"2024-09-10T15:53:04.176488+0000","last_clean":"2024-09-10T15:53:04.176488+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:04.176488+0000","last_undegraded":"2024-09-10T15:53:04.176488+0000","last_fullsized":"2024-09-10T15:53:04.176488+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3050,"log_dups_size":0,"ondisk_log_size":3050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12492800,"num_objects":3050,"num_object_clones":0,"num_object_copies":6100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3050,"num_write_kb":12200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'3167","reported_seq":3272,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:06.074084+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:06.074084+0000","last_peered":"2024-09-10T15:53:06.074084+0000","last_clean":"2024-09-10T15:53:06.074084+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:06.074084+0000","last_undegraded":"2024-09-10T15:53:06.074084+0000","last_fullsized":"2024-09-10T15:53:06.074084+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":3167,"log_dups_size":0,"ondisk_log_size":3167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12943360,"num_objects":3160,"num_object_clones":0,"num_object_copies":6320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":238,"num_write_kb":952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'3186","reported_seq":3254,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:06.078278+0000","last_change":"2024-09-10T15:53:01.031975+0000","last_active":"2024-09-10T15:53:06.078278+0000","last_peered":"2024-09-10T15:53:06.078278+0000","last_clean":"2024-09-10T15:53:06.078278+0000","last_became_active":"2024-09-10T15:52:30.152332+0000","last_became_peered":"2024-09-10T15:52:30.152332+0000","last_unstale":"2024-09-10T15:53:06.078278+0000","last_undegraded":"2024-09-10T15:53:06.078278+0000","last_fullsized":"2024-09-10T15:53:06.078278+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3186,"log_dups_size":0,"ondisk_log_size":3186,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13030342,"num_objects":3183,"num_object_clones":0,"num_object_copies":6366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3184,"num_write_kb":12727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'3158","reported_seq":3225,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:06.080528+0000","last_change":"2024-09-10T15:53:01.078257+0000","last_active":"2024-09-10T15:53:06.080528+0000","last_peered":"2024-09-10T15:53:06.080528+0000","last_clean":"2024-09-10T15:53:06.080528+0000","last_became_active":"2024-09-10T15:52:30.156955+0000","last_became_peered":"2024-09-10T15:52:30.156955+0000","last_unstale":"2024-09-10T15:53:06.080528+0000","last_undegraded":"2024-09-10T15:53:06.080528+0000","last_fullsized":"2024-09-10T15:53:06.080528+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3158,"log_dups_size":0,"ondisk_log_size":3158,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12911558,"num_objects":3154,"num_object_clones":0,"num_object_copies":6308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3156,"num_write_kb":12612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'3237","reported_seq":3323,"reported_epoch":37,"state":"active+clean+scrubbing","last_fresh":"2024-09-10T15:53:06.081223+0000","last_change":"2024-09-10T15:53:05.595032+0000","last_active":"2024-09-10T15:53:06.081223+0000","last_peered":"2024-09-10T15:53:06.081223+0000","last_clean":"2024-09-10T15:53:06.081223+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:06.081223+0000","last_undegraded":"2024-09-10T15:53:06.081223+0000","last_fullsized":"2024-09-10T15:53:06.081223+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":1224,"log_size":3237,"log_dups_size":0,"ondisk_log_size":3237,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13251043,"num_objects":3236,"num_object_clones":0,"num_object_copies":6472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3236,"num_write_kb":12941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.151621+0000","last_change":"2024-09-10T15:52:30.152637+0000","last_active":"2024-09-10T15:52:31.151621+0000","last_peered":"2024-09-10T15:52:31.151621+0000","last_clean":"2024-09-10T15:52:31.151621+0000","last_became_active":"2024-09-10T15:52:30.151730+0000","last_became_peered":"2024-09-10T15:52:30.151730+0000","last_unstale":"2024-09-10T15:52:31.151621+0000","last_undegraded":"2024-09-10T15:52:31.151621+0000","last_fullsized":"2024-09-10T15:52:31.151621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:09.577308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'3143","reported_seq":3247,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:04.149475+0000","last_change":"2024-09-10T15:52:52.645800+0000","last_active":"2024-09-10T15:53:04.149475+0000","last_peered":"2024-09-10T15:53:04.149475+0000","last_clean":"2024-09-10T15:53:04.149475+0000","last_became_active":"2024-09-10T15:52:31.157754+0000","last_became_peered":"2024-09-10T15:52:31.157754+0000","last_unstale":"2024-09-10T15:53:04.149475+0000","last_undegraded":"2024-09-10T15:53:04.149475+0000","last_fullsized":"2024-09-10T15:53:04.149475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":3143,"log_dups_size":0,"ondisk_log_size":3143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:37.611240+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12845056,"num_objects":3136,"num_object_clones":0,"num_object_copies":6272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1120,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.160328+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:52:31.160328+0000","last_peered":"2024-09-10T15:52:31.160328+0000","last_clean":"2024-09-10T15:52:31.160328+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:52:31.160328+0000","last_undegraded":"2024-09-10T15:52:31.160328+0000","last_fullsized":"2024-09-10T15:52:31.160328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'3215","reported_seq":3315,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:06.055012+0000","last_change":"2024-09-10T15:53:05.799313+0000","last_active":"2024-09-10T15:53:06.055012+0000","last_peered":"2024-09-10T15:53:06.055012+0000","last_clean":"2024-09-10T15:53:06.055012+0000","last_became_active":"2024-09-10T15:52:30.152112+0000","last_became_peered":"2024-09-10T15:52:30.152112+0000","last_unstale":"2024-09-10T15:53:06.055012+0000","last_undegraded":"2024-09-10T15:53:06.055012+0000","last_fullsized":"2024-09-10T15:53:06.055012+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":3215,"log_dups_size":0,"ondisk_log_size":3215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:09.853665+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13135872,"num_objects":3207,"num_object_clones":0,"num_object_copies":6414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":71,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:01.234273+0000","last_change":"2024-09-10T15:53:01.234182+0000","last_active":"2024-09-10T15:53:01.234273+0000","last_peered":"2024-09-10T15:53:01.234273+0000","last_clean":"2024-09-10T15:53:01.234273+0000","last_became_active":"2024-09-10T15:52:30.151717+0000","last_became_peered":"2024-09-10T15:52:30.151717+0000","last_unstale":"2024-09-10T15:53:01.234273+0000","last_undegraded":"2024-09-10T15:53:01.234273+0000","last_fullsized":"2024-09-10T15:53:01.234273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:44.204957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'2999","reported_seq":3099,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:04.160058+0000","last_change":"2024-09-10T15:52:53.666934+0000","last_active":"2024-09-10T15:53:04.160058+0000","last_peered":"2024-09-10T15:53:04.160058+0000","last_clean":"2024-09-10T15:53:04.160058+0000","last_became_active":"2024-09-10T15:52:31.157172+0000","last_became_peered":"2024-09-10T15:52:31.157172+0000","last_unstale":"2024-09-10T15:53:04.160058+0000","last_undegraded":"2024-09-10T15:53:04.160058+0000","last_fullsized":"2024-09-10T15:53:04.160058+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":2999,"log_dups_size":0,"ondisk_log_size":2999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:39.074879+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12263424,"num_objects":2994,"num_object_clones":0,"num_object_copies":5988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":973,"num_write_kb":3892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":91,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.161559+0000","last_change":"2024-09-10T15:52:31.159524+0000","last_active":"2024-09-10T15:52:31.161559+0000","last_peered":"2024-09-10T15:52:31.161559+0000","last_clean":"2024-09-10T15:52:31.161559+0000","last_became_active":"2024-09-10T15:52:31.159205+0000","last_became_peered":"2024-09-10T15:52:31.159205+0000","last_unstale":"2024-09-10T15:52:31.161559+0000","last_undegraded":"2024-09-10T15:52:31.161559+0000","last_fullsized":"2024-09-10T15:52:31.161559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:04.415297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'2997","reported_seq":3098,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:04.162137+0000","last_change":"2024-09-10T15:52:54.903563+0000","last_active":"2024-09-10T15:53:04.162137+0000","last_peered":"2024-09-10T15:53:04.162137+0000","last_clean":"2024-09-10T15:53:04.162137+0000","last_became_active":"2024-09-10T15:52:31.157034+0000","last_became_peered":"2024-09-10T15:52:31.157034+0000","last_unstale":"2024-09-10T15:53:04.162137+0000","last_undegraded":"2024-09-10T15:53:04.162137+0000","last_fullsized":"2024-09-10T15:53:04.162137+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":2997,"log_dups_size":0,"ondisk_log_size":2997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:09.119902+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12251136,"num_objects":2991,"num_object_clones":0,"num_object_copies":5982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":866,"num_write_kb":3464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'3142","reported_seq":3244,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:06.100814+0000","last_change":"2024-09-10T15:53:02.560438+0000","last_active":"2024-09-10T15:53:06.100814+0000","last_peered":"2024-09-10T15:53:06.100814+0000","last_clean":"2024-09-10T15:53:06.100814+0000","last_became_active":"2024-09-10T15:52:30.151806+0000","last_became_peered":"2024-09-10T15:52:30.151806+0000","last_unstale":"2024-09-10T15:53:06.100814+0000","last_undegraded":"2024-09-10T15:53:06.100814+0000","last_fullsized":"2024-09-10T15:53:06.100814+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":3142,"log_dups_size":0,"ondisk_log_size":3142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:45.403580+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12861440,"num_objects":3140,"num_object_clones":0,"num_object_copies":6280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":334,"num_write_kb":1336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.149171+0000","last_change":"2024-09-10T15:52:30.154393+0000","last_active":"2024-09-10T15:52:31.149171+0000","last_peered":"2024-09-10T15:52:31.149171+0000","last_clean":"2024-09-10T15:52:31.149171+0000","last_became_active":"2024-09-10T15:52:30.154111+0000","last_became_peered":"2024-09-10T15:52:30.154111+0000","last_unstale":"2024-09-10T15:52:31.149171+0000","last_undegraded":"2024-09-10T15:52:31.149171+0000","last_fullsized":"2024-09-10T15:52:31.149171+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:53.944111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'3248","reported_seq":3318,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:06.093559+0000","last_change":"2024-09-10T15:53:01.216723+0000","last_active":"2024-09-10T15:53:06.093559+0000","last_peered":"2024-09-10T15:53:06.093559+0000","last_clean":"2024-09-10T15:53:06.093559+0000","last_became_active":"2024-09-10T15:52:30.159635+0000","last_became_peered":"2024-09-10T15:52:30.159635+0000","last_unstale":"2024-09-10T15:53:06.093559+0000","last_undegraded":"2024-09-10T15:53:06.093559+0000","last_fullsized":"2024-09-10T15:53:06.093559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3248,"log_dups_size":0,"ondisk_log_size":3248,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13272489,"num_objects":3243,"num_object_clones":0,"num_object_copies":6486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3245,"num_write_kb":12964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.157409+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:52:31.157409+0000","last_peered":"2024-09-10T15:52:31.157409+0000","last_clean":"2024-09-10T15:52:31.157409+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:52:31.157409+0000","last_undegraded":"2024-09-10T15:52:31.157409+0000","last_fullsized":"2024-09-10T15:52:31.157409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'3192","reported_seq":3259,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:06.066668+0000","last_change":"2024-09-10T15:53:01.159721+0000","last_active":"2024-09-10T15:53:06.066668+0000","last_peered":"2024-09-10T15:53:06.066668+0000","last_clean":"2024-09-10T15:53:06.066668+0000","last_became_active":"2024-09-10T15:52:30.159560+0000","last_became_peered":"2024-09-10T15:52:30.159560+0000","last_unstale":"2024-09-10T15:53:06.066668+0000","last_undegraded":"2024-09-10T15:53:06.066668+0000","last_fullsized":"2024-09-10T15:53:06.066668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3192,"log_dups_size":0,"ondisk_log_size":3192,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13059014,"num_objects":3190,"num_object_clones":0,"num_object_copies":6380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3190,"num_write_kb":12754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.152595+0000","last_change":"2024-09-10T15:52:30.149270+0000","last_active":"2024-09-10T15:52:31.152595+0000","last_peered":"2024-09-10T15:52:31.152595+0000","last_clean":"2024-09-10T15:52:31.152595+0000","last_became_active":"2024-09-10T15:52:30.148972+0000","last_became_peered":"2024-09-10T15:52:30.148972+0000","last_unstale":"2024-09-10T15:52:31.152595+0000","last_undegraded":"2024-09-10T15:52:31.152595+0000","last_fullsized":"2024-09-10T15:52:31.152595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:21.166160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:52:31.158991+0000","last_change":"2024-09-10T15:52:31.155379+0000","last_active":"2024-09-10T15:52:31.158991+0000","last_peered":"2024-09-10T15:52:31.158991+0000","last_clean":"2024-09-10T15:52:31.158991+0000","last_became_active":"2024-09-10T15:52:31.155102+0000","last_became_peered":"2024-09-10T15:52:31.155102+0000","last_unstale":"2024-09-10T15:52:31.158991+0000","last_undegraded":"2024-09-10T15:52:31.158991+0000","last_fullsized":"2024-09-10T15:52:31.158991+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:04.976123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'3347","reported_seq":3460,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T15:53:06.087703+0000","last_change":"2024-09-10T15:53:04.965563+0000","last_active":"2024-09-10T15:53:06.087703+0000","last_peered":"2024-09-10T15:53:06.087703+0000","last_clean":"2024-09-10T15:53:06.087703+0000","last_became_active":"2024-09-10T15:52:30.152069+0000","last_became_peered":"2024-09-10T15:52:30.152069+0000","last_unstale":"2024-09-10T15:53:06.087703+0000","last_undegraded":"2024-09-10T15:53:06.087703+0000","last_fullsized":"2024-09-10T15:53:06.087703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":3347,"log_dups_size":0,"ondisk_log_size":3347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:03.697093+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13660643,"num_objects":3336,"num_object_clones":0,"num_object_copies":6672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":206300353,"num_objects":50376,"num_object_clones":0,"num_object_copies":100752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28807,"num_write_kb":115182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409714688,"data_stored":409606298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50447,"ondisk_log_size":50447,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149715,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302640,"kb_used_data":202976,"kb_used_omap":4,"kb_used_meta":99643,"kb_avail":94069200,"statfs":{"total":96636764160,"available":96326860800,"internally_reserved":0,"allocated":207847424,"data_stored":207625170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":102035117},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,5,4,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953484,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305188,"kb_used_data":193044,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94066652,"statfs":{"total":96636764160,"available":96324251648,"internally_reserved":0,"allocated":197677056,"data_stored":197454802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114814634},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,2,2,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":210583552,"data_stored":210579939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":199016448,"data_stored":199012835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:07.100 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-10T15:53:07.100 INFO:tasks.util.rados:rados -p unfounddel_pool get new_1 - 2024-09-10T15:53:07.100 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_1 - 2024-09-10T15:53:07.196 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/new_1: (2) No such file or directory 2024-09-10T15:53:07.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:07.206 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_1 - 2024-09-10T15:53:07.207 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_1 - 2024-09-10T15:53:07.306 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existed_1: (2) No such file or directory 2024-09-10T15:53:07.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:07.326 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_1 - 2024-09-10T15:53:07.326 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_1 - 2024-09-10T15:53:07.433 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existing_1: (2) No such file or directory 2024-09-10T15:53:07.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:07.449 INFO:tasks.util.rados:rados -p unfounddel_pool get new_2 - 2024-09-10T15:53:07.449 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_2 - 2024-09-10T15:53:07.546 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/new_2: (2) No such file or directory 2024-09-10T15:53:07.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:07.561 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_2 - 2024-09-10T15:53:07.561 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_2 - 2024-09-10T15:53:07.664 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existed_2: (2) No such file or directory 2024-09-10T15:53:07.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:07.680 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_2 - 2024-09-10T15:53:07.681 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_2 - 2024-09-10T15:53:07.781 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existing_2: (2) No such file or directory 2024-09-10T15:53:07.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:07.793 INFO:tasks.util.rados:rados -p unfounddel_pool get new_3 - 2024-09-10T15:53:07.793 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_3 - 2024-09-10T15:53:07.801 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 30 20 53628 53608 6.97917 4.92969 0.0152092 0.0111874 2024-09-10T15:53:07.878 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/new_3: (2) No such file or directory 2024-09-10T15:53:07.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:07.885 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_3 - 2024-09-10T15:53:07.885 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_3 - 2024-09-10T15:53:08.015 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existed_3: (2) No such file or directory 2024-09-10T15:53:08.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:08.022 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_3 - 2024-09-10T15:53:08.022 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_3 - 2024-09-10T15:53:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existing_3: (2) No such file or directory 2024-09-10T15:53:08.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:08.084 INFO:tasks.util.rados:rados -p unfounddel_pool get new_4 - 2024-09-10T15:53:08.084 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_4 - 2024-09-10T15:53:08.128 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/new_4: (2) No such file or directory 2024-09-10T15:53:08.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:08.130 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_4 - 2024-09-10T15:53:08.131 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_4 - 2024-09-10T15:53:08.175 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existed_4: (2) No such file or directory 2024-09-10T15:53:08.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:08.208 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_4 - 2024-09-10T15:53:08.208 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_4 - 2024-09-10T15:53:08.253 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existing_4: (2) No such file or directory 2024-09-10T15:53:08.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:08.255 INFO:tasks.util.rados:rados -p unfounddel_pool get new_5 - 2024-09-10T15:53:08.256 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_5 - 2024-09-10T15:53:08.342 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/new_5: (2) No such file or directory 2024-09-10T15:53:08.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:08.347 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_5 - 2024-09-10T15:53:08.347 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_5 - 2024-09-10T15:53:08.447 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existed_5: (2) No such file or directory 2024-09-10T15:53:08.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:08.463 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_5 - 2024-09-10T15:53:08.463 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_5 - 2024-09-10T15:53:08.577 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existing_5: (2) No such file or directory 2024-09-10T15:53:08.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:08.588 INFO:tasks.util.rados:rados -p unfounddel_pool get new_6 - 2024-09-10T15:53:08.588 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_6 - 2024-09-10T15:53:08.676 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/new_6: (2) No such file or directory 2024-09-10T15:53:08.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:08.680 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_6 - 2024-09-10T15:53:08.680 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_6 - 2024-09-10T15:53:08.770 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existed_6: (2) No such file or directory 2024-09-10T15:53:08.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:08.778 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_6 - 2024-09-10T15:53:08.778 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_6 - 2024-09-10T15:53:08.801 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 31 20 54531 54511 6.86781 3.52734 0.0342651 0.0113691 2024-09-10T15:53:08.870 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existing_6: (2) No such file or directory 2024-09-10T15:53:08.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:08.878 INFO:tasks.util.rados:rados -p unfounddel_pool get new_7 - 2024-09-10T15:53:08.879 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_7 - 2024-09-10T15:53:09.022 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/new_7: (2) No such file or directory 2024-09-10T15:53:09.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:09.033 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_7 - 2024-09-10T15:53:09.034 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_7 - 2024-09-10T15:53:09.134 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existed_7: (2) No such file or directory 2024-09-10T15:53:09.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:09.137 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_7 - 2024-09-10T15:53:09.138 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_7 - 2024-09-10T15:53:09.234 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existing_7: (2) No such file or directory 2024-09-10T15:53:09.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:09.245 INFO:tasks.util.rados:rados -p unfounddel_pool get new_8 - 2024-09-10T15:53:09.245 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_8 - 2024-09-10T15:53:09.355 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/new_8: (2) No such file or directory 2024-09-10T15:53:09.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:09.362 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_8 - 2024-09-10T15:53:09.363 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_8 - 2024-09-10T15:53:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existed_8: (2) No such file or directory 2024-09-10T15:53:09.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:09.473 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_8 - 2024-09-10T15:53:09.474 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_8 - 2024-09-10T15:53:09.568 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existing_8: (2) No such file or directory 2024-09-10T15:53:09.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:09.582 INFO:tasks.util.rados:rados -p unfounddel_pool get new_9 - 2024-09-10T15:53:09.582 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_9 - 2024-09-10T15:53:09.685 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/new_9: (2) No such file or directory 2024-09-10T15:53:09.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:09.695 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_9 - 2024-09-10T15:53:09.695 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_9 - 2024-09-10T15:53:09.791 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existed_9: (2) No such file or directory 2024-09-10T15:53:09.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:09.801 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_9 - 2024-09-10T15:53:09.801 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_9 - 2024-09-10T15:53:09.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 32 20 55674 55654 6.79272 4.46484 0.0147459 0.011495 2024-09-10T15:53:09.881 INFO:teuthology.orchestra.run.smithi121.stderr:error getting unfounddel_pool/existing_9: (2) No such file or directory 2024-09-10T15:53:09.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T15:53:09.898 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-10T15:53:10.302 INFO:tasks.ceph.mon.c.smithi121.stderr:2024-09-10T15:53:10.300+0000 7fb1890bc640 -1 mon.c@2(leader).osd e37 definitely_dead 0 2024-09-10T15:53:10.389 INFO:teuthology.orchestra.run.smithi121.stderr:marked in osd.1. 2024-09-10T15:53:10.415 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T15:53:10.415 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T15:53:10.418 INFO:tasks.ceph.osd.1:Started 2024-09-10T15:53:10.418 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T15:53:10.515 INFO:teuthology.orchestra.run.smithi121.stderr:no valid command found; 10 closest matches: 2024-09-10T15:53:10.515 INFO:teuthology.orchestra.run.smithi121.stderr:0 2024-09-10T15:53:10.516 INFO:teuthology.orchestra.run.smithi121.stderr:1 2024-09-10T15:53:10.516 INFO:teuthology.orchestra.run.smithi121.stderr:2 2024-09-10T15:53:10.516 INFO:teuthology.orchestra.run.smithi121.stderr:abort 2024-09-10T15:53:10.516 INFO:teuthology.orchestra.run.smithi121.stderr:assert 2024-09-10T15:53:10.516 INFO:teuthology.orchestra.run.smithi121.stderr:config diff 2024-09-10T15:53:10.516 INFO:teuthology.orchestra.run.smithi121.stderr:config diff get 2024-09-10T15:53:10.516 INFO:teuthology.orchestra.run.smithi121.stderr:config get 2024-09-10T15:53:10.516 INFO:teuthology.orchestra.run.smithi121.stderr:config help [] 2024-09-10T15:53:10.516 INFO:teuthology.orchestra.run.smithi121.stderr:config set ... 2024-09-10T15:53:10.516 INFO:teuthology.orchestra.run.smithi121.stderr:admin_socket: invalid command 2024-09-10T15:53:10.517 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T15:53:10.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-10T15:53:10.801 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 33 20 56137 56117 6.64168 1.80859 0.00609609 0.0116178 2024-09-10T15:53:11.582 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-09-10T15:53:11.580+0000 7f3bc20e6640 -1 Falling back to public interface 2024-09-10T15:53:11.801 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 34 20 56137 56117 6.44634 0 - 0.0116178 2024-09-10T15:53:12.801 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 35 20 56598 56578 6.31361 0.900391 0.0109543 0.0123693 2024-09-10T15:53:13.804 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 36 20 57632 57612 6.25043 4.03906 0.0122529 0.0124935 2024-09-10T15:53:14.801 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 37 20 58976 58956 6.22338 5.25 0.0144505 0.0125486 2024-09-10T15:53:15.518 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T15:53:15.683 INFO:teuthology.orchestra.run.smithi121.stderr:no valid command found; 10 closest matches: 2024-09-10T15:53:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:0 2024-09-10T15:53:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:1 2024-09-10T15:53:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:2 2024-09-10T15:53:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:abort 2024-09-10T15:53:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:assert 2024-09-10T15:53:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:bluefs debug_inject_read_zeros 2024-09-10T15:53:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:bluefs files list 2024-09-10T15:53:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:bluefs stats 2024-09-10T15:53:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:bluestore allocator dump block 2024-09-10T15:53:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:bluestore allocator fragmentation block 2024-09-10T15:53:15.684 INFO:teuthology.orchestra.run.smithi121.stderr:admin_socket: invalid command 2024-09-10T15:53:15.686 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T15:53:15.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-10T15:53:15.737 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-09-10T15:53:15.735+0000 7f3bc20e6640 -1 osd.1 27 log_to_monitors true 2024-09-10T15:53:15.801 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 38 20 60193 60173 6.1847 4.75391 0.0123793 0.0126263 2024-09-10T15:53:16.470 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-09-10T15:53:16.468+0000 7f3bb9e9c640 -1 osd.1 27 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:53:16.801 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 39 20 61530 61510 6.16002 5.22266 0.0101212 0.0126768 2024-09-10T15:53:17.458 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-09-10T15:53:17.457+0000 7f3bb5a8b640 -1 osd.1 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:53:17.460 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-09-10T15:53:17.458+0000 7f3bb5a8b640 -1 osd.1 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:53:17.461 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-09-10T15:53:17.459+0000 7f3bb5a8b640 -1 osd.1 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T15:53:17.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:2024-09-10T15:53:17.801486+0000 min lat: 0.00198913 max lat: 2.08471 avg lat: 0.0127217 2024-09-10T15:53:17.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:53:17.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 40 20 62243 62223 6.07565 2.78516 0.208192 0.0127217 2024-09-10T15:53:18.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 41 20 62243 62223 5.92746 0 - 0.0127217 2024-09-10T15:53:19.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 42 20 62243 62223 5.78634 0 - 0.0127217 2024-09-10T15:53:20.688 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-10T15:53:20.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 43 20 62849 62829 5.70682 0.789062 0.0634775 0.013679 2024-09-10T15:53:20.867 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T15:53:20.868 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-09-10T15:53:20.868 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.868 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62866 2.f e44/43)", 2024-09-10T15:53:20.868 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.832379+0000", 2024-09-10T15:53:20.868 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.033631182000000003, 2024-09-10T15:53:20.868 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.033654128999999998, 2024-09-10T15:53:20.868 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.868 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.868 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "queued for pg", 2024-09-10T15:53:20.868 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.832379+0000", 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.832379+0000", 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.832380+0000", 2024-09-10T15:53:20.869 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4249999999999999e-06 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.832393+0000", 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2624e-05 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.832401+0000", 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4829999999999999e-06 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.870 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.832409+0000", 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.0830000000000004e-06 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62868 2.7 e44/43)", 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.835688+0000", 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.030321529, 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.030383298, 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.871 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "queued for pg", 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.835688+0000", 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.835688+0000", 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.872 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.835690+0000", 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.643e-06 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.835702+0000", 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1867e-05 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.835712+0000", 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0319999999999999e-05 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.873 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.835720+0000", 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1799999999999996e-06 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62875 2.f e44/43)", 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.851444+0000", 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.014565307, 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01465935, 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.874 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "queued for pg", 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851444+0000", 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851444+0000", 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.875 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851447+0000", 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3460000000000001e-06 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851464+0000", 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7003e-05 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851476+0000", 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2582000000000001e-05 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.876 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851485+0000", 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.6789999999999994e-06 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62876 2.7 e44/43)", 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.851563+0000", 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.014447108, 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014572603, 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.877 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "queued for pg", 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851563+0000", 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851563+0000", 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.878 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851563+0000", 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9999999999999999e-07 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851571+0000", 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.7170000000000008e-06 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851575+0000", 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5020000000000003e-06 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.879 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851579+0000", 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5279999999999999e-06 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62856 2.3 e44/43 2:cd80b193:::benchmark_data_smithi121_43865_object62855:head v 44'4037, mlcod=37'28)", 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.851616+0000", 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.014394122000000001, 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01455361, 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.880 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851616+0000", 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851616+0000", 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.881 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851616+0000", 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3099999999999999e-07 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851623+0000", 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9700000000000002e-06 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851627+0000", 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2180000000000001e-06 2024-09-10T15:53:20.882 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851631+0000", 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.455e-06 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851792+0000", 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016128799999999999 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.851812+0000", 2024-09-10T15:53:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9797e-05 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62874 2.3 e44/43 2:c48c73d4:::benchmark_data_smithi121_43865_object62873:head v 44'4038, mlcod=37'28)", 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.852634+0000", 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.013375740000000001, 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013576494999999999, 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-09-10T15:53:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.852634+0000", 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.852634+0000", 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.852635+0000", 2024-09-10T15:53:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9500000000000001e-07 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.852644+0000", 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.7809999999999994e-06 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.852655+0000", 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1391e-05 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.886 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.852661+0000", 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6210000000000001e-06 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.858218+0000", 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0055571309999999999 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.858237+0000", 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9558000000000001e-05 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.887 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62842 2.f e44/43 2:fe55bcd9:::benchmark_data_smithi121_43865_object62841:head v 44'4018, mlcod=0'0)", 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.797317+0000", 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.068692810000000007, 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.068933155999999995, 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.797317+0000", 2024-09-10T15:53:20.888 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.797317+0000", 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.797317+0000", 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7599999999999997e-07 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.797331+0000", 2024-09-10T15:53:20.889 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3745e-05 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.797337+0000", 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3539999999999999e-06 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.797341+0000", 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5170000000000001e-06 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-09-10T15:53:20.890 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.808026+0000", 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010684865 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.808901+0000", 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00087468400000000003 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62883 2.6 e44/43 2:6d02ba1e:::benchmark_data_smithi121_43865_object62882:head v 44'3970, mlcod=24'5)", 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.857021+0000", 2024-09-10T15:53:20.891 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.0089886940000000002, 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0092688149999999997, 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857021+0000", 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857021+0000", 2024-09-10T15:53:20.892 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857023+0000", 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4389999999999999e-06 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857044+0000", 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1365999999999998e-05 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857054+0000", 2024-09-10T15:53:20.893 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0288999999999999e-05 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857061+0000", 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9500000000000004e-06 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857648+0000", 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00058640799999999996 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-09-10T15:53:20.894 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857672+0000", 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4831999999999999e-05 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62880 2.7 e44/43)", 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.857136+0000", 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.0088736079999999998, 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0091895529999999996, 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "queued for pg", 2024-09-10T15:53:20.895 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857136+0000", 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857136+0000", 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857137+0000", 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7400000000000003e-07 2024-09-10T15:53:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857144+0000", 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4580000000000004e-06 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857150+0000", 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.7819999999999999e-06 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.897 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.857155+0000", 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5549999999999996e-06 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62884 2.6 e44/43 2:69369124:::benchmark_data_smithi121_43865_object62883:head v 44'3971, mlcod=24'5)", 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.860011+0000", 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.005998682, 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0063495349999999999, 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-09-10T15:53:20.898 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.860011+0000", 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.860011+0000", 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.860014+0000", 2024-09-10T15:53:20.899 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4310000000000001e-06 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.860028+0000", 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4951e-05 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.860040+0000", 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.184e-05 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.900 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.860050+0000", 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4010000000000003e-06 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.864229+0000", 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0041791170000000004 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.864260+0000", 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0898000000000001e-05 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.901 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62886 2.a e44/43 2:51ba4cb2:::benchmark_data_smithi121_43865_object62885:head v 44'3914, mlcod=24'3)", 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.866056+0000", 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4294967295.9999542, 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00034573600000000003, 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.866056+0000", 2024-09-10T15:53:20.902 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.866056+0000", 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.866057+0000", 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5039999999999999e-06 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.903 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.866070+0000", 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2942000000000001e-05 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.866082+0000", 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2267e-05 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.866090+0000", 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8439999999999998e-06 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-09-10T15:53:20.904 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.866146+0000", 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5779000000000002e-05 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.866172+0000", 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5806e-05 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62857 2.7 e44/43 2:e2c94f2c:::benchmark_data_smithi121_43865_object62856:head v 44'3876, mlcod=24'1)", 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.825049+0000", 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.040960284, 2024-09-10T15:53:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.041391681999999999, 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.825049+0000", 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.825049+0000", 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.906 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.825051+0000", 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.716e-06 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.825065+0000", 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3346000000000001e-05 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.825080+0000", 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5667000000000001e-05 2024-09-10T15:53:20.907 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.825088+0000", 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.0770000000000008e-06 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.832464+0000", 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073757140000000002 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.832488+0000", 2024-09-10T15:53:20.908 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3975e-05 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62859 2.f e44/43 2:f827fe35:::benchmark_data_smithi121_43865_object62858:head v 44'4019, mlcod=0'0)", 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.825105+0000", 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.040904531000000001, 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.041374340000000003, 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.909 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.825105+0000", 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.825105+0000", 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.825106+0000", 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9299999999999998e-07 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.910 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.825115+0000", 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9649999999999997e-06 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.825120+0000", 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0119999999999996e-06 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.825124+0000", 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9790000000000004e-06 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.911 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.843053+0000", 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.017929593000000001 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.843079+0000", 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5658999999999999e-05 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.912 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_repop(client.5205.0:62864 2.f e44/43 2:fcc4800d:::benchmark_data_smithi121_43865_object62863:head v 44'4020, mlcod=0'0)", 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-09-10T15:53:20.829943+0000", 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.036066516999999999, 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.036575539999999997, 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: "continuous": false, 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.829943+0000", 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.913 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.829943+0000", 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.829945+0000", 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9250000000000002e-06 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.829957+0000", 2024-09-10T15:53:20.914 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1674e-05 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.829968+0000", 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0688e-05 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.829975+0000", 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8560000000000007e-06 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-09-10T15:53:20.915 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.854822+0000", 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.024846133999999999 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-09-10T15:53:20.854879+0000", 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.7417999999999998e-05 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 14 2024-09-10T15:53:20.916 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T15:53:20.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.1 to be up 2024-09-10T15:53:20.918 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T15:53:21.467 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:21.468 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":44,"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","created":"2024-09-10T15:51:14.060262+0000","modified":"2024-09-10T15:53:19.457197+0000","last_up_change":"2024-09-10T15:53:17.454878+0000","last_in_change":"2024-09-10T15:53:10.382924+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:51:29.314004+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-10T15:51:40.401037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"53b58107-009b-46ac-921d-2e8d9580081e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":22,"up_from":26,"up_thru":43,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6800","nonce":824549766}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6801","nonce":824549766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6803","nonce":824549766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6802","nonce":824549766}]},"public_addr":"172.21.15.121:6800/824549766","cluster_addr":"172.21.15.121:6801/824549766","heartbeat_back_addr":"172.21.15.121:6803/824549766","heartbeat_front_addr":"172.21.15.121:6802/824549766","state":["exists","up"]},{"osd":1,"uuid":"aced9681-6a62-49a4-a1e2-6ff7d9a544ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":27,"up_from":42,"up_thru":43,"down_at":28,"lost_at":28,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6808","nonce":2529147590}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6809","nonce":2529147590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6811","nonce":2529147590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6810","nonce":2529147590}]},"public_addr":"172.21.15.121:6808/2529147590","cluster_addr":"172.21.15.121:6809/2529147590","heartbeat_back_addr":"172.21.15.121:6811/2529147590","heartbeat_front_addr":"172.21.15.121:6810/2529147590","state":["exists","up"]},{"osd":2,"uuid":"0cfba235-9896-429e-8396-69dfeb7a44bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":18,"up_from":32,"up_thru":43,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6804","nonce":4255745125}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6805","nonce":4255745125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6807","nonce":4255745125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6806","nonce":4255745125}]},"public_addr":"172.21.15.121:6804/4255745125","cluster_addr":"172.21.15.121:6805/4255745125","heartbeat_back_addr":"172.21.15.121:6807/4255745125","heartbeat_front_addr":"172.21.15.121:6806/4255745125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:22.164346+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.644539+0000","dead_epoch":28},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.413080+0000","dead_epoch":20}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"2.0","osds":[2]},{"pgid":"2.1","osds":[2]},{"pgid":"2.2","osds":[0]},{"pgid":"2.3","osds":[2]},{"pgid":"2.4","osds":[0]},{"pgid":"2.5","osds":[0]},{"pgid":"2.6","osds":[0]},{"pgid":"2.7","osds":[0]},{"pgid":"2.9","osds":[0]},{"pgid":"2.a","osds":[0]},{"pgid":"2.d","osds":[2]},{"pgid":"2.f","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T15:53:21.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.1 is up 2024-09-10T15:53:21.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-10T15:53:21.493 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:21.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 44 20 63463 63443 5.63163 2.39844 0.066045 0.0138616 2024-09-10T15:53:22.057 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:22.058 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:22.087 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-10T15:53:20.439825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":250696182,"num_objects":61208,"num_object_clones":0,"num_object_copies":122416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":30354,"num_objects_unfound":0,"num_objects_dirty":61208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22810,"num_write_kb":91234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":8211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61290,"ondisk_log_size":61290,"up":34,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":792016,"kb_used_data":488864,"kb_used_omap":12,"kb_used_meta":303091,"kb_avail":282323504,"statfs":{"total":289910292480,"available":289099268096,"internally_reserved":0,"allocated":500596736,"data_stored":499770990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":310365189},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,1,16,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6152192,"num_objects":1502,"num_object_clones":0,"num_object_copies":3004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-14479,"num_objects_unfound":0,"num_objects_dirty":1502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1502,"num_write_kb":6008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000712"},"pg_stats":[{"pgid":"2.f","version":"40'3853","reported_seq":3915,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:16.103250+0000","last_change":"2024-09-10T15:53:12.442070+0000","last_active":"2024-09-10T15:53:16.103250+0000","last_peered":"2024-09-10T15:53:16.103250+0000","last_clean":"2024-09-10T15:53:16.103250+0000","last_became_active":"2024-09-10T15:53:12.432657+0000","last_became_peered":"2024-09-10T15:53:12.432657+0000","last_unstale":"2024-09-10T15:53:16.103250+0000","last_undegraded":"2024-09-10T15:53:16.103250+0000","last_fullsized":"2024-09-10T15:53:16.103250+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3853,"log_dups_size":0,"ondisk_log_size":3853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:27.140450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15781888,"num_objects":3853,"num_object_clones":0,"num_object_copies":7706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3853,"num_objects_unfound":0,"num_objects_dirty":3853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3853,"num_write_kb":15412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3853}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'3966","reported_seq":4066,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.464296+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:18.464296+0000","last_peered":"2024-09-10T15:53:18.464296+0000","last_clean":"2024-09-10T15:53:18.464296+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:18.464296+0000","last_undegraded":"2024-09-10T15:53:18.464296+0000","last_fullsized":"2024-09-10T15:53:18.464296+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":3966,"log_dups_size":0,"ondisk_log_size":3966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16220160,"num_objects":3960,"num_object_clones":0,"num_object_copies":7920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2302,"num_write_kb":9208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'3806","reported_seq":3918,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.465374+0000","last_change":"2024-09-10T15:53:17.535147+0000","last_active":"2024-09-10T15:53:17.535147+0000","last_peered":"2024-09-10T15:53:17.329672+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:12.448159+0000","last_became_peered":"2024-09-10T15:53:12.448159+0000","last_unstale":"2024-09-10T15:53:18.465374+0000","last_undegraded":"2024-09-10T15:53:18.465374+0000","last_fullsized":"2024-09-10T15:53:18.465374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15572992,"num_objects":3802,"num_object_clones":0,"num_object_copies":7604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2074,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'3885","reported_seq":3957,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.463395+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:18.463395+0000","last_peered":"2024-09-10T15:53:18.463395+0000","last_clean":"2024-09-10T15:53:18.463395+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:18.463395+0000","last_undegraded":"2024-09-10T15:53:18.463395+0000","last_fullsized":"2024-09-10T15:53:18.463395+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3885,"log_dups_size":0,"ondisk_log_size":3885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15912960,"num_objects":3885,"num_object_clones":0,"num_object_copies":7770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3885,"num_write_kb":15540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"40'3760","reported_seq":3873,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T15:53:16.054750+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:16.054750+0000","last_peered":"2024-09-10T15:53:16.054750+0000","last_clean":"2024-09-10T15:53:16.054750+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:16.054750+0000","last_undegraded":"2024-09-10T15:53:16.054750+0000","last_fullsized":"2024-09-10T15:53:16.054750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":3760,"log_dups_size":0,"ondisk_log_size":3760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15372288,"num_objects":3753,"num_object_clones":0,"num_object_copies":7506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":831,"num_write_kb":3324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'3776","reported_seq":3894,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:16.113363+0000","last_change":"2024-09-10T15:53:12.436070+0000","last_active":"2024-09-10T15:53:16.113363+0000","last_peered":"2024-09-10T15:53:16.113363+0000","last_clean":"2024-09-10T15:53:16.113363+0000","last_became_active":"2024-09-10T15:53:12.428160+0000","last_became_peered":"2024-09-10T15:53:12.428160+0000","last_unstale":"2024-09-10T15:53:16.113363+0000","last_undegraded":"2024-09-10T15:53:16.113363+0000","last_fullsized":"2024-09-10T15:53:16.113363+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":3776,"log_dups_size":0,"ondisk_log_size":3776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:20.080593+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15446016,"num_objects":3771,"num_object_clones":0,"num_object_copies":7542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3771,"num_objects_unfound":0,"num_objects_dirty":3771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":494,"num_write_kb":1976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3771}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'3753","reported_seq":3878,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:16.110996+0000","last_change":"2024-09-10T15:53:12.472610+0000","last_active":"2024-09-10T15:53:16.110996+0000","last_peered":"2024-09-10T15:53:16.110996+0000","last_clean":"2024-09-10T15:53:16.110996+0000","last_became_active":"2024-09-10T15:53:12.457829+0000","last_became_peered":"2024-09-10T15:53:12.457829+0000","last_unstale":"2024-09-10T15:53:16.110996+0000","last_undegraded":"2024-09-10T15:53:16.110996+0000","last_fullsized":"2024-09-10T15:53:16.110996+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":3753,"log_dups_size":0,"ondisk_log_size":3753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:22.125908+0000","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15347712,"num_objects":3747,"num_object_clones":0,"num_object_copies":7494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3747,"num_objects_unfound":0,"num_objects_dirty":3747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":403,"num_write_kb":1612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3747}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'3803","reported_seq":3921,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T15:53:16.110905+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:16.110905+0000","last_peered":"2024-09-10T15:53:16.110905+0000","last_clean":"2024-09-10T15:53:16.110905+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:16.110905+0000","last_undegraded":"2024-09-10T15:53:16.110905+0000","last_fullsized":"2024-09-10T15:53:16.110905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":3803,"log_dups_size":0,"ondisk_log_size":3803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15568896,"num_objects":3801,"num_object_clones":0,"num_object_copies":7602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":562,"num_write_kb":2248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.437074+0000","last_change":"2024-09-10T15:53:12.429585+0000","last_active":"2024-09-10T15:53:12.437074+0000","last_peered":"2024-09-10T15:53:12.437074+0000","last_clean":"2024-09-10T15:53:12.437074+0000","last_became_active":"2024-09-10T15:53:12.428614+0000","last_became_peered":"2024-09-10T15:53:12.428614+0000","last_unstale":"2024-09-10T15:53:12.437074+0000","last_undegraded":"2024-09-10T15:53:12.437074+0000","last_fullsized":"2024-09-10T15:53:12.437074+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:39.542044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"41'3999","reported_seq":4116,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.467481+0000","last_change":"2024-09-10T15:53:17.534792+0000","last_active":"2024-09-10T15:53:17.534792+0000","last_peered":"2024-09-10T15:53:17.326712+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:12.430138+0000","last_became_peered":"2024-09-10T15:53:12.430138+0000","last_unstale":"2024-09-10T15:53:18.467481+0000","last_undegraded":"2024-09-10T15:53:18.467481+0000","last_fullsized":"2024-09-10T15:53:18.467481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":3999,"log_dups_size":0,"ondisk_log_size":3999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16351232,"num_objects":3992,"num_object_clones":0,"num_object_copies":7984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1976,"num_write_kb":7904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.465387+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:18.465387+0000","last_peered":"2024-09-10T15:53:18.465387+0000","last_clean":"2024-09-10T15:53:18.465387+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:18.465387+0000","last_undegraded":"2024-09-10T15:53:18.465387+0000","last_fullsized":"2024-09-10T15:53:18.465387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'3769","reported_seq":3879,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:16.089154+0000","last_change":"2024-09-10T15:53:12.436615+0000","last_active":"2024-09-10T15:53:16.089154+0000","last_peered":"2024-09-10T15:53:16.089154+0000","last_clean":"2024-09-10T15:53:16.089154+0000","last_became_active":"2024-09-10T15:53:12.428668+0000","last_became_peered":"2024-09-10T15:53:12.428668+0000","last_unstale":"2024-09-10T15:53:16.089154+0000","last_undegraded":"2024-09-10T15:53:16.089154+0000","last_fullsized":"2024-09-10T15:53:16.089154+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":3769,"log_dups_size":0,"ondisk_log_size":3769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:38.276474+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15405056,"num_objects":3761,"num_object_clones":0,"num_object_copies":7522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3761,"num_objects_unfound":0,"num_objects_dirty":3761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":633,"num_write_kb":2532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3761}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":82,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.436711+0000","last_change":"2024-09-10T15:53:12.436539+0000","last_active":"2024-09-10T15:53:12.436711+0000","last_peered":"2024-09-10T15:53:12.436711+0000","last_clean":"2024-09-10T15:53:12.436711+0000","last_became_active":"2024-09-10T15:53:12.434203+0000","last_became_peered":"2024-09-10T15:53:12.434203+0000","last_unstale":"2024-09-10T15:53:12.436711+0000","last_undegraded":"2024-09-10T15:53:12.436711+0000","last_fullsized":"2024-09-10T15:53:12.436711+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:55.094621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'3837","reported_seq":3950,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.469520+0000","last_change":"2024-09-10T15:53:17.533011+0000","last_active":"2024-09-10T15:53:17.533011+0000","last_peered":"2024-09-10T15:53:17.326494+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:12.445206+0000","last_became_peered":"2024-09-10T15:53:12.445206+0000","last_unstale":"2024-09-10T15:53:18.469520+0000","last_undegraded":"2024-09-10T15:53:18.469520+0000","last_fullsized":"2024-09-10T15:53:18.469520+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":3837,"log_dups_size":0,"ondisk_log_size":3837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15695872,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1811,"num_write_kb":7244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.427585+0000","last_change":"2024-09-10T15:53:12.427463+0000","last_active":"2024-09-10T15:53:12.427585+0000","last_peered":"2024-09-10T15:53:12.427585+0000","last_clean":"2024-09-10T15:53:12.427585+0000","last_became_active":"2024-09-10T15:53:12.427112+0000","last_became_peered":"2024-09-10T15:53:12.427112+0000","last_unstale":"2024-09-10T15:53:12.427585+0000","last_undegraded":"2024-09-10T15:53:12.427585+0000","last_fullsized":"2024-09-10T15:53:12.427585+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:07.176446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"41'3835","reported_seq":3950,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.460262+0000","last_change":"2024-09-10T15:53:17.533291+0000","last_active":"2024-09-10T15:53:17.533291+0000","last_peered":"2024-09-10T15:53:17.327041+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:12.427344+0000","last_became_peered":"2024-09-10T15:53:12.427344+0000","last_unstale":"2024-09-10T15:53:18.460262+0000","last_undegraded":"2024-09-10T15:53:18.460262+0000","last_fullsized":"2024-09-10T15:53:18.460262+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15683584,"num_objects":3829,"num_object_clones":0,"num_object_copies":7658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1704,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'3746","reported_seq":3859,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:16.100074+0000","last_change":"2024-09-10T15:53:12.434987+0000","last_active":"2024-09-10T15:53:16.100074+0000","last_peered":"2024-09-10T15:53:16.100074+0000","last_clean":"2024-09-10T15:53:16.100074+0000","last_became_active":"2024-09-10T15:53:12.429261+0000","last_became_peered":"2024-09-10T15:53:12.429261+0000","last_unstale":"2024-09-10T15:53:16.100074+0000","last_undegraded":"2024-09-10T15:53:16.100074+0000","last_fullsized":"2024-09-10T15:53:16.100074+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":3746,"log_dups_size":0,"ondisk_log_size":3746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:54.110370+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15335424,"num_objects":3744,"num_object_clones":0,"num_object_copies":7488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3744,"num_objects_unfound":0,"num_objects_dirty":3744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":938,"num_write_kb":3752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3744}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.429956+0000","last_change":"2024-09-10T15:53:12.429825+0000","last_active":"2024-09-10T15:53:12.429956+0000","last_peered":"2024-09-10T15:53:12.429956+0000","last_clean":"2024-09-10T15:53:12.429956+0000","last_became_active":"2024-09-10T15:53:12.428710+0000","last_became_peered":"2024-09-10T15:53:12.428710+0000","last_unstale":"2024-09-10T15:53:12.429956+0000","last_undegraded":"2024-09-10T15:53:12.429956+0000","last_fullsized":"2024-09-10T15:53:12.429956+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:45.544397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"40'3811","reported_seq":3959,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:16.064309+0000","last_change":"2024-09-10T15:53:13.900566+0000","last_active":"2024-09-10T15:53:16.064309+0000","last_peered":"2024-09-10T15:53:16.064309+0000","last_clean":"2024-09-10T15:53:16.064309+0000","last_became_active":"2024-09-10T15:53:12.428149+0000","last_became_peered":"2024-09-10T15:53:12.428149+0000","last_unstale":"2024-09-10T15:53:16.064309+0000","last_undegraded":"2024-09-10T15:53:16.064309+0000","last_fullsized":"2024-09-10T15:53:16.064309+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":3811,"log_dups_size":0,"ondisk_log_size":3811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:44.920308+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15577088,"num_objects":3803,"num_object_clones":0,"num_object_copies":7606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3803,"num_objects_unfound":0,"num_objects_dirty":3803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":229,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3803}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.464266+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:18.464266+0000","last_peered":"2024-09-10T15:53:18.464266+0000","last_clean":"2024-09-10T15:53:18.464266+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:18.464266+0000","last_undegraded":"2024-09-10T15:53:18.464266+0000","last_fullsized":"2024-09-10T15:53:18.464266+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'3786","reported_seq":3909,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:16.092758+0000","last_change":"2024-09-10T15:53:12.479587+0000","last_active":"2024-09-10T15:53:16.092758+0000","last_peered":"2024-09-10T15:53:16.092758+0000","last_clean":"2024-09-10T15:53:16.092758+0000","last_became_active":"2024-09-10T15:53:12.456599+0000","last_became_peered":"2024-09-10T15:53:12.456599+0000","last_unstale":"2024-09-10T15:53:16.092758+0000","last_undegraded":"2024-09-10T15:53:16.092758+0000","last_fullsized":"2024-09-10T15:53:16.092758+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":3786,"log_dups_size":0,"ondisk_log_size":3786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:24.752045+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15491072,"num_objects":3782,"num_object_clones":0,"num_object_copies":7564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3782,"num_objects_unfound":0,"num_objects_dirty":3782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":1568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3782}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.429568+0000","last_change":"2024-09-10T15:53:12.429456+0000","last_active":"2024-09-10T15:53:12.429568+0000","last_peered":"2024-09-10T15:53:12.429568+0000","last_clean":"2024-09-10T15:53:12.429568+0000","last_became_active":"2024-09-10T15:53:12.428173+0000","last_became_peered":"2024-09-10T15:53:12.428173+0000","last_unstale":"2024-09-10T15:53:12.429568+0000","last_undegraded":"2024-09-10T15:53:12.429568+0000","last_fullsized":"2024-09-10T15:53:12.429568+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:42.683814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.431101+0000","last_change":"2024-09-10T15:53:12.430966+0000","last_active":"2024-09-10T15:53:12.431101+0000","last_peered":"2024-09-10T15:53:12.431101+0000","last_clean":"2024-09-10T15:53:12.431101+0000","last_became_active":"2024-09-10T15:53:12.430496+0000","last_became_peered":"2024-09-10T15:53:12.430496+0000","last_unstale":"2024-09-10T15:53:12.431101+0000","last_undegraded":"2024-09-10T15:53:12.431101+0000","last_fullsized":"2024-09-10T15:53:12.431101+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:04.534041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"40'3902","reported_seq":4025,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:16.058383+0000","last_change":"2024-09-10T15:53:12.436268+0000","last_active":"2024-09-10T15:53:16.058383+0000","last_peered":"2024-09-10T15:53:16.058383+0000","last_clean":"2024-09-10T15:53:16.058383+0000","last_became_active":"2024-09-10T15:53:12.430108+0000","last_became_peered":"2024-09-10T15:53:12.430108+0000","last_unstale":"2024-09-10T15:53:16.058383+0000","last_undegraded":"2024-09-10T15:53:16.058383+0000","last_fullsized":"2024-09-10T15:53:16.058383+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":3902,"log_dups_size":0,"ondisk_log_size":3902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:36.194371+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15933923,"num_objects":3891,"num_object_clones":0,"num_object_copies":7782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3891,"num_objects_unfound":0,"num_objects_dirty":3891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":721,"num_write_kb":2884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3891}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":250696163,"num_objects":61206,"num_object_clones":0,"num_object_copies":122412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":30352,"num_objects_unfound":0,"num_objects_dirty":61206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22808,"num_write_kb":91232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":503345152,"data_stored":503247601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61287,"ondisk_log_size":61287,"up":24,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":10,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626434,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":462848,"data_stored":124286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149717,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374864,"kb_used_data":239104,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93996976,"statfs":{"total":96636764160,"available":96252903424,"internally_reserved":0,"allocated":244842496,"data_stored":244605803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":138997421},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,7,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953487,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389228,"kb_used_data":249308,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93982612,"statfs":{"total":96636764160,"available":96238194688,"internally_reserved":0,"allocated":255291392,"data_stored":255040901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,9,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":248315904,"data_stored":248312291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":12075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":254926848,"data_stored":254923235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:22.088 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:22.638 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:22.667 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-10T15:53:22.442955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257085942,"num_objects":62768,"num_object_clones":0,"num_object_copies":125536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31183,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":62768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24370,"num_write_kb":97474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":8211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62850,"ondisk_log_size":62850,"up":43,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":811648,"kb_used_data":500176,"kb_used_omap":12,"kb_used_meta":311411,"kb_avail":282303872,"statfs":{"total":289910292480,"available":289079164928,"internally_reserved":0,"allocated":512180224,"data_stored":511335513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":318884869},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,9,4,0,0,4,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12541952,"num_objects":3062,"num_object_clones":0,"num_object_copies":6124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31183,"num_objects_misplaced":-44832,"num_objects_unfound":0,"num_objects_dirty":3062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3062,"num_write_kb":12248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.983592"},"pg_stats":[{"pgid":"2.f","version":"44'4034","reported_seq":4104,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.118934+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:21.118934+0000","last_peered":"2024-09-10T15:53:21.118934+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:21.118934+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4034,"log_dups_size":0,"ondisk_log_size":4034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16523264,"num_objects":4034,"num_object_clones":0,"num_object_copies":8068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4034,"num_write_kb":16136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":61},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'3966","reported_seq":4066,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.464296+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:18.464296+0000","last_peered":"2024-09-10T15:53:18.464296+0000","last_clean":"2024-09-10T15:53:18.464296+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:18.464296+0000","last_undegraded":"2024-09-10T15:53:18.464296+0000","last_fullsized":"2024-09-10T15:53:18.464296+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":3966,"log_dups_size":0,"ondisk_log_size":3966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16220160,"num_objects":3960,"num_object_clones":0,"num_object_copies":7920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2302,"num_write_kb":9208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'3806","reported_seq":3918,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.465374+0000","last_change":"2024-09-10T15:53:17.535147+0000","last_active":"2024-09-10T15:53:17.535147+0000","last_peered":"2024-09-10T15:53:17.329672+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:12.448159+0000","last_became_peered":"2024-09-10T15:53:12.448159+0000","last_unstale":"2024-09-10T15:53:18.465374+0000","last_undegraded":"2024-09-10T15:53:18.465374+0000","last_fullsized":"2024-09-10T15:53:18.465374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15572992,"num_objects":3802,"num_object_clones":0,"num_object_copies":7604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2074,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'3885","reported_seq":3957,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.463395+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:18.463395+0000","last_peered":"2024-09-10T15:53:18.463395+0000","last_clean":"2024-09-10T15:53:18.463395+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:18.463395+0000","last_undegraded":"2024-09-10T15:53:18.463395+0000","last_fullsized":"2024-09-10T15:53:18.463395+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3885,"log_dups_size":0,"ondisk_log_size":3885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15912960,"num_objects":3885,"num_object_clones":0,"num_object_copies":7770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3885,"num_write_kb":15540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'3914","reported_seq":4034,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:21.069541+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:21.069541+0000","last_peered":"2024-09-10T15:53:21.069541+0000","last_clean":"2024-09-10T15:53:21.069541+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:21.069541+0000","last_undegraded":"2024-09-10T15:53:21.069541+0000","last_fullsized":"2024-09-10T15:53:21.069541+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":3914,"log_dups_size":0,"ondisk_log_size":3914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16003072,"num_objects":3907,"num_object_clones":0,"num_object_copies":7814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":985,"num_write_kb":3940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'3925","reported_seq":4052,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.155397+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:21.155397+0000","last_peered":"2024-09-10T15:53:21.155397+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:21.155397+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":3925,"log_dups_size":0,"ondisk_log_size":3925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16056320,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":643,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":52},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'3894","reported_seq":4029,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.124600+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:21.124600+0000","last_peered":"2024-09-10T15:53:21.124600+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:21.124600+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":3894,"log_dups_size":0,"ondisk_log_size":3894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":3888,"num_object_clones":0,"num_object_copies":7776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":54},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'3972","reported_seq":4098,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:21.098719+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:21.098719+0000","last_peered":"2024-09-10T15:53:21.098719+0000","last_clean":"2024-09-10T15:53:21.098719+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:21.098719+0000","last_undegraded":"2024-09-10T15:53:21.098719+0000","last_fullsized":"2024-09-10T15:53:21.098719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":3972,"log_dups_size":0,"ondisk_log_size":3972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16261120,"num_objects":3970,"num_object_clones":0,"num_object_copies":7940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":731,"num_write_kb":2924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.437074+0000","last_change":"2024-09-10T15:53:12.429585+0000","last_active":"2024-09-10T15:53:12.437074+0000","last_peered":"2024-09-10T15:53:12.437074+0000","last_clean":"2024-09-10T15:53:12.437074+0000","last_became_active":"2024-09-10T15:53:12.428614+0000","last_became_peered":"2024-09-10T15:53:12.428614+0000","last_unstale":"2024-09-10T15:53:12.437074+0000","last_undegraded":"2024-09-10T15:53:12.437074+0000","last_fullsized":"2024-09-10T15:53:12.437074+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:39.542044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"41'3999","reported_seq":4116,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.467481+0000","last_change":"2024-09-10T15:53:17.534792+0000","last_active":"2024-09-10T15:53:17.534792+0000","last_peered":"2024-09-10T15:53:17.326712+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:12.430138+0000","last_became_peered":"2024-09-10T15:53:12.430138+0000","last_unstale":"2024-09-10T15:53:18.467481+0000","last_undegraded":"2024-09-10T15:53:18.467481+0000","last_fullsized":"2024-09-10T15:53:18.467481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":3999,"log_dups_size":0,"ondisk_log_size":3999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16351232,"num_objects":3992,"num_object_clones":0,"num_object_copies":7984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1976,"num_write_kb":7904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.465387+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:18.465387+0000","last_peered":"2024-09-10T15:53:18.465387+0000","last_clean":"2024-09-10T15:53:18.465387+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:18.465387+0000","last_undegraded":"2024-09-10T15:53:18.465387+0000","last_fullsized":"2024-09-10T15:53:18.465387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'3909","reported_seq":4030,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.148367+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:21.148367+0000","last_peered":"2024-09-10T15:53:21.148367+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:21.148367+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":3909,"log_dups_size":0,"ondisk_log_size":3909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":773,"num_write_kb":3092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":38},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'3837","reported_seq":3950,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.469520+0000","last_change":"2024-09-10T15:53:17.533011+0000","last_active":"2024-09-10T15:53:17.533011+0000","last_peered":"2024-09-10T15:53:17.326494+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:12.445206+0000","last_became_peered":"2024-09-10T15:53:12.445206+0000","last_unstale":"2024-09-10T15:53:18.469520+0000","last_undegraded":"2024-09-10T15:53:18.469520+0000","last_fullsized":"2024-09-10T15:53:18.469520+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":3837,"log_dups_size":0,"ondisk_log_size":3837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15695872,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1811,"num_write_kb":7244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.427585+0000","last_change":"2024-09-10T15:53:12.427463+0000","last_active":"2024-09-10T15:53:12.427585+0000","last_peered":"2024-09-10T15:53:12.427585+0000","last_clean":"2024-09-10T15:53:12.427585+0000","last_became_active":"2024-09-10T15:53:12.427112+0000","last_became_peered":"2024-09-10T15:53:12.427112+0000","last_unstale":"2024-09-10T15:53:12.427585+0000","last_undegraded":"2024-09-10T15:53:12.427585+0000","last_fullsized":"2024-09-10T15:53:12.427585+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:07.176446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"41'3835","reported_seq":3950,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.460262+0000","last_change":"2024-09-10T15:53:17.533291+0000","last_active":"2024-09-10T15:53:17.533291+0000","last_peered":"2024-09-10T15:53:17.327041+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:12.427344+0000","last_became_peered":"2024-09-10T15:53:12.427344+0000","last_unstale":"2024-09-10T15:53:18.460262+0000","last_undegraded":"2024-09-10T15:53:18.460262+0000","last_fullsized":"2024-09-10T15:53:18.460262+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15683584,"num_objects":3829,"num_object_clones":0,"num_object_copies":7658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1704,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'3888","reported_seq":4010,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.054072+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:21.054072+0000","last_peered":"2024-09-10T15:53:21.054072+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:21.054072+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":3888,"log_dups_size":0,"ondisk_log_size":3888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15917056,"num_objects":3886,"num_object_clones":0,"num_object_copies":7772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1080,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":53},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.429956+0000","last_change":"2024-09-10T15:53:12.429825+0000","last_active":"2024-09-10T15:53:12.429956+0000","last_peered":"2024-09-10T15:53:12.429956+0000","last_clean":"2024-09-10T15:53:12.429956+0000","last_became_active":"2024-09-10T15:53:12.428710+0000","last_became_peered":"2024-09-10T15:53:12.428710+0000","last_unstale":"2024-09-10T15:53:12.429956+0000","last_undegraded":"2024-09-10T15:53:12.429956+0000","last_fullsized":"2024-09-10T15:53:12.429956+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:45.544397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"44'3985","reported_seq":4142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.143387+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:21.143387+0000","last_peered":"2024-09-10T15:53:21.143387+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:21.143387+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":3985,"log_dups_size":0,"ondisk_log_size":3985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16289792,"num_objects":3977,"num_object_clones":0,"num_object_copies":7954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":403,"num_write_kb":1612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":60},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.464266+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:18.464266+0000","last_peered":"2024-09-10T15:53:18.464266+0000","last_clean":"2024-09-10T15:53:18.464266+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:18.464266+0000","last_undegraded":"2024-09-10T15:53:18.464266+0000","last_fullsized":"2024-09-10T15:53:18.464266+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'3929","reported_seq":4063,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.123029+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:21.123029+0000","last_peered":"2024-09-10T15:53:21.123029+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:21.123029+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16076800,"num_objects":3925,"num_object_clones":0,"num_object_copies":7850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":535,"num_write_kb":2140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":48},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.429568+0000","last_change":"2024-09-10T15:53:12.429456+0000","last_active":"2024-09-10T15:53:12.429568+0000","last_peered":"2024-09-10T15:53:12.429568+0000","last_clean":"2024-09-10T15:53:12.429568+0000","last_became_active":"2024-09-10T15:53:12.428173+0000","last_became_peered":"2024-09-10T15:53:12.428173+0000","last_unstale":"2024-09-10T15:53:12.429568+0000","last_undegraded":"2024-09-10T15:53:12.429568+0000","last_fullsized":"2024-09-10T15:53:12.429568+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:42.683814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.431101+0000","last_change":"2024-09-10T15:53:12.430966+0000","last_active":"2024-09-10T15:53:12.431101+0000","last_peered":"2024-09-10T15:53:12.431101+0000","last_clean":"2024-09-10T15:53:12.431101+0000","last_became_active":"2024-09-10T15:53:12.430496+0000","last_became_peered":"2024-09-10T15:53:12.430496+0000","last_unstale":"2024-09-10T15:53:12.431101+0000","last_undegraded":"2024-09-10T15:53:12.431101+0000","last_fullsized":"2024-09-10T15:53:12.431101+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:04.534041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"44'4069","reported_seq":4202,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.108890+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:21.108890+0000","last_peered":"2024-09-10T15:53:21.108890+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:21.108890+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4069,"log_dups_size":0,"ondisk_log_size":4069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16617955,"num_objects":4058,"num_object_clones":0,"num_object_copies":8116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":55},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":257085923,"num_objects":62766,"num_object_clones":0,"num_object_copies":125532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24368,"num_write_kb":97472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512561152,"data_stored":512463601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62847,"ondisk_log_size":62847,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":11,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626434,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":462848,"data_stored":124286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149718,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394496,"kb_used_data":250416,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93977344,"statfs":{"total":96636764160,"available":96232800256,"internally_reserved":0,"allocated":256425984,"data_stored":256170326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":147517101},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,4,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953487,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389228,"kb_used_data":249308,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93982612,"statfs":{"total":96636764160,"available":96238194688,"internally_reserved":0,"allocated":255291392,"data_stored":255040901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,9,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257531904,"data_stored":257528291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":12075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":254926848,"data_stored":254923235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:22.669 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:22.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 45 20 64089 64069 5.56082 2.44531 0.0604922 0.0140405 2024-09-10T15:53:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:23.087 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:23.111 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-10T15:53:22.442955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257085942,"num_objects":62768,"num_object_clones":0,"num_object_copies":125536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31183,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":62768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24370,"num_write_kb":97474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":8211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62850,"ondisk_log_size":62850,"up":43,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":811648,"kb_used_data":500176,"kb_used_omap":12,"kb_used_meta":311411,"kb_avail":282303872,"statfs":{"total":289910292480,"available":289079164928,"internally_reserved":0,"allocated":512180224,"data_stored":511335513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":318884869},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,9,4,0,0,4,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12541952,"num_objects":3062,"num_object_clones":0,"num_object_copies":6124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31183,"num_objects_misplaced":-44832,"num_objects_unfound":0,"num_objects_dirty":3062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3062,"num_write_kb":12248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.983592"},"pg_stats":[{"pgid":"2.f","version":"44'4034","reported_seq":4104,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.118934+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:21.118934+0000","last_peered":"2024-09-10T15:53:21.118934+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:21.118934+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4034,"log_dups_size":0,"ondisk_log_size":4034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16523264,"num_objects":4034,"num_object_clones":0,"num_object_copies":8068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4034,"num_write_kb":16136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":61},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'3966","reported_seq":4066,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.464296+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:18.464296+0000","last_peered":"2024-09-10T15:53:18.464296+0000","last_clean":"2024-09-10T15:53:18.464296+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:18.464296+0000","last_undegraded":"2024-09-10T15:53:18.464296+0000","last_fullsized":"2024-09-10T15:53:18.464296+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":3966,"log_dups_size":0,"ondisk_log_size":3966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16220160,"num_objects":3960,"num_object_clones":0,"num_object_copies":7920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2302,"num_write_kb":9208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'3806","reported_seq":3918,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.465374+0000","last_change":"2024-09-10T15:53:17.535147+0000","last_active":"2024-09-10T15:53:17.535147+0000","last_peered":"2024-09-10T15:53:17.329672+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:12.448159+0000","last_became_peered":"2024-09-10T15:53:12.448159+0000","last_unstale":"2024-09-10T15:53:18.465374+0000","last_undegraded":"2024-09-10T15:53:18.465374+0000","last_fullsized":"2024-09-10T15:53:18.465374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15572992,"num_objects":3802,"num_object_clones":0,"num_object_copies":7604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2074,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'3885","reported_seq":3957,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.463395+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:18.463395+0000","last_peered":"2024-09-10T15:53:18.463395+0000","last_clean":"2024-09-10T15:53:18.463395+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:18.463395+0000","last_undegraded":"2024-09-10T15:53:18.463395+0000","last_fullsized":"2024-09-10T15:53:18.463395+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3885,"log_dups_size":0,"ondisk_log_size":3885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15912960,"num_objects":3885,"num_object_clones":0,"num_object_copies":7770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3885,"num_write_kb":15540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'3914","reported_seq":4034,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:21.069541+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:21.069541+0000","last_peered":"2024-09-10T15:53:21.069541+0000","last_clean":"2024-09-10T15:53:21.069541+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:21.069541+0000","last_undegraded":"2024-09-10T15:53:21.069541+0000","last_fullsized":"2024-09-10T15:53:21.069541+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":3914,"log_dups_size":0,"ondisk_log_size":3914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16003072,"num_objects":3907,"num_object_clones":0,"num_object_copies":7814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":985,"num_write_kb":3940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'3925","reported_seq":4052,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.155397+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:21.155397+0000","last_peered":"2024-09-10T15:53:21.155397+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:21.155397+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":3925,"log_dups_size":0,"ondisk_log_size":3925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16056320,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":643,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":52},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'3894","reported_seq":4029,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.124600+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:21.124600+0000","last_peered":"2024-09-10T15:53:21.124600+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:21.124600+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":3894,"log_dups_size":0,"ondisk_log_size":3894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":3888,"num_object_clones":0,"num_object_copies":7776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":54},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'3972","reported_seq":4098,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:21.098719+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:21.098719+0000","last_peered":"2024-09-10T15:53:21.098719+0000","last_clean":"2024-09-10T15:53:21.098719+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:21.098719+0000","last_undegraded":"2024-09-10T15:53:21.098719+0000","last_fullsized":"2024-09-10T15:53:21.098719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":3972,"log_dups_size":0,"ondisk_log_size":3972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16261120,"num_objects":3970,"num_object_clones":0,"num_object_copies":7940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":731,"num_write_kb":2924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.437074+0000","last_change":"2024-09-10T15:53:12.429585+0000","last_active":"2024-09-10T15:53:12.437074+0000","last_peered":"2024-09-10T15:53:12.437074+0000","last_clean":"2024-09-10T15:53:12.437074+0000","last_became_active":"2024-09-10T15:53:12.428614+0000","last_became_peered":"2024-09-10T15:53:12.428614+0000","last_unstale":"2024-09-10T15:53:12.437074+0000","last_undegraded":"2024-09-10T15:53:12.437074+0000","last_fullsized":"2024-09-10T15:53:12.437074+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:39.542044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"41'3999","reported_seq":4116,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.467481+0000","last_change":"2024-09-10T15:53:17.534792+0000","last_active":"2024-09-10T15:53:17.534792+0000","last_peered":"2024-09-10T15:53:17.326712+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:12.430138+0000","last_became_peered":"2024-09-10T15:53:12.430138+0000","last_unstale":"2024-09-10T15:53:18.467481+0000","last_undegraded":"2024-09-10T15:53:18.467481+0000","last_fullsized":"2024-09-10T15:53:18.467481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":3999,"log_dups_size":0,"ondisk_log_size":3999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16351232,"num_objects":3992,"num_object_clones":0,"num_object_copies":7984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1976,"num_write_kb":7904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.465387+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:18.465387+0000","last_peered":"2024-09-10T15:53:18.465387+0000","last_clean":"2024-09-10T15:53:18.465387+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:18.465387+0000","last_undegraded":"2024-09-10T15:53:18.465387+0000","last_fullsized":"2024-09-10T15:53:18.465387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'3909","reported_seq":4030,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.148367+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:21.148367+0000","last_peered":"2024-09-10T15:53:21.148367+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:21.148367+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":3909,"log_dups_size":0,"ondisk_log_size":3909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":773,"num_write_kb":3092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":38},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'3837","reported_seq":3950,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.469520+0000","last_change":"2024-09-10T15:53:17.533011+0000","last_active":"2024-09-10T15:53:17.533011+0000","last_peered":"2024-09-10T15:53:17.326494+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:12.445206+0000","last_became_peered":"2024-09-10T15:53:12.445206+0000","last_unstale":"2024-09-10T15:53:18.469520+0000","last_undegraded":"2024-09-10T15:53:18.469520+0000","last_fullsized":"2024-09-10T15:53:18.469520+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":3837,"log_dups_size":0,"ondisk_log_size":3837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15695872,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1811,"num_write_kb":7244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.427585+0000","last_change":"2024-09-10T15:53:12.427463+0000","last_active":"2024-09-10T15:53:12.427585+0000","last_peered":"2024-09-10T15:53:12.427585+0000","last_clean":"2024-09-10T15:53:12.427585+0000","last_became_active":"2024-09-10T15:53:12.427112+0000","last_became_peered":"2024-09-10T15:53:12.427112+0000","last_unstale":"2024-09-10T15:53:12.427585+0000","last_undegraded":"2024-09-10T15:53:12.427585+0000","last_fullsized":"2024-09-10T15:53:12.427585+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:07.176446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"41'3835","reported_seq":3950,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.460262+0000","last_change":"2024-09-10T15:53:17.533291+0000","last_active":"2024-09-10T15:53:17.533291+0000","last_peered":"2024-09-10T15:53:17.327041+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:12.427344+0000","last_became_peered":"2024-09-10T15:53:12.427344+0000","last_unstale":"2024-09-10T15:53:18.460262+0000","last_undegraded":"2024-09-10T15:53:18.460262+0000","last_fullsized":"2024-09-10T15:53:18.460262+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15683584,"num_objects":3829,"num_object_clones":0,"num_object_copies":7658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1704,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'3888","reported_seq":4010,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.054072+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:21.054072+0000","last_peered":"2024-09-10T15:53:21.054072+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:21.054072+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":3888,"log_dups_size":0,"ondisk_log_size":3888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15917056,"num_objects":3886,"num_object_clones":0,"num_object_copies":7772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1080,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":53},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.429956+0000","last_change":"2024-09-10T15:53:12.429825+0000","last_active":"2024-09-10T15:53:12.429956+0000","last_peered":"2024-09-10T15:53:12.429956+0000","last_clean":"2024-09-10T15:53:12.429956+0000","last_became_active":"2024-09-10T15:53:12.428710+0000","last_became_peered":"2024-09-10T15:53:12.428710+0000","last_unstale":"2024-09-10T15:53:12.429956+0000","last_undegraded":"2024-09-10T15:53:12.429956+0000","last_fullsized":"2024-09-10T15:53:12.429956+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:45.544397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"44'3985","reported_seq":4142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.143387+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:21.143387+0000","last_peered":"2024-09-10T15:53:21.143387+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:21.143387+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":3985,"log_dups_size":0,"ondisk_log_size":3985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16289792,"num_objects":3977,"num_object_clones":0,"num_object_copies":7954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":403,"num_write_kb":1612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":60},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.464266+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:18.464266+0000","last_peered":"2024-09-10T15:53:18.464266+0000","last_clean":"2024-09-10T15:53:18.464266+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:18.464266+0000","last_undegraded":"2024-09-10T15:53:18.464266+0000","last_fullsized":"2024-09-10T15:53:18.464266+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'3929","reported_seq":4063,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.123029+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:21.123029+0000","last_peered":"2024-09-10T15:53:21.123029+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:21.123029+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16076800,"num_objects":3925,"num_object_clones":0,"num_object_copies":7850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":535,"num_write_kb":2140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":48},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.429568+0000","last_change":"2024-09-10T15:53:12.429456+0000","last_active":"2024-09-10T15:53:12.429568+0000","last_peered":"2024-09-10T15:53:12.429568+0000","last_clean":"2024-09-10T15:53:12.429568+0000","last_became_active":"2024-09-10T15:53:12.428173+0000","last_became_peered":"2024-09-10T15:53:12.428173+0000","last_unstale":"2024-09-10T15:53:12.429568+0000","last_undegraded":"2024-09-10T15:53:12.429568+0000","last_fullsized":"2024-09-10T15:53:12.429568+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:42.683814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.431101+0000","last_change":"2024-09-10T15:53:12.430966+0000","last_active":"2024-09-10T15:53:12.431101+0000","last_peered":"2024-09-10T15:53:12.431101+0000","last_clean":"2024-09-10T15:53:12.431101+0000","last_became_active":"2024-09-10T15:53:12.430496+0000","last_became_peered":"2024-09-10T15:53:12.430496+0000","last_unstale":"2024-09-10T15:53:12.431101+0000","last_undegraded":"2024-09-10T15:53:12.431101+0000","last_fullsized":"2024-09-10T15:53:12.431101+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:04.534041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"44'4069","reported_seq":4202,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.108890+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:21.108890+0000","last_peered":"2024-09-10T15:53:21.108890+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:21.108890+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4069,"log_dups_size":0,"ondisk_log_size":4069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16617955,"num_objects":4058,"num_object_clones":0,"num_object_copies":8116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":55},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":257085923,"num_objects":62766,"num_object_clones":0,"num_object_copies":125532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24368,"num_write_kb":97472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512561152,"data_stored":512463601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62847,"ondisk_log_size":62847,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":11,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626434,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":462848,"data_stored":124286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149718,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394496,"kb_used_data":250416,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93977344,"statfs":{"total":96636764160,"available":96232800256,"internally_reserved":0,"allocated":256425984,"data_stored":256170326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":147517101},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,4,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953487,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389228,"kb_used_data":249308,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93982612,"statfs":{"total":96636764160,"available":96238194688,"internally_reserved":0,"allocated":255291392,"data_stored":255040901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,9,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257531904,"data_stored":257528291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":12075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":254926848,"data_stored":254923235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:23.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:53:23.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'4034", 'reported_seq': 4104, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:21.118934+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:53:21.118934+0000', 'last_peered': '2024-09-10T15:53:21.118934+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:53:21.118934+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 4034, 'log_dups_size': 0, 'ondisk_log_size': 4034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16523264, 'num_objects': 4034, 'num_object_clones': 0, 'num_object_copies': 8068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4034, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4034, 'num_write_kb': 16136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 61}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:23.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:53:23.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "41'3806", 'reported_seq': 3918, 'reported_epoch': 43, 'state': 'remapped+peering', 'last_fresh': '2024-09-10T15:53:18.465374+0000', 'last_change': '2024-09-10T15:53:17.535147+0000', 'last_active': '2024-09-10T15:53:17.535147+0000', 'last_peered': '2024-09-10T15:53:17.329672+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:12.448159+0000', 'last_became_peered': '2024-09-10T15:53:12.448159+0000', 'last_unstale': '2024-09-10T15:53:18.465374+0000', 'last_undegraded': '2024-09-10T15:53:18.465374+0000', 'last_fullsized': '2024-09-10T15:53:18.465374+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 3806, 'log_dups_size': 0, 'ondisk_log_size': 3806, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15572992, 'num_objects': 3802, 'num_object_clones': 0, 'num_object_copies': 7604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3802, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2074, 'num_write_kb': 8296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:23.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:53:23.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'3925", 'reported_seq': 4052, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:21.155397+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:53:21.155397+0000', 'last_peered': '2024-09-10T15:53:21.155397+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:53:21.155397+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 3925, 'log_dups_size': 0, 'ondisk_log_size': 3925, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16056320, 'num_objects': 3920, 'num_object_clones': 0, 'num_object_copies': 7840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 643, 'num_write_kb': 2572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 52}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:23.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:53:23.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'3894", 'reported_seq': 4029, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:21.124600+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:53:21.124600+0000', 'last_peered': '2024-09-10T15:53:21.124600+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:53:21.124600+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 3894, 'log_dups_size': 0, 'ondisk_log_size': 3894, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15925248, 'num_objects': 3888, 'num_object_clones': 0, 'num_object_copies': 7776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 544, 'num_write_kb': 2176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 54}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:23.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.0 is not active+clean 2024-09-10T15:53:23.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 90, 'reported_epoch': 40, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T15:53:12.437074+0000', 'last_change': '2024-09-10T15:53:12.429585+0000', 'last_active': '2024-09-10T15:53:12.437074+0000', 'last_peered': '2024-09-10T15:53:12.437074+0000', 'last_clean': '2024-09-10T15:53:12.437074+0000', 'last_became_active': '2024-09-10T15:53:12.428614+0000', 'last_became_peered': '2024-09-10T15:53:12.428614+0000', 'last_unstale': '2024-09-10T15:53:12.437074+0000', 'last_undegraded': '2024-09-10T15:53:12.437074+0000', 'last_fullsized': '2024-09-10T15:53:12.437074+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 12, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-12T02:42:39.542044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000809975, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0], 'acting': [0, 2], 'avail_no_missing': ['0', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:23.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:53:23.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "41'3999", 'reported_seq': 4116, 'reported_epoch': 43, 'state': 'remapped+peering', 'last_fresh': '2024-09-10T15:53:18.467481+0000', 'last_change': '2024-09-10T15:53:17.534792+0000', 'last_active': '2024-09-10T15:53:17.534792+0000', 'last_peered': '2024-09-10T15:53:17.326712+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:12.430138+0000', 'last_became_peered': '2024-09-10T15:53:12.430138+0000', 'last_unstale': '2024-09-10T15:53:18.467481+0000', 'last_undegraded': '2024-09-10T15:53:18.467481+0000', 'last_fullsized': '2024-09-10T15:53:18.467481+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 3999, 'log_dups_size': 0, 'ondisk_log_size': 3999, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16351232, 'num_objects': 3992, 'num_object_clones': 0, 'num_object_copies': 7984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3992, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1976, 'num_write_kb': 7904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:23.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:53:23.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'3909", 'reported_seq': 4030, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:21.148367+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:53:21.148367+0000', 'last_peered': '2024-09-10T15:53:21.148367+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:53:21.148367+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 3909, 'log_dups_size': 0, 'ondisk_log_size': 3909, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15978496, 'num_objects': 3901, 'num_object_clones': 0, 'num_object_copies': 7802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 773, 'num_write_kb': 3092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 38}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:23.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:53:23.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "41'3837", 'reported_seq': 3950, 'reported_epoch': 43, 'state': 'remapped+peering', 'last_fresh': '2024-09-10T15:53:18.469520+0000', 'last_change': '2024-09-10T15:53:17.533011+0000', 'last_active': '2024-09-10T15:53:17.533011+0000', 'last_peered': '2024-09-10T15:53:17.326494+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:12.445206+0000', 'last_became_peered': '2024-09-10T15:53:12.445206+0000', 'last_unstale': '2024-09-10T15:53:18.469520+0000', 'last_undegraded': '2024-09-10T15:53:18.469520+0000', 'last_fullsized': '2024-09-10T15:53:18.469520+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 3837, 'log_dups_size': 0, 'ondisk_log_size': 3837, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15695872, 'num_objects': 3832, 'num_object_clones': 0, 'num_object_copies': 7664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3832, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1811, 'num_write_kb': 7244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:23.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.3 is not active+clean 2024-09-10T15:53:23.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.3', 'version': "13'1", 'reported_seq': 102, 'reported_epoch': 40, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T15:53:12.427585+0000', 'last_change': '2024-09-10T15:53:12.427463+0000', 'last_active': '2024-09-10T15:53:12.427585+0000', 'last_peered': '2024-09-10T15:53:12.427585+0000', 'last_clean': '2024-09-10T15:53:12.427585+0000', 'last_became_active': '2024-09-10T15:53:12.427112+0000', 'last_became_peered': '2024-09-10T15:53:12.427112+0000', 'last_unstale': '2024-09-10T15:53:12.427585+0000', 'last_undegraded': '2024-09-10T15:53:12.427585+0000', 'last_fullsized': '2024-09-10T15:53:12.427585+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 12, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-11T16:50:07.176446+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000763951, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:23.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:53:23.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "41'3835", 'reported_seq': 3950, 'reported_epoch': 43, 'state': 'remapped+peering', 'last_fresh': '2024-09-10T15:53:18.460262+0000', 'last_change': '2024-09-10T15:53:17.533291+0000', 'last_active': '2024-09-10T15:53:17.533291+0000', 'last_peered': '2024-09-10T15:53:17.327041+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:12.427344+0000', 'last_became_peered': '2024-09-10T15:53:12.427344+0000', 'last_unstale': '2024-09-10T15:53:18.460262+0000', 'last_undegraded': '2024-09-10T15:53:18.460262+0000', 'last_fullsized': '2024-09-10T15:53:18.460262+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 3835, 'log_dups_size': 0, 'ondisk_log_size': 3835, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15683584, 'num_objects': 3829, 'num_object_clones': 0, 'num_object_copies': 7658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3829, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1704, 'num_write_kb': 6816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:23.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:53:23.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'3888", 'reported_seq': 4010, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:21.054072+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:53:21.054072+0000', 'last_peered': '2024-09-10T15:53:21.054072+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:53:21.054072+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 3888, 'log_dups_size': 0, 'ondisk_log_size': 3888, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15917056, 'num_objects': 3886, 'num_object_clones': 0, 'num_object_copies': 7772, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3886, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1080, 'num_write_kb': 4320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 53}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:23.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.4 is not active+clean 2024-09-10T15:53:23.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.4', 'version': "0'0", 'reported_seq': 90, 'reported_epoch': 40, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T15:53:12.429956+0000', 'last_change': '2024-09-10T15:53:12.429825+0000', 'last_active': '2024-09-10T15:53:12.429956+0000', 'last_peered': '2024-09-10T15:53:12.429956+0000', 'last_clean': '2024-09-10T15:53:12.429956+0000', 'last_became_active': '2024-09-10T15:53:12.428710+0000', 'last_became_peered': '2024-09-10T15:53:12.428710+0000', 'last_unstale': '2024-09-10T15:53:12.429956+0000', 'last_undegraded': '2024-09-10T15:53:12.429956+0000', 'last_fullsized': '2024-09-10T15:53:12.429956+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 12, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-11T18:48:45.544397+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000510747, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0], 'acting': [0, 2], 'avail_no_missing': ['0', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:23.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:53:23.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'3985", 'reported_seq': 4142, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:21.143387+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:53:21.143387+0000', 'last_peered': '2024-09-10T15:53:21.143387+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:53:21.143387+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 3985, 'log_dups_size': 0, 'ondisk_log_size': 3985, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16289792, 'num_objects': 3977, 'num_object_clones': 0, 'num_object_copies': 7954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3977, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 403, 'num_write_kb': 1612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 60}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:23.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:53:23.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'3929", 'reported_seq': 4063, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:21.123029+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:53:21.123029+0000', 'last_peered': '2024-09-10T15:53:21.123029+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:53:21.123029+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 3929, 'log_dups_size': 0, 'ondisk_log_size': 3929, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16076800, 'num_objects': 3925, 'num_object_clones': 0, 'num_object_copies': 7850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3925, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 535, 'num_write_kb': 2140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 48}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:23.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.6 is not active+clean 2024-09-10T15:53:23.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.6', 'version': "0'0", 'reported_seq': 90, 'reported_epoch': 40, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T15:53:12.429568+0000', 'last_change': '2024-09-10T15:53:12.429456+0000', 'last_active': '2024-09-10T15:53:12.429568+0000', 'last_peered': '2024-09-10T15:53:12.429568+0000', 'last_clean': '2024-09-10T15:53:12.429568+0000', 'last_became_active': '2024-09-10T15:53:12.428173+0000', 'last_became_peered': '2024-09-10T15:53:12.428173+0000', 'last_unstale': '2024-09-10T15:53:12.429568+0000', 'last_undegraded': '2024-09-10T15:53:12.429568+0000', 'last_fullsized': '2024-09-10T15:53:12.429568+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 12, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-11T17:42:42.683814+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000495975, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0], 'acting': [0, 2], 'avail_no_missing': ['0', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:23.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.7 is not active+clean 2024-09-10T15:53:23.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.7', 'version': "0'0", 'reported_seq': 97, 'reported_epoch': 40, 'state': 'active+clean+remapped', 'last_fresh': '2024-09-10T15:53:12.431101+0000', 'last_change': '2024-09-10T15:53:12.430966+0000', 'last_active': '2024-09-10T15:53:12.431101+0000', 'last_peered': '2024-09-10T15:53:12.431101+0000', 'last_clean': '2024-09-10T15:53:12.431101+0000', 'last_became_active': '2024-09-10T15:53:12.430496+0000', 'last_became_peered': '2024-09-10T15:53:12.430496+0000', 'last_unstale': '2024-09-10T15:53:12.431101+0000', 'last_undegraded': '2024-09-10T15:53:12.431101+0000', 'last_fullsized': '2024-09-10T15:53:12.431101+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 12, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:29.596349+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-11T19:27:04.534041+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.001001947, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:23.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:53:23.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'4069", 'reported_seq': 4202, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:21.108890+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:53:21.108890+0000', 'last_peered': '2024-09-10T15:53:21.108890+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:53:21.108890+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 4069, 'log_dups_size': 0, 'ondisk_log_size': 4069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16617955, 'num_objects': 4058, 'num_object_clones': 0, 'num_object_copies': 8116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4058, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 888, 'num_write_kb': 3552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 55}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:23.116 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:53:23.675 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:23.679 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31183/125536 objects degraded (24.840%), 8 pgs degraded","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":129,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+clean","count":7},{"state_name":"active+clean+remapped","count":5},{"state_name":"remapped+peering","count":4}],"num_pgs":24,"num_pools":2,"num_objects":62768,"data_bytes":257085942,"bytes_used":831127552,"bytes_avail":289079164928,"bytes_total":289910292480,"inactive_pgs_ratio":0.1666666716337204,"degraded_objects":31183,"degraded_total":125536,"degraded_ratio":0.24839886566403263,"misplaced_objects":1,"misplaced_total":125536,"misplaced_ratio":7.9658424674993632e-06,"write_bytes_sec":2096057,"read_op_per_sec":0,"write_op_per_sec":511},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-10T15:52:32.427210+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:53:23.713 INFO:tasks.rep_lost_unfound_delete.ceph_manager:no progress seen, keeping timeout for now 2024-09-10T15:53:23.714 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:23.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 46 20 64556 64536 5.47959 1.82422 0.011526 0.0142485 2024-09-10T15:53:24.265 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:24.268 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:24.291 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-10T15:53:22.442955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257085942,"num_objects":62768,"num_object_clones":0,"num_object_copies":125536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31183,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":62768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24370,"num_write_kb":97474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":8211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62850,"ondisk_log_size":62850,"up":43,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":811648,"kb_used_data":500176,"kb_used_omap":12,"kb_used_meta":311411,"kb_avail":282303872,"statfs":{"total":289910292480,"available":289079164928,"internally_reserved":0,"allocated":512180224,"data_stored":511335513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":318884869},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,9,4,0,0,4,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12541952,"num_objects":3062,"num_object_clones":0,"num_object_copies":6124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31183,"num_objects_misplaced":-44832,"num_objects_unfound":0,"num_objects_dirty":3062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3062,"num_write_kb":12248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.983592"},"pg_stats":[{"pgid":"2.f","version":"44'4034","reported_seq":4104,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.118934+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:21.118934+0000","last_peered":"2024-09-10T15:53:21.118934+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:21.118934+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4034,"log_dups_size":0,"ondisk_log_size":4034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16523264,"num_objects":4034,"num_object_clones":0,"num_object_copies":8068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4034,"num_write_kb":16136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":61},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'3966","reported_seq":4066,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.464296+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:18.464296+0000","last_peered":"2024-09-10T15:53:18.464296+0000","last_clean":"2024-09-10T15:53:18.464296+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:18.464296+0000","last_undegraded":"2024-09-10T15:53:18.464296+0000","last_fullsized":"2024-09-10T15:53:18.464296+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":3966,"log_dups_size":0,"ondisk_log_size":3966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16220160,"num_objects":3960,"num_object_clones":0,"num_object_copies":7920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2302,"num_write_kb":9208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'3806","reported_seq":3918,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.465374+0000","last_change":"2024-09-10T15:53:17.535147+0000","last_active":"2024-09-10T15:53:17.535147+0000","last_peered":"2024-09-10T15:53:17.329672+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:12.448159+0000","last_became_peered":"2024-09-10T15:53:12.448159+0000","last_unstale":"2024-09-10T15:53:18.465374+0000","last_undegraded":"2024-09-10T15:53:18.465374+0000","last_fullsized":"2024-09-10T15:53:18.465374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15572992,"num_objects":3802,"num_object_clones":0,"num_object_copies":7604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2074,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'3885","reported_seq":3957,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.463395+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:18.463395+0000","last_peered":"2024-09-10T15:53:18.463395+0000","last_clean":"2024-09-10T15:53:18.463395+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:18.463395+0000","last_undegraded":"2024-09-10T15:53:18.463395+0000","last_fullsized":"2024-09-10T15:53:18.463395+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":3885,"log_dups_size":0,"ondisk_log_size":3885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15912960,"num_objects":3885,"num_object_clones":0,"num_object_copies":7770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3885,"num_write_kb":15540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'3914","reported_seq":4034,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:21.069541+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:21.069541+0000","last_peered":"2024-09-10T15:53:21.069541+0000","last_clean":"2024-09-10T15:53:21.069541+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:21.069541+0000","last_undegraded":"2024-09-10T15:53:21.069541+0000","last_fullsized":"2024-09-10T15:53:21.069541+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":3914,"log_dups_size":0,"ondisk_log_size":3914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16003072,"num_objects":3907,"num_object_clones":0,"num_object_copies":7814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":985,"num_write_kb":3940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'3925","reported_seq":4052,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.155397+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:21.155397+0000","last_peered":"2024-09-10T15:53:21.155397+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:21.155397+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":3925,"log_dups_size":0,"ondisk_log_size":3925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16056320,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":643,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":52},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'3894","reported_seq":4029,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.124600+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:21.124600+0000","last_peered":"2024-09-10T15:53:21.124600+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:21.124600+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":3894,"log_dups_size":0,"ondisk_log_size":3894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":3888,"num_object_clones":0,"num_object_copies":7776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":544,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":54},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'3972","reported_seq":4098,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:21.098719+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:21.098719+0000","last_peered":"2024-09-10T15:53:21.098719+0000","last_clean":"2024-09-10T15:53:21.098719+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:21.098719+0000","last_undegraded":"2024-09-10T15:53:21.098719+0000","last_fullsized":"2024-09-10T15:53:21.098719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":3972,"log_dups_size":0,"ondisk_log_size":3972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16261120,"num_objects":3970,"num_object_clones":0,"num_object_copies":7940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":731,"num_write_kb":2924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.437074+0000","last_change":"2024-09-10T15:53:12.429585+0000","last_active":"2024-09-10T15:53:12.437074+0000","last_peered":"2024-09-10T15:53:12.437074+0000","last_clean":"2024-09-10T15:53:12.437074+0000","last_became_active":"2024-09-10T15:53:12.428614+0000","last_became_peered":"2024-09-10T15:53:12.428614+0000","last_unstale":"2024-09-10T15:53:12.437074+0000","last_undegraded":"2024-09-10T15:53:12.437074+0000","last_fullsized":"2024-09-10T15:53:12.437074+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:39.542044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"41'3999","reported_seq":4116,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.467481+0000","last_change":"2024-09-10T15:53:17.534792+0000","last_active":"2024-09-10T15:53:17.534792+0000","last_peered":"2024-09-10T15:53:17.326712+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:12.430138+0000","last_became_peered":"2024-09-10T15:53:12.430138+0000","last_unstale":"2024-09-10T15:53:18.467481+0000","last_undegraded":"2024-09-10T15:53:18.467481+0000","last_fullsized":"2024-09-10T15:53:18.467481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":3999,"log_dups_size":0,"ondisk_log_size":3999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16351232,"num_objects":3992,"num_object_clones":0,"num_object_copies":7984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1976,"num_write_kb":7904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.465387+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:18.465387+0000","last_peered":"2024-09-10T15:53:18.465387+0000","last_clean":"2024-09-10T15:53:18.465387+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:18.465387+0000","last_undegraded":"2024-09-10T15:53:18.465387+0000","last_fullsized":"2024-09-10T15:53:18.465387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'3909","reported_seq":4030,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.148367+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:21.148367+0000","last_peered":"2024-09-10T15:53:21.148367+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:21.148367+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":3909,"log_dups_size":0,"ondisk_log_size":3909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":773,"num_write_kb":3092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":38},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'3837","reported_seq":3950,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.469520+0000","last_change":"2024-09-10T15:53:17.533011+0000","last_active":"2024-09-10T15:53:17.533011+0000","last_peered":"2024-09-10T15:53:17.326494+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:12.445206+0000","last_became_peered":"2024-09-10T15:53:12.445206+0000","last_unstale":"2024-09-10T15:53:18.469520+0000","last_undegraded":"2024-09-10T15:53:18.469520+0000","last_fullsized":"2024-09-10T15:53:18.469520+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":3837,"log_dups_size":0,"ondisk_log_size":3837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15695872,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1811,"num_write_kb":7244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.427585+0000","last_change":"2024-09-10T15:53:12.427463+0000","last_active":"2024-09-10T15:53:12.427585+0000","last_peered":"2024-09-10T15:53:12.427585+0000","last_clean":"2024-09-10T15:53:12.427585+0000","last_became_active":"2024-09-10T15:53:12.427112+0000","last_became_peered":"2024-09-10T15:53:12.427112+0000","last_unstale":"2024-09-10T15:53:12.427585+0000","last_undegraded":"2024-09-10T15:53:12.427585+0000","last_fullsized":"2024-09-10T15:53:12.427585+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:07.176446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"41'3835","reported_seq":3950,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-10T15:53:18.460262+0000","last_change":"2024-09-10T15:53:17.533291+0000","last_active":"2024-09-10T15:53:17.533291+0000","last_peered":"2024-09-10T15:53:17.327041+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:12.427344+0000","last_became_peered":"2024-09-10T15:53:12.427344+0000","last_unstale":"2024-09-10T15:53:18.460262+0000","last_undegraded":"2024-09-10T15:53:18.460262+0000","last_fullsized":"2024-09-10T15:53:18.460262+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15683584,"num_objects":3829,"num_object_clones":0,"num_object_copies":7658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1704,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'3888","reported_seq":4010,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.054072+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:21.054072+0000","last_peered":"2024-09-10T15:53:21.054072+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:21.054072+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":3888,"log_dups_size":0,"ondisk_log_size":3888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15917056,"num_objects":3886,"num_object_clones":0,"num_object_copies":7772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1080,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":53},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":90,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.429956+0000","last_change":"2024-09-10T15:53:12.429825+0000","last_active":"2024-09-10T15:53:12.429956+0000","last_peered":"2024-09-10T15:53:12.429956+0000","last_clean":"2024-09-10T15:53:12.429956+0000","last_became_active":"2024-09-10T15:53:12.428710+0000","last_became_peered":"2024-09-10T15:53:12.428710+0000","last_unstale":"2024-09-10T15:53:12.429956+0000","last_undegraded":"2024-09-10T15:53:12.429956+0000","last_fullsized":"2024-09-10T15:53:12.429956+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:45.544397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"44'3985","reported_seq":4142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.143387+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:21.143387+0000","last_peered":"2024-09-10T15:53:21.143387+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:21.143387+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":3985,"log_dups_size":0,"ondisk_log_size":3985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16289792,"num_objects":3977,"num_object_clones":0,"num_object_copies":7954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":403,"num_write_kb":1612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":60},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T15:53:18.464266+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:18.464266+0000","last_peered":"2024-09-10T15:53:18.464266+0000","last_clean":"2024-09-10T15:53:18.464266+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:18.464266+0000","last_undegraded":"2024-09-10T15:53:18.464266+0000","last_fullsized":"2024-09-10T15:53:18.464266+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'3929","reported_seq":4063,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.123029+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:21.123029+0000","last_peered":"2024-09-10T15:53:21.123029+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:21.123029+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16076800,"num_objects":3925,"num_object_clones":0,"num_object_copies":7850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":535,"num_write_kb":2140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":48},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.429568+0000","last_change":"2024-09-10T15:53:12.429456+0000","last_active":"2024-09-10T15:53:12.429568+0000","last_peered":"2024-09-10T15:53:12.429568+0000","last_clean":"2024-09-10T15:53:12.429568+0000","last_became_active":"2024-09-10T15:53:12.428173+0000","last_became_peered":"2024-09-10T15:53:12.428173+0000","last_unstale":"2024-09-10T15:53:12.429568+0000","last_undegraded":"2024-09-10T15:53:12.429568+0000","last_fullsized":"2024-09-10T15:53:12.429568+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:42.683814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":40,"state":"active+clean+remapped","last_fresh":"2024-09-10T15:53:12.431101+0000","last_change":"2024-09-10T15:53:12.430966+0000","last_active":"2024-09-10T15:53:12.431101+0000","last_peered":"2024-09-10T15:53:12.431101+0000","last_clean":"2024-09-10T15:53:12.431101+0000","last_became_active":"2024-09-10T15:53:12.430496+0000","last_became_peered":"2024-09-10T15:53:12.430496+0000","last_unstale":"2024-09-10T15:53:12.431101+0000","last_undegraded":"2024-09-10T15:53:12.431101+0000","last_fullsized":"2024-09-10T15:53:12.431101+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:04.534041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"44'4069","reported_seq":4202,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:21.108890+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:21.108890+0000","last_peered":"2024-09-10T15:53:21.108890+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:21.108890+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4069,"log_dups_size":0,"ondisk_log_size":4069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16617955,"num_objects":4058,"num_object_clones":0,"num_object_copies":8116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":55},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":257085923,"num_objects":62766,"num_object_clones":0,"num_object_copies":125532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24368,"num_write_kb":97472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512561152,"data_stored":512463601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62847,"ondisk_log_size":62847,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":11,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626434,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":462848,"data_stored":124286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":28110510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149718,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394496,"kb_used_data":250416,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93977344,"statfs":{"total":96636764160,"available":96232800256,"internally_reserved":0,"allocated":256425984,"data_stored":256170326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":147517101},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,4,0,9],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953487,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":389228,"kb_used_data":249308,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93982612,"statfs":{"total":96636764160,"available":96238194688,"internally_reserved":0,"allocated":255291392,"data_stored":255040901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,9,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257531904,"data_stored":257528291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":12075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":254926848,"data_stored":254923235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:24.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 47 20 65218 65198 5.41802 2.58594 0.018769 0.0144113 2024-09-10T15:53:25.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 48 20 65799 65779 5.35243 2.26953 0.0150297 0.0145847 2024-09-10T15:53:26.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 49 20 66491 66471 5.29836 2.70312 0.0160297 0.0147364 2024-09-10T15:53:27.293 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:27.736 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:27.736 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:27.758 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-10T15:53:26.443830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":268710390,"num_objects":65606,"num_object_clones":0,"num_object_copies":131212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3881,"num_objects_unfound":0,"num_objects_dirty":65606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27208,"num_write_kb":108826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":454675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65688,"ondisk_log_size":65688,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":859632,"kb_used_data":517632,"kb_used_omap":12,"kb_used_meta":341939,"kb_avail":282255888,"statfs":{"total":289910292480,"available":289030029312,"internally_reserved":0,"allocated":530055168,"data_stored":529208708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":350145541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,12,4,11,20,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24166400,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":-40952,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5900,"num_write_kb":23600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":446464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004357"},"pg_stats":[{"pgid":"2.f","version":"44'4224","reported_seq":4295,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.156858+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:26.156858+0000","last_peered":"2024-09-10T15:53:26.156858+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:26.156858+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4224,"log_dups_size":0,"ondisk_log_size":4224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17301504,"num_objects":4224,"num_object_clones":0,"num_object_copies":8448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4224,"num_write_kb":16896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":251},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4140","reported_seq":4242,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:24.195481+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:24.195481+0000","last_peered":"2024-09-10T15:53:24.195481+0000","last_clean":"2024-09-10T15:53:24.195481+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:24.195481+0000","last_undegraded":"2024-09-10T15:53:24.195481+0000","last_fullsized":"2024-09-10T15:53:24.195481+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4140,"log_dups_size":0,"ondisk_log_size":4140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16932864,"num_objects":4134,"num_object_clones":0,"num_object_copies":8268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2476,"num_write_kb":9904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'3980","reported_seq":4098,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:24.184666+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:24.184666+0000","last_peered":"2024-09-10T15:53:24.184666+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:24.184666+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":3980,"log_dups_size":0,"ondisk_log_size":3980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16285696,"num_objects":3976,"num_object_clones":0,"num_object_copies":7952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2248,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4042","reported_seq":4116,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:24.224155+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:24.224155+0000","last_peered":"2024-09-10T15:53:24.224155+0000","last_clean":"2024-09-10T15:53:24.224155+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:24.224155+0000","last_undegraded":"2024-09-10T15:53:24.224155+0000","last_fullsized":"2024-09-10T15:53:24.224155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4042,"log_dups_size":0,"ondisk_log_size":4042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16556032,"num_objects":4042,"num_object_clones":0,"num_object_copies":8084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4042,"num_write_kb":16168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4074","reported_seq":4195,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:26.127254+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:26.127254+0000","last_peered":"2024-09-10T15:53:26.127254+0000","last_clean":"2024-09-10T15:53:26.127254+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:26.127254+0000","last_undegraded":"2024-09-10T15:53:26.127254+0000","last_fullsized":"2024-09-10T15:53:26.127254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4074,"log_dups_size":0,"ondisk_log_size":4074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16658432,"num_objects":4067,"num_object_clones":0,"num_object_copies":8134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1145,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4113","reported_seq":4240,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.173284+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:26.173284+0000","last_peered":"2024-09-10T15:53:26.173284+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:26.173284+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4113,"log_dups_size":0,"ondisk_log_size":4113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16826368,"num_objects":4108,"num_object_clones":0,"num_object_copies":8216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":831,"num_write_kb":3324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":240},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4075","reported_seq":4210,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.166765+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:26.166765+0000","last_peered":"2024-09-10T15:53:26.166765+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:26.166765+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4075,"log_dups_size":0,"ondisk_log_size":4075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16666624,"num_objects":4069,"num_object_clones":0,"num_object_copies":8138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":725,"num_write_kb":2900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":235},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4170","reported_seq":4296,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:26.130381+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:26.130381+0000","last_peered":"2024-09-10T15:53:26.130381+0000","last_clean":"2024-09-10T15:53:26.130381+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:26.130381+0000","last_undegraded":"2024-09-10T15:53:26.130381+0000","last_fullsized":"2024-09-10T15:53:26.130381+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4170,"log_dups_size":0,"ondisk_log_size":4170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17072128,"num_objects":4168,"num_object_clones":0,"num_object_copies":8336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":929,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4168","reported_seq":4510,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:24.199941+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:24.199941+0000","last_peered":"2024-09-10T15:53:24.199941+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:24.199941+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4168,"log_dups_size":0,"ondisk_log_size":4168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17043456,"num_objects":4161,"num_object_clones":0,"num_object_copies":8322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3881,"num_objects_unfound":0,"num_objects_dirty":4161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2145,"num_write_kb":8580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":446464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3881},{"shards":"2","objects":278}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4110","reported_seq":4231,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.137243+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:26.137243+0000","last_peered":"2024-09-10T15:53:26.137243+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:26.137243+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4110,"log_dups_size":0,"ondisk_log_size":4110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16801792,"num_objects":4102,"num_object_clones":0,"num_object_copies":8204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":974,"num_write_kb":3896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":239},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'3958","reported_seq":4077,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:24.115905+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:24.115905+0000","last_peered":"2024-09-10T15:53:24.115905+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:24.115905+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":3958,"log_dups_size":0,"ondisk_log_size":3958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16191488,"num_objects":3953,"num_object_clones":0,"num_object_copies":7906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1932,"num_write_kb":7728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":121}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'3984","reported_seq":4105,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:24.182419+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:24.182419+0000","last_peered":"2024-09-10T15:53:24.182419+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:24.182419+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":3984,"log_dups_size":0,"ondisk_log_size":3984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16293888,"num_objects":3978,"num_object_clones":0,"num_object_copies":7956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1853,"num_write_kb":7412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":149}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4086","reported_seq":4208,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.178213+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:26.178213+0000","last_peered":"2024-09-10T15:53:26.178213+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:26.178213+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4086,"log_dups_size":0,"ondisk_log_size":4086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16728064,"num_objects":4084,"num_object_clones":0,"num_object_copies":8168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1278,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":251},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4183","reported_seq":4340,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.165104+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:26.165104+0000","last_peered":"2024-09-10T15:53:26.165104+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:26.165104+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4183,"log_dups_size":0,"ondisk_log_size":4183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":601,"num_write_kb":2404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":258},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4123","reported_seq":4257,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.071571+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:26.071571+0000","last_peered":"2024-09-10T15:53:26.071571+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:26.071571+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4123,"log_dups_size":0,"ondisk_log_size":4123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16871424,"num_objects":4119,"num_object_clones":0,"num_object_copies":8238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":729,"num_write_kb":2916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":242},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4255","reported_seq":4388,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.125720+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:26.125720+0000","last_peered":"2024-09-10T15:53:26.125720+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:26.125720+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4255,"log_dups_size":0,"ondisk_log_size":4255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17379811,"num_objects":4244,"num_object_clones":0,"num_object_copies":8488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1074,"num_write_kb":4296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":241},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":268710371,"num_objects":65604,"num_object_clones":0,"num_object_copies":131208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3881,"num_objects_unfound":0,"num_objects_dirty":65604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27206,"num_write_kb":108824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":454656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532422656,"data_stored":532332331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65685,"ondisk_log_size":65685,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626435,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54044,"kb_used_data":4364,"kb_used_omap":4,"kb_used_meta":49659,"kb_avail":94317796,"statfs":{"total":96636764160,"available":96581423104,"internally_reserved":0,"allocated":4468736,"data_stored":4133592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50851502},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,9,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149719,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407588,"kb_used_data":259348,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93964252,"statfs":{"total":96636764160,"available":96219394048,"internally_reserved":0,"allocated":265572352,"data_stored":265316694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":151776941},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398000,"kb_used_data":253920,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93973840,"statfs":{"total":96636764160,"available":96229212160,"internally_reserved":0,"allocated":260014080,"data_stored":259758422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,11,0,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":266850304,"data_stored":266846691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5464064,"data_stored":5380965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260108288,"data_stored":260104675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:27.759 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:27.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 50 20 67077 67057 5.23817 2.28906 0.0196169 0.0149038 2024-09-10T15:53:28.307 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:28.308 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:28.334 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-10T15:53:26.443830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":268710390,"num_objects":65606,"num_object_clones":0,"num_object_copies":131212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3881,"num_objects_unfound":0,"num_objects_dirty":65606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27208,"num_write_kb":108826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":454675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65688,"ondisk_log_size":65688,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":859632,"kb_used_data":517632,"kb_used_omap":12,"kb_used_meta":341939,"kb_avail":282255888,"statfs":{"total":289910292480,"available":289030029312,"internally_reserved":0,"allocated":530055168,"data_stored":529208708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":350145541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,12,4,11,20,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24166400,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":-40952,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5900,"num_write_kb":23600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":446464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004357"},"pg_stats":[{"pgid":"2.f","version":"44'4224","reported_seq":4295,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.156858+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:26.156858+0000","last_peered":"2024-09-10T15:53:26.156858+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:26.156858+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4224,"log_dups_size":0,"ondisk_log_size":4224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17301504,"num_objects":4224,"num_object_clones":0,"num_object_copies":8448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4224,"num_write_kb":16896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":251},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4140","reported_seq":4242,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:24.195481+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:24.195481+0000","last_peered":"2024-09-10T15:53:24.195481+0000","last_clean":"2024-09-10T15:53:24.195481+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:24.195481+0000","last_undegraded":"2024-09-10T15:53:24.195481+0000","last_fullsized":"2024-09-10T15:53:24.195481+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4140,"log_dups_size":0,"ondisk_log_size":4140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16932864,"num_objects":4134,"num_object_clones":0,"num_object_copies":8268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2476,"num_write_kb":9904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'3980","reported_seq":4098,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:24.184666+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:24.184666+0000","last_peered":"2024-09-10T15:53:24.184666+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:24.184666+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":3980,"log_dups_size":0,"ondisk_log_size":3980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16285696,"num_objects":3976,"num_object_clones":0,"num_object_copies":7952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2248,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4042","reported_seq":4116,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:24.224155+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:24.224155+0000","last_peered":"2024-09-10T15:53:24.224155+0000","last_clean":"2024-09-10T15:53:24.224155+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:24.224155+0000","last_undegraded":"2024-09-10T15:53:24.224155+0000","last_fullsized":"2024-09-10T15:53:24.224155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4042,"log_dups_size":0,"ondisk_log_size":4042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16556032,"num_objects":4042,"num_object_clones":0,"num_object_copies":8084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4042,"num_write_kb":16168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4074","reported_seq":4195,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:26.127254+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:26.127254+0000","last_peered":"2024-09-10T15:53:26.127254+0000","last_clean":"2024-09-10T15:53:26.127254+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:26.127254+0000","last_undegraded":"2024-09-10T15:53:26.127254+0000","last_fullsized":"2024-09-10T15:53:26.127254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4074,"log_dups_size":0,"ondisk_log_size":4074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16658432,"num_objects":4067,"num_object_clones":0,"num_object_copies":8134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1145,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4113","reported_seq":4240,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.173284+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:26.173284+0000","last_peered":"2024-09-10T15:53:26.173284+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:26.173284+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4113,"log_dups_size":0,"ondisk_log_size":4113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16826368,"num_objects":4108,"num_object_clones":0,"num_object_copies":8216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":831,"num_write_kb":3324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":240},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4075","reported_seq":4210,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.166765+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:26.166765+0000","last_peered":"2024-09-10T15:53:26.166765+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:26.166765+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4075,"log_dups_size":0,"ondisk_log_size":4075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16666624,"num_objects":4069,"num_object_clones":0,"num_object_copies":8138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":725,"num_write_kb":2900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":235},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4170","reported_seq":4296,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:26.130381+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:26.130381+0000","last_peered":"2024-09-10T15:53:26.130381+0000","last_clean":"2024-09-10T15:53:26.130381+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:26.130381+0000","last_undegraded":"2024-09-10T15:53:26.130381+0000","last_fullsized":"2024-09-10T15:53:26.130381+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4170,"log_dups_size":0,"ondisk_log_size":4170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17072128,"num_objects":4168,"num_object_clones":0,"num_object_copies":8336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":929,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4168","reported_seq":4510,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:24.199941+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:24.199941+0000","last_peered":"2024-09-10T15:53:24.199941+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:24.199941+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4168,"log_dups_size":0,"ondisk_log_size":4168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17043456,"num_objects":4161,"num_object_clones":0,"num_object_copies":8322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3881,"num_objects_unfound":0,"num_objects_dirty":4161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2145,"num_write_kb":8580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":446464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3881},{"shards":"2","objects":278}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4110","reported_seq":4231,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.137243+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:26.137243+0000","last_peered":"2024-09-10T15:53:26.137243+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:26.137243+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4110,"log_dups_size":0,"ondisk_log_size":4110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16801792,"num_objects":4102,"num_object_clones":0,"num_object_copies":8204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":974,"num_write_kb":3896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":239},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'3958","reported_seq":4077,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:24.115905+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:24.115905+0000","last_peered":"2024-09-10T15:53:24.115905+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:24.115905+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":3958,"log_dups_size":0,"ondisk_log_size":3958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16191488,"num_objects":3953,"num_object_clones":0,"num_object_copies":7906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1932,"num_write_kb":7728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":121}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'3984","reported_seq":4105,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:24.182419+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:24.182419+0000","last_peered":"2024-09-10T15:53:24.182419+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:24.182419+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":3984,"log_dups_size":0,"ondisk_log_size":3984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16293888,"num_objects":3978,"num_object_clones":0,"num_object_copies":7956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1853,"num_write_kb":7412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":149}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4086","reported_seq":4208,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.178213+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:26.178213+0000","last_peered":"2024-09-10T15:53:26.178213+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:26.178213+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4086,"log_dups_size":0,"ondisk_log_size":4086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16728064,"num_objects":4084,"num_object_clones":0,"num_object_copies":8168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1278,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":251},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4183","reported_seq":4340,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.165104+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:26.165104+0000","last_peered":"2024-09-10T15:53:26.165104+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:26.165104+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4183,"log_dups_size":0,"ondisk_log_size":4183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":601,"num_write_kb":2404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":258},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4123","reported_seq":4257,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.071571+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:26.071571+0000","last_peered":"2024-09-10T15:53:26.071571+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:26.071571+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4123,"log_dups_size":0,"ondisk_log_size":4123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16871424,"num_objects":4119,"num_object_clones":0,"num_object_copies":8238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":729,"num_write_kb":2916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":242},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4255","reported_seq":4388,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.125720+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:26.125720+0000","last_peered":"2024-09-10T15:53:26.125720+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:26.125720+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4255,"log_dups_size":0,"ondisk_log_size":4255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17379811,"num_objects":4244,"num_object_clones":0,"num_object_copies":8488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1074,"num_write_kb":4296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":241},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":268710371,"num_objects":65604,"num_object_clones":0,"num_object_copies":131208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3881,"num_objects_unfound":0,"num_objects_dirty":65604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27206,"num_write_kb":108824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":454656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532422656,"data_stored":532332331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65685,"ondisk_log_size":65685,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626435,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54044,"kb_used_data":4364,"kb_used_omap":4,"kb_used_meta":49659,"kb_avail":94317796,"statfs":{"total":96636764160,"available":96581423104,"internally_reserved":0,"allocated":4468736,"data_stored":4133592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50851502},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,9,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149719,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407588,"kb_used_data":259348,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93964252,"statfs":{"total":96636764160,"available":96219394048,"internally_reserved":0,"allocated":265572352,"data_stored":265316694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":151776941},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398000,"kb_used_data":253920,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93973840,"statfs":{"total":96636764160,"available":96229212160,"internally_reserved":0,"allocated":260014080,"data_stored":259758422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,11,0,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":266850304,"data_stored":266846691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5464064,"data_stored":5380965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260108288,"data_stored":260104675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:28.335 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:53:28.335 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'4224", 'reported_seq': 4295, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:26.156858+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:53:26.156858+0000', 'last_peered': '2024-09-10T15:53:26.156858+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:53:26.156858+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 4224, 'log_dups_size': 0, 'ondisk_log_size': 4224, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17301504, 'num_objects': 4224, 'num_object_clones': 0, 'num_object_copies': 8448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4224, 'num_write_kb': 16896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 251}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:28.335 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:53:28.335 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'3980", 'reported_seq': 4098, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:24.184666+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:53:24.184666+0000', 'last_peered': '2024-09-10T15:53:24.184666+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:53:24.184666+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 3980, 'log_dups_size': 0, 'ondisk_log_size': 3980, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16285696, 'num_objects': 3976, 'num_object_clones': 0, 'num_object_copies': 7952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3976, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2248, 'num_write_kb': 8992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:28.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:53:28.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'4113", 'reported_seq': 4240, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:26.173284+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:53:26.173284+0000', 'last_peered': '2024-09-10T15:53:26.173284+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:53:26.173284+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 4113, 'log_dups_size': 0, 'ondisk_log_size': 4113, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16826368, 'num_objects': 4108, 'num_object_clones': 0, 'num_object_copies': 8216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4108, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 831, 'num_write_kb': 3324, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 240}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:28.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:53:28.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'4075", 'reported_seq': 4210, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:26.166765+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:53:26.166765+0000', 'last_peered': '2024-09-10T15:53:26.166765+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:53:26.166765+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 4075, 'log_dups_size': 0, 'ondisk_log_size': 4075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16666624, 'num_objects': 4069, 'num_object_clones': 0, 'num_object_copies': 8138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4069, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 725, 'num_write_kb': 2900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 235}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:28.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:53:28.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'4168", 'reported_seq': 4510, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:24.199941+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:53:24.199941+0000', 'last_peered': '2024-09-10T15:53:24.199941+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:53:24.199941+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 4168, 'log_dups_size': 0, 'ondisk_log_size': 4168, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17043456, 'num_objects': 4161, 'num_object_clones': 0, 'num_object_copies': 8322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3881, 'num_objects_unfound': 0, 'num_objects_dirty': 4161, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2145, 'num_write_kb': 8580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 109, 'num_bytes_recovered': 446464, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3881}, {'shards': '2', 'objects': 278}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:28.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:53:28.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'4110", 'reported_seq': 4231, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:26.137243+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:53:26.137243+0000', 'last_peered': '2024-09-10T15:53:26.137243+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:53:26.137243+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 4110, 'log_dups_size': 0, 'ondisk_log_size': 4110, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16801792, 'num_objects': 4102, 'num_object_clones': 0, 'num_object_copies': 8204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4102, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 974, 'num_write_kb': 3896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 239}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:28.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:53:28.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'3958", 'reported_seq': 4077, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:24.115905+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:53:24.115905+0000', 'last_peered': '2024-09-10T15:53:24.115905+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:53:24.115905+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 3958, 'log_dups_size': 0, 'ondisk_log_size': 3958, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16191488, 'num_objects': 3953, 'num_object_clones': 0, 'num_object_copies': 7906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3953, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1932, 'num_write_kb': 7728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 121}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:28.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:53:28.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'3984", 'reported_seq': 4105, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:24.182419+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:53:24.182419+0000', 'last_peered': '2024-09-10T15:53:24.182419+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:53:24.182419+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 3984, 'log_dups_size': 0, 'ondisk_log_size': 3984, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16293888, 'num_objects': 3978, 'num_object_clones': 0, 'num_object_copies': 7956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3978, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1853, 'num_write_kb': 7412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 149}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:28.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:53:28.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'4086", 'reported_seq': 4208, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:26.178213+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:53:26.178213+0000', 'last_peered': '2024-09-10T15:53:26.178213+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:53:26.178213+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 4086, 'log_dups_size': 0, 'ondisk_log_size': 4086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16728064, 'num_objects': 4084, 'num_object_clones': 0, 'num_object_copies': 8168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4084, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1278, 'num_write_kb': 5112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 251}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:28.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:53:28.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'4183", 'reported_seq': 4340, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:26.165104+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:53:26.165104+0000', 'last_peered': '2024-09-10T15:53:26.165104+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:53:26.165104+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 4183, 'log_dups_size': 0, 'ondisk_log_size': 4183, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17100800, 'num_objects': 4175, 'num_object_clones': 0, 'num_object_copies': 8350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4175, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 601, 'num_write_kb': 2404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 258}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:28.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:53:28.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'4123", 'reported_seq': 4257, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:26.071571+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:53:26.071571+0000', 'last_peered': '2024-09-10T15:53:26.071571+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:53:26.071571+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 4123, 'log_dups_size': 0, 'ondisk_log_size': 4123, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16871424, 'num_objects': 4119, 'num_object_clones': 0, 'num_object_copies': 8238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4119, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 729, 'num_write_kb': 2916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 242}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:28.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:53:28.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'4255", 'reported_seq': 4388, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:26.125720+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:53:26.125720+0000', 'last_peered': '2024-09-10T15:53:26.125720+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:53:26.125720+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 4255, 'log_dups_size': 0, 'ondisk_log_size': 4255, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17379811, 'num_objects': 4244, 'num_object_clones': 0, 'num_object_copies': 8488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4244, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1074, 'num_write_kb': 4296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 241}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:28.338 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:53:28.802 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 51 20 67697 67677 5.18295 2.42188 0.0415997 0.0150649 2024-09-10T15:53:29.056 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:29.056 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/131212 objects degraded (32.507%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":134,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":65606,"data_bytes":268710390,"bytes_used":898945024,"bytes_avail":289011347456,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":131212,"degraded_ratio":0.32506935341279763,"misplaced_objects":3881,"misplaced_total":131212,"misplaced_ratio":0.029578087370057616,"recovering_objects_per_sec":10,"recovering_bytes_per_sec":44708,"recovering_keys_per_sec":0,"num_objects_recovered":109,"num_bytes_recovered":446464,"num_keys_recovered":0,"write_bytes_sec":2419979,"read_op_per_sec":0,"write_op_per_sec":590},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-10T15:53:24.443468+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:53:29.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:53:29.084 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:29.578 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:29.579 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:29.590 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-10T15:53:28.444289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":268710390,"num_objects":65606,"num_object_clones":0,"num_object_copies":131212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3881,"num_objects_unfound":0,"num_objects_dirty":65606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27208,"num_write_kb":108826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":454675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65688,"ondisk_log_size":65688,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":877876,"kb_used_data":527556,"kb_used_omap":12,"kb_used_meta":350259,"kb_avail":282237644,"statfs":{"total":289910292480,"available":289011347456,"internally_reserved":0,"allocated":540217344,"data_stored":539370884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":358665221},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,11,6,8,12,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24166400,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":-40952,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5900,"num_write_kb":23600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":446464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.986201"},"pg_stats":[{"pgid":"2.f","version":"44'4224","reported_seq":4295,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.156858+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:26.156858+0000","last_peered":"2024-09-10T15:53:26.156858+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:26.156858+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4224,"log_dups_size":0,"ondisk_log_size":4224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17301504,"num_objects":4224,"num_object_clones":0,"num_object_copies":8448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4224,"num_write_kb":16896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":251},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4140","reported_seq":4242,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:24.195481+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:24.195481+0000","last_peered":"2024-09-10T15:53:24.195481+0000","last_clean":"2024-09-10T15:53:24.195481+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:24.195481+0000","last_undegraded":"2024-09-10T15:53:24.195481+0000","last_fullsized":"2024-09-10T15:53:24.195481+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4140,"log_dups_size":0,"ondisk_log_size":4140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16932864,"num_objects":4134,"num_object_clones":0,"num_object_copies":8268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2476,"num_write_kb":9904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'3980","reported_seq":4098,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:24.184666+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:24.184666+0000","last_peered":"2024-09-10T15:53:24.184666+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:24.184666+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":3980,"log_dups_size":0,"ondisk_log_size":3980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16285696,"num_objects":3976,"num_object_clones":0,"num_object_copies":7952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2248,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4042","reported_seq":4116,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:24.224155+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:24.224155+0000","last_peered":"2024-09-10T15:53:24.224155+0000","last_clean":"2024-09-10T15:53:24.224155+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:24.224155+0000","last_undegraded":"2024-09-10T15:53:24.224155+0000","last_fullsized":"2024-09-10T15:53:24.224155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4042,"log_dups_size":0,"ondisk_log_size":4042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16556032,"num_objects":4042,"num_object_clones":0,"num_object_copies":8084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4042,"num_write_kb":16168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4074","reported_seq":4195,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:26.127254+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:26.127254+0000","last_peered":"2024-09-10T15:53:26.127254+0000","last_clean":"2024-09-10T15:53:26.127254+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:26.127254+0000","last_undegraded":"2024-09-10T15:53:26.127254+0000","last_fullsized":"2024-09-10T15:53:26.127254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4074,"log_dups_size":0,"ondisk_log_size":4074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16658432,"num_objects":4067,"num_object_clones":0,"num_object_copies":8134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1145,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4113","reported_seq":4240,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.173284+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:26.173284+0000","last_peered":"2024-09-10T15:53:26.173284+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:26.173284+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4113,"log_dups_size":0,"ondisk_log_size":4113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16826368,"num_objects":4108,"num_object_clones":0,"num_object_copies":8216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":831,"num_write_kb":3324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":240},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4075","reported_seq":4210,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.166765+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:26.166765+0000","last_peered":"2024-09-10T15:53:26.166765+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:26.166765+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4075,"log_dups_size":0,"ondisk_log_size":4075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16666624,"num_objects":4069,"num_object_clones":0,"num_object_copies":8138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":725,"num_write_kb":2900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":235},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4170","reported_seq":4296,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:26.130381+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:26.130381+0000","last_peered":"2024-09-10T15:53:26.130381+0000","last_clean":"2024-09-10T15:53:26.130381+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:26.130381+0000","last_undegraded":"2024-09-10T15:53:26.130381+0000","last_fullsized":"2024-09-10T15:53:26.130381+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4170,"log_dups_size":0,"ondisk_log_size":4170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17072128,"num_objects":4168,"num_object_clones":0,"num_object_copies":8336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":929,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4168","reported_seq":4510,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:24.199941+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:24.199941+0000","last_peered":"2024-09-10T15:53:24.199941+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:24.199941+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4168,"log_dups_size":0,"ondisk_log_size":4168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17043456,"num_objects":4161,"num_object_clones":0,"num_object_copies":8322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3881,"num_objects_unfound":0,"num_objects_dirty":4161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2145,"num_write_kb":8580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":446464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3881},{"shards":"2","objects":278}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4110","reported_seq":4231,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.137243+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:26.137243+0000","last_peered":"2024-09-10T15:53:26.137243+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:26.137243+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4110,"log_dups_size":0,"ondisk_log_size":4110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16801792,"num_objects":4102,"num_object_clones":0,"num_object_copies":8204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":974,"num_write_kb":3896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":239},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'3958","reported_seq":4077,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:24.115905+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:24.115905+0000","last_peered":"2024-09-10T15:53:24.115905+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:24.115905+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":3958,"log_dups_size":0,"ondisk_log_size":3958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16191488,"num_objects":3953,"num_object_clones":0,"num_object_copies":7906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1932,"num_write_kb":7728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":121}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'3984","reported_seq":4105,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:24.182419+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:24.182419+0000","last_peered":"2024-09-10T15:53:24.182419+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:24.182419+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":3984,"log_dups_size":0,"ondisk_log_size":3984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16293888,"num_objects":3978,"num_object_clones":0,"num_object_copies":7956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1853,"num_write_kb":7412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":149}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4086","reported_seq":4208,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.178213+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:26.178213+0000","last_peered":"2024-09-10T15:53:26.178213+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:26.178213+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4086,"log_dups_size":0,"ondisk_log_size":4086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16728064,"num_objects":4084,"num_object_clones":0,"num_object_copies":8168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1278,"num_write_kb":5112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":251},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4183","reported_seq":4340,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.165104+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:26.165104+0000","last_peered":"2024-09-10T15:53:26.165104+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:26.165104+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4183,"log_dups_size":0,"ondisk_log_size":4183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":601,"num_write_kb":2404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":258},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4123","reported_seq":4257,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.071571+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:26.071571+0000","last_peered":"2024-09-10T15:53:26.071571+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:26.071571+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4123,"log_dups_size":0,"ondisk_log_size":4123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16871424,"num_objects":4119,"num_object_clones":0,"num_object_copies":8238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":729,"num_write_kb":2916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":242},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4255","reported_seq":4388,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:26.125720+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:26.125720+0000","last_peered":"2024-09-10T15:53:26.125720+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:26.125720+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4255,"log_dups_size":0,"ondisk_log_size":4255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17379811,"num_objects":4244,"num_object_clones":0,"num_object_copies":8488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1074,"num_write_kb":4296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":241},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":268710371,"num_objects":65604,"num_object_clones":0,"num_object_copies":131208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3881,"num_objects_unfound":0,"num_objects_dirty":65604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27206,"num_write_kb":108824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":454656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":542281728,"data_stored":542191403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65685,"ondisk_log_size":65685,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626436,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72288,"kb_used_data":14288,"kb_used_omap":4,"kb_used_meta":57979,"kb_avail":94299552,"statfs":{"total":96636764160,"available":96562741248,"internally_reserved":0,"allocated":14630912,"data_stored":14295768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":59371182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149719,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407588,"kb_used_data":259348,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93964252,"statfs":{"total":96636764160,"available":96219394048,"internally_reserved":0,"allocated":265572352,"data_stored":265316694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":151776941},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398000,"kb_used_data":253920,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93973840,"statfs":{"total":96636764160,"available":96229212160,"internally_reserved":0,"allocated":260014080,"data_stored":259758422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,11,0,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":266850304,"data_stored":266846691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15323136,"data_stored":15240037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260108288,"data_stored":260104675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:29.803 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 52 20 68238 68218 5.12391 2.11328 0.0341893 0.0152384 2024-09-10T15:53:30.804 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 53 20 68929 68909 5.07807 2.69922 0.0507753 0.0153761 2024-09-10T15:53:31.804 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 54 20 69621 69601 5.03409 2.70312 0.0244652 0.0155094 2024-09-10T15:53:32.593 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:32.804 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 55 20 70171 70151 4.98162 2.14844 0.0307308 0.0156751 2024-09-10T15:53:33.144 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:33.145 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:33.172 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-10T15:53:32.446626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":281616886,"num_objects":68757,"num_object_clones":0,"num_object_copies":137514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3741,"num_objects_unfound":0,"num_objects_dirty":68757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30359,"num_write_kb":121430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":1028115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68839,"ondisk_log_size":68839,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":885300,"kb_used_data":543300,"kb_used_omap":12,"kb_used_meta":341939,"kb_avail":282230220,"statfs":{"total":289910292480,"available":289003745280,"internally_reserved":0,"allocated":556339200,"data_stored":555492740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":350145541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,18,10,14,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30920704,"num_objects":7549,"num_object_clones":0,"num_object_copies":15098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":-26613,"num_objects_unfound":0,"num_objects_dirty":7549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7549,"num_write_kb":30196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006801"},"pg_stats":[{"pgid":"2.f","version":"44'4421","reported_seq":4492,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.122069+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:31.122069+0000","last_peered":"2024-09-10T15:53:31.122069+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:31.122069+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4421,"log_dups_size":0,"ondisk_log_size":4421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18108416,"num_objects":4421,"num_object_clones":0,"num_object_copies":8842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4421,"num_write_kb":17684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":448},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4333","reported_seq":4435,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:29.259680+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:29.259680+0000","last_peered":"2024-09-10T15:53:29.259680+0000","last_clean":"2024-09-10T15:53:29.259680+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:29.259680+0000","last_undegraded":"2024-09-10T15:53:29.259680+0000","last_fullsized":"2024-09-10T15:53:29.259680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4333,"log_dups_size":0,"ondisk_log_size":4333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17723392,"num_objects":4327,"num_object_clones":0,"num_object_copies":8654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2669,"num_write_kb":10676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4167","reported_seq":4285,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:29.257578+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:29.257578+0000","last_peered":"2024-09-10T15:53:29.257578+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:29.257578+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":4167,"log_dups_size":0,"ondisk_log_size":4167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17051648,"num_objects":4163,"num_object_clones":0,"num_object_copies":8326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2435,"num_write_kb":9740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":361}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4251","reported_seq":4325,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:29.225856+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:29.225856+0000","last_peered":"2024-09-10T15:53:29.225856+0000","last_clean":"2024-09-10T15:53:29.225856+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:29.225856+0000","last_undegraded":"2024-09-10T15:53:29.225856+0000","last_fullsized":"2024-09-10T15:53:29.225856+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4251,"log_dups_size":0,"ondisk_log_size":4251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17412096,"num_objects":4251,"num_object_clones":0,"num_object_copies":8502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4251,"num_write_kb":17004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4264","reported_seq":4385,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:31.162566+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:31.162566+0000","last_peered":"2024-09-10T15:53:31.162566+0000","last_clean":"2024-09-10T15:53:31.162566+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:31.162566+0000","last_undegraded":"2024-09-10T15:53:31.162566+0000","last_fullsized":"2024-09-10T15:53:31.162566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4264,"log_dups_size":0,"ondisk_log_size":4264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17436672,"num_objects":4257,"num_object_clones":0,"num_object_copies":8514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1335,"num_write_kb":5340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4297","reported_seq":4426,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.084344+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:31.084344+0000","last_peered":"2024-09-10T15:53:31.084344+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:31.084344+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4297,"log_dups_size":0,"ondisk_log_size":4297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17580032,"num_objects":4292,"num_object_clones":0,"num_object_copies":8584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1015,"num_write_kb":4060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":424},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4260","reported_seq":4395,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.204592+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:31.204592+0000","last_peered":"2024-09-10T15:53:31.204592+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:31.204592+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4260,"log_dups_size":0,"ondisk_log_size":4260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17424384,"num_objects":4254,"num_object_clones":0,"num_object_copies":8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":910,"num_write_kb":3640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":420},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4374","reported_seq":4500,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:31.126008+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:31.126008+0000","last_peered":"2024-09-10T15:53:31.126008+0000","last_clean":"2024-09-10T15:53:31.126008+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:31.126008+0000","last_undegraded":"2024-09-10T15:53:31.126008+0000","last_fullsized":"2024-09-10T15:53:31.126008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4374,"log_dups_size":0,"ondisk_log_size":4374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907712,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1133,"num_write_kb":4532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4353","reported_seq":4975,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:29.256633+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:29.256633+0000","last_peered":"2024-09-10T15:53:29.256633+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:29.256633+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4353,"log_dups_size":0,"ondisk_log_size":4353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3741,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2330,"num_write_kb":9320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3741},{"shards":"2","objects":603}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4333","reported_seq":4450,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.193977+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:31.193977+0000","last_peered":"2024-09-10T15:53:31.193977+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:31.193977+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4333,"log_dups_size":0,"ondisk_log_size":4333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17715200,"num_objects":4325,"num_object_clones":0,"num_object_copies":8650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1197,"num_write_kb":4788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":462},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4166","reported_seq":4283,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:29.268272+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:29.268272+0000","last_peered":"2024-09-10T15:53:29.268272+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:29.268272+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":4166,"log_dups_size":0,"ondisk_log_size":4166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17043456,"num_objects":4161,"num_object_clones":0,"num_object_copies":8322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2140,"num_write_kb":8560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":329}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4180","reported_seq":4301,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:29.208609+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:29.208609+0000","last_peered":"2024-09-10T15:53:29.208609+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:29.208609+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":4180,"log_dups_size":0,"ondisk_log_size":4180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17096704,"num_objects":4174,"num_object_clones":0,"num_object_copies":8348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2049,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":345}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4279","reported_seq":4401,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.201727+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:31.201727+0000","last_peered":"2024-09-10T15:53:31.201727+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:31.201727+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4279,"log_dups_size":0,"ondisk_log_size":4279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17518592,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1471,"num_write_kb":5884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":444},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4394","reported_seq":4551,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.176812+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:31.176812+0000","last_peered":"2024-09-10T15:53:31.176812+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:31.176812+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17965056,"num_objects":4386,"num_object_clones":0,"num_object_copies":8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":812,"num_write_kb":3248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":469},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4325","reported_seq":4458,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.205017+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:31.205017+0000","last_peered":"2024-09-10T15:53:31.205017+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:31.205017+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4325,"log_dups_size":0,"ondisk_log_size":4325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17698816,"num_objects":4321,"num_object_clones":0,"num_object_copies":8642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":931,"num_write_kb":3724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":444},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4439","reported_seq":4572,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.150130+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:31.150130+0000","last_peered":"2024-09-10T15:53:31.150130+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:31.150130+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4439,"log_dups_size":0,"ondisk_log_size":4439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18133475,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1258,"num_write_kb":5032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":425},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":281616867,"num_objects":68755,"num_object_clones":0,"num_object_copies":137510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3741,"num_objects_unfound":0,"num_objects_dirty":68755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30357,"num_write_kb":121428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":1028096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":558514176,"data_stored":558423851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68836,"ondisk_log_size":68836,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626436,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72288,"kb_used_data":14288,"kb_used_omap":4,"kb_used_meta":57979,"kb_avail":94299552,"statfs":{"total":96636764160,"available":96562741248,"internally_reserved":0,"allocated":14630912,"data_stored":14295768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":59371182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149720,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404716,"kb_used_data":268956,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93967124,"statfs":{"total":96636764160,"available":96222334976,"internally_reserved":0,"allocated":275410944,"data_stored":275155286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":138997421},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,17,6,6,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408296,"kb_used_data":260056,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93963544,"statfs":{"total":96636764160,"available":96218669056,"internally_reserved":0,"allocated":266297344,"data_stored":266041686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,0,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":276590592,"data_stored":276586979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15323136,"data_stored":15240037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266600448,"data_stored":266596835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:33.173 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:33.703 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:33.704 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:33.732 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-10T15:53:32.446626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":281616886,"num_objects":68757,"num_object_clones":0,"num_object_copies":137514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3741,"num_objects_unfound":0,"num_objects_dirty":68757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30359,"num_write_kb":121430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":1028115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68839,"ondisk_log_size":68839,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":885300,"kb_used_data":543300,"kb_used_omap":12,"kb_used_meta":341939,"kb_avail":282230220,"statfs":{"total":289910292480,"available":289003745280,"internally_reserved":0,"allocated":556339200,"data_stored":555492740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":350145541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,18,10,14,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30920704,"num_objects":7549,"num_object_clones":0,"num_object_copies":15098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":-26613,"num_objects_unfound":0,"num_objects_dirty":7549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7549,"num_write_kb":30196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006801"},"pg_stats":[{"pgid":"2.f","version":"44'4421","reported_seq":4492,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.122069+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:31.122069+0000","last_peered":"2024-09-10T15:53:31.122069+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:31.122069+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4421,"log_dups_size":0,"ondisk_log_size":4421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18108416,"num_objects":4421,"num_object_clones":0,"num_object_copies":8842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4421,"num_write_kb":17684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":448},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4333","reported_seq":4435,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:29.259680+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:29.259680+0000","last_peered":"2024-09-10T15:53:29.259680+0000","last_clean":"2024-09-10T15:53:29.259680+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:29.259680+0000","last_undegraded":"2024-09-10T15:53:29.259680+0000","last_fullsized":"2024-09-10T15:53:29.259680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4333,"log_dups_size":0,"ondisk_log_size":4333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17723392,"num_objects":4327,"num_object_clones":0,"num_object_copies":8654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2669,"num_write_kb":10676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4167","reported_seq":4285,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:29.257578+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:29.257578+0000","last_peered":"2024-09-10T15:53:29.257578+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:29.257578+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":4167,"log_dups_size":0,"ondisk_log_size":4167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17051648,"num_objects":4163,"num_object_clones":0,"num_object_copies":8326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2435,"num_write_kb":9740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":361}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4251","reported_seq":4325,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:29.225856+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:29.225856+0000","last_peered":"2024-09-10T15:53:29.225856+0000","last_clean":"2024-09-10T15:53:29.225856+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:29.225856+0000","last_undegraded":"2024-09-10T15:53:29.225856+0000","last_fullsized":"2024-09-10T15:53:29.225856+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4251,"log_dups_size":0,"ondisk_log_size":4251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17412096,"num_objects":4251,"num_object_clones":0,"num_object_copies":8502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4251,"num_write_kb":17004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4264","reported_seq":4385,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:31.162566+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:31.162566+0000","last_peered":"2024-09-10T15:53:31.162566+0000","last_clean":"2024-09-10T15:53:31.162566+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:31.162566+0000","last_undegraded":"2024-09-10T15:53:31.162566+0000","last_fullsized":"2024-09-10T15:53:31.162566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4264,"log_dups_size":0,"ondisk_log_size":4264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17436672,"num_objects":4257,"num_object_clones":0,"num_object_copies":8514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1335,"num_write_kb":5340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4297","reported_seq":4426,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.084344+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:31.084344+0000","last_peered":"2024-09-10T15:53:31.084344+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:31.084344+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4297,"log_dups_size":0,"ondisk_log_size":4297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17580032,"num_objects":4292,"num_object_clones":0,"num_object_copies":8584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1015,"num_write_kb":4060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":424},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4260","reported_seq":4395,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.204592+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:31.204592+0000","last_peered":"2024-09-10T15:53:31.204592+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:31.204592+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4260,"log_dups_size":0,"ondisk_log_size":4260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17424384,"num_objects":4254,"num_object_clones":0,"num_object_copies":8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":910,"num_write_kb":3640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":420},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4374","reported_seq":4500,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:31.126008+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:31.126008+0000","last_peered":"2024-09-10T15:53:31.126008+0000","last_clean":"2024-09-10T15:53:31.126008+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:31.126008+0000","last_undegraded":"2024-09-10T15:53:31.126008+0000","last_fullsized":"2024-09-10T15:53:31.126008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4374,"log_dups_size":0,"ondisk_log_size":4374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907712,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1133,"num_write_kb":4532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4353","reported_seq":4975,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:29.256633+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:29.256633+0000","last_peered":"2024-09-10T15:53:29.256633+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:29.256633+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4353,"log_dups_size":0,"ondisk_log_size":4353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3741,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2330,"num_write_kb":9320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3741},{"shards":"2","objects":603}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4333","reported_seq":4450,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.193977+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:31.193977+0000","last_peered":"2024-09-10T15:53:31.193977+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:31.193977+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4333,"log_dups_size":0,"ondisk_log_size":4333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17715200,"num_objects":4325,"num_object_clones":0,"num_object_copies":8650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1197,"num_write_kb":4788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":462},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4166","reported_seq":4283,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:29.268272+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:29.268272+0000","last_peered":"2024-09-10T15:53:29.268272+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:29.268272+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":4166,"log_dups_size":0,"ondisk_log_size":4166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17043456,"num_objects":4161,"num_object_clones":0,"num_object_copies":8322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2140,"num_write_kb":8560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":329}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4180","reported_seq":4301,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:29.208609+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:29.208609+0000","last_peered":"2024-09-10T15:53:29.208609+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:29.208609+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":4180,"log_dups_size":0,"ondisk_log_size":4180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17096704,"num_objects":4174,"num_object_clones":0,"num_object_copies":8348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2049,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":345}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4279","reported_seq":4401,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.201727+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:31.201727+0000","last_peered":"2024-09-10T15:53:31.201727+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:31.201727+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4279,"log_dups_size":0,"ondisk_log_size":4279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17518592,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1471,"num_write_kb":5884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":444},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4394","reported_seq":4551,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.176812+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:31.176812+0000","last_peered":"2024-09-10T15:53:31.176812+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:31.176812+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17965056,"num_objects":4386,"num_object_clones":0,"num_object_copies":8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":812,"num_write_kb":3248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":469},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4325","reported_seq":4458,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.205017+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:31.205017+0000","last_peered":"2024-09-10T15:53:31.205017+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:31.205017+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4325,"log_dups_size":0,"ondisk_log_size":4325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17698816,"num_objects":4321,"num_object_clones":0,"num_object_copies":8642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":931,"num_write_kb":3724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":444},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4439","reported_seq":4572,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.150130+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:31.150130+0000","last_peered":"2024-09-10T15:53:31.150130+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:31.150130+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4439,"log_dups_size":0,"ondisk_log_size":4439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18133475,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1258,"num_write_kb":5032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":425},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":281616867,"num_objects":68755,"num_object_clones":0,"num_object_copies":137510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3741,"num_objects_unfound":0,"num_objects_dirty":68755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30357,"num_write_kb":121428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":1028096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":558514176,"data_stored":558423851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68836,"ondisk_log_size":68836,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626436,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72288,"kb_used_data":14288,"kb_used_omap":4,"kb_used_meta":57979,"kb_avail":94299552,"statfs":{"total":96636764160,"available":96562741248,"internally_reserved":0,"allocated":14630912,"data_stored":14295768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":59371182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149720,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404716,"kb_used_data":268956,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93967124,"statfs":{"total":96636764160,"available":96222334976,"internally_reserved":0,"allocated":275410944,"data_stored":275155286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":138997421},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,17,6,6,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408296,"kb_used_data":260056,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93963544,"statfs":{"total":96636764160,"available":96218669056,"internally_reserved":0,"allocated":266297344,"data_stored":266041686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,0,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":276590592,"data_stored":276586979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15323136,"data_stored":15240037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266600448,"data_stored":266596835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:33.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:53:33.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'4421", 'reported_seq': 4492, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:31.122069+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:53:31.122069+0000', 'last_peered': '2024-09-10T15:53:31.122069+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:53:31.122069+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 4421, 'log_dups_size': 0, 'ondisk_log_size': 4421, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18108416, 'num_objects': 4421, 'num_object_clones': 0, 'num_object_copies': 8842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4421, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4421, 'num_write_kb': 17684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 448}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:33.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:53:33.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'4167", 'reported_seq': 4285, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:29.257578+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:53:29.257578+0000', 'last_peered': '2024-09-10T15:53:29.257578+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:53:29.257578+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 4167, 'log_dups_size': 0, 'ondisk_log_size': 4167, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17051648, 'num_objects': 4163, 'num_object_clones': 0, 'num_object_copies': 8326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4163, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2435, 'num_write_kb': 9740, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 361}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:33.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:53:33.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'4297", 'reported_seq': 4426, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:31.084344+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:53:31.084344+0000', 'last_peered': '2024-09-10T15:53:31.084344+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:53:31.084344+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 4297, 'log_dups_size': 0, 'ondisk_log_size': 4297, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17580032, 'num_objects': 4292, 'num_object_clones': 0, 'num_object_copies': 8584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4292, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1015, 'num_write_kb': 4060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 424}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:33.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:53:33.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'4260", 'reported_seq': 4395, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:31.204592+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:53:31.204592+0000', 'last_peered': '2024-09-10T15:53:31.204592+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:53:31.204592+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 4260, 'log_dups_size': 0, 'ondisk_log_size': 4260, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17424384, 'num_objects': 4254, 'num_object_clones': 0, 'num_object_copies': 8508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4254, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 910, 'num_write_kb': 3640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 420}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:33.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:53:33.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'4353", 'reported_seq': 4975, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:29.256633+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:53:29.256633+0000', 'last_peered': '2024-09-10T15:53:29.256633+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:53:29.256633+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 4353, 'log_dups_size': 0, 'ondisk_log_size': 4353, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17801216, 'num_objects': 4346, 'num_object_clones': 0, 'num_object_copies': 8692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3741, 'num_objects_unfound': 0, 'num_objects_dirty': 4346, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2330, 'num_write_kb': 9320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 249, 'num_bytes_recovered': 1019904, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3741}, {'shards': '2', 'objects': 603}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:33.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:53:33.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'4333", 'reported_seq': 4450, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:31.193977+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:53:31.193977+0000', 'last_peered': '2024-09-10T15:53:31.193977+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:53:31.193977+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 4333, 'log_dups_size': 0, 'ondisk_log_size': 4333, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17715200, 'num_objects': 4325, 'num_object_clones': 0, 'num_object_copies': 8650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4325, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1197, 'num_write_kb': 4788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 462}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:33.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:53:33.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'4166", 'reported_seq': 4283, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:29.268272+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:53:29.268272+0000', 'last_peered': '2024-09-10T15:53:29.268272+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:53:29.268272+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 4166, 'log_dups_size': 0, 'ondisk_log_size': 4166, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17043456, 'num_objects': 4161, 'num_object_clones': 0, 'num_object_copies': 8322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4161, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2140, 'num_write_kb': 8560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 329}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:33.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:53:33.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'4180", 'reported_seq': 4301, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:29.208609+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:53:29.208609+0000', 'last_peered': '2024-09-10T15:53:29.208609+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:53:29.208609+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 4180, 'log_dups_size': 0, 'ondisk_log_size': 4180, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17096704, 'num_objects': 4174, 'num_object_clones': 0, 'num_object_copies': 8348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4174, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2049, 'num_write_kb': 8196, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 345}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:33.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:53:33.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'4279", 'reported_seq': 4401, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:31.201727+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:53:31.201727+0000', 'last_peered': '2024-09-10T15:53:31.201727+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:53:31.201727+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 4279, 'log_dups_size': 0, 'ondisk_log_size': 4279, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17518592, 'num_objects': 4277, 'num_object_clones': 0, 'num_object_copies': 8554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4277, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1471, 'num_write_kb': 5884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 444}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:33.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:53:33.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'4394", 'reported_seq': 4551, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:31.176812+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:53:31.176812+0000', 'last_peered': '2024-09-10T15:53:31.176812+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:53:31.176812+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 4394, 'log_dups_size': 0, 'ondisk_log_size': 4394, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17965056, 'num_objects': 4386, 'num_object_clones': 0, 'num_object_copies': 8772, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4386, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 812, 'num_write_kb': 3248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 469}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:33.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:53:33.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'4325", 'reported_seq': 4458, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:31.205017+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:53:31.205017+0000', 'last_peered': '2024-09-10T15:53:31.205017+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:53:31.205017+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 4325, 'log_dups_size': 0, 'ondisk_log_size': 4325, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17698816, 'num_objects': 4321, 'num_object_clones': 0, 'num_object_copies': 8642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4321, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 931, 'num_write_kb': 3724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 444}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:33.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:53:33.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'4439", 'reported_seq': 4572, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:31.150130+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:53:31.150130+0000', 'last_peered': '2024-09-10T15:53:31.150130+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:53:31.150130+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 4439, 'log_dups_size': 0, 'ondisk_log_size': 4439, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18133475, 'num_objects': 4428, 'num_object_clones': 0, 'num_object_copies': 8856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1258, 'num_write_kb': 5032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 425}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:33.736 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:53:33.805 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 56 20 70772 70752 4.93458 2.34766 0.0453621 0.0158231 2024-09-10T15:53:34.406 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:34.406 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/137514 objects degraded (31.017%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":140,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":68757,"data_bytes":281616886,"bytes_used":906547200,"bytes_avail":289003745280,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":137514,"degraded_ratio":0.31017205520892416,"misplaced_objects":3741,"misplaced_total":137514,"misplaced_ratio":0.027204502814258912,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":84943,"recovering_keys_per_sec":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"write_bytes_sec":2575265,"read_op_per_sec":0,"write_op_per_sec":628},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-10T15:53:24.443468+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:53:34.430 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:53:34.430 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:34.804 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 57 20 71360 71340 4.88829 2.29688 0.0283805 0.015972 2024-09-10T15:53:34.990 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:34.990 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:35.016 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-10T15:53:34.447054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":286401014,"num_objects":69925,"num_object_clones":0,"num_object_copies":139850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3600,"num_objects_unfound":0,"num_objects_dirty":69925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31527,"num_write_kb":126102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":1605651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70007,"ondisk_log_size":70007,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":880244,"kb_used_data":559044,"kb_used_omap":12,"kb_used_meta":321139,"kb_avail":282235276,"statfs":{"total":289910292480,"available":289008922624,"internally_reserved":0,"allocated":572461056,"data_stored":571614596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":328846341},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,22,7,19,7,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11470,"num_objects_misplaced":3599,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7157,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":1597440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004099"},"pg_stats":[{"pgid":"2.f","version":"44'4421","reported_seq":4492,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.122069+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:31.122069+0000","last_peered":"2024-09-10T15:53:31.122069+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:31.122069+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4421,"log_dups_size":0,"ondisk_log_size":4421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18108416,"num_objects":4421,"num_object_clones":0,"num_object_copies":8842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4421,"num_write_kb":17684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":448},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4515","reported_seq":4616,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:34.289557+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:34.289557+0000","last_peered":"2024-09-10T15:53:34.289557+0000","last_clean":"2024-09-10T15:53:34.289557+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:34.289557+0000","last_undegraded":"2024-09-10T15:53:34.289557+0000","last_fullsized":"2024-09-10T15:53:34.289557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4515,"log_dups_size":0,"ondisk_log_size":4515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18468864,"num_objects":4509,"num_object_clones":0,"num_object_copies":9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2851,"num_write_kb":11404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4354","reported_seq":4472,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.144079+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:34.144079+0000","last_peered":"2024-09-10T15:53:34.144079+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:34.144079+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":4354,"log_dups_size":0,"ondisk_log_size":4354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17817600,"num_objects":4350,"num_object_clones":0,"num_object_copies":8700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2622,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":548}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4460","reported_seq":4534,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:34.260396+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:34.260396+0000","last_peered":"2024-09-10T15:53:34.260396+0000","last_clean":"2024-09-10T15:53:34.260396+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:34.260396+0000","last_undegraded":"2024-09-10T15:53:34.260396+0000","last_fullsized":"2024-09-10T15:53:34.260396+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18268160,"num_objects":4460,"num_object_clones":0,"num_object_copies":8920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4460,"num_write_kb":17840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4264","reported_seq":4385,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:31.162566+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:31.162566+0000","last_peered":"2024-09-10T15:53:31.162566+0000","last_clean":"2024-09-10T15:53:31.162566+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:31.162566+0000","last_undegraded":"2024-09-10T15:53:31.162566+0000","last_fullsized":"2024-09-10T15:53:31.162566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4264,"log_dups_size":0,"ondisk_log_size":4264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17436672,"num_objects":4257,"num_object_clones":0,"num_object_copies":8514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1335,"num_write_kb":5340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4297","reported_seq":4426,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.084344+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:31.084344+0000","last_peered":"2024-09-10T15:53:31.084344+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:31.084344+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4297,"log_dups_size":0,"ondisk_log_size":4297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17580032,"num_objects":4292,"num_object_clones":0,"num_object_copies":8584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1015,"num_write_kb":4060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":424},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4260","reported_seq":4395,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.204592+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:31.204592+0000","last_peered":"2024-09-10T15:53:31.204592+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:31.204592+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4260,"log_dups_size":0,"ondisk_log_size":4260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17424384,"num_objects":4254,"num_object_clones":0,"num_object_copies":8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":910,"num_write_kb":3640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":420},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4374","reported_seq":4500,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:31.126008+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:31.126008+0000","last_peered":"2024-09-10T15:53:31.126008+0000","last_clean":"2024-09-10T15:53:31.126008+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:31.126008+0000","last_undegraded":"2024-09-10T15:53:31.126008+0000","last_fullsized":"2024-09-10T15:53:31.126008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4374,"log_dups_size":0,"ondisk_log_size":4374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907712,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1133,"num_write_kb":4532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4542","reported_seq":5446,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.301107+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:34.301107+0000","last_peered":"2024-09-10T15:53:34.301107+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:34.301107+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4542,"log_dups_size":0,"ondisk_log_size":4542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18575360,"num_objects":4535,"num_object_clones":0,"num_object_copies":9070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3600,"num_objects_unfound":0,"num_objects_dirty":4535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2519,"num_write_kb":10076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":1597440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3600},{"shards":"2","objects":933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4333","reported_seq":4450,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.193977+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:31.193977+0000","last_peered":"2024-09-10T15:53:31.193977+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:31.193977+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4333,"log_dups_size":0,"ondisk_log_size":4333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17715200,"num_objects":4325,"num_object_clones":0,"num_object_copies":8650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1197,"num_write_kb":4788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":462},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4368","reported_seq":4487,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.228530+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:34.228530+0000","last_peered":"2024-09-10T15:53:34.228530+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:34.228530+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":4368,"log_dups_size":0,"ondisk_log_size":4368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17870848,"num_objects":4363,"num_object_clones":0,"num_object_copies":8726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":9368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":531}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4379","reported_seq":4500,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.284987+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:34.284987+0000","last_peered":"2024-09-10T15:53:34.284987+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:34.284987+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":4379,"log_dups_size":0,"ondisk_log_size":4379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17911808,"num_objects":4373,"num_object_clones":0,"num_object_copies":8746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2248,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":544}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4279","reported_seq":4401,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.201727+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:31.201727+0000","last_peered":"2024-09-10T15:53:31.201727+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:31.201727+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4279,"log_dups_size":0,"ondisk_log_size":4279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17518592,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1471,"num_write_kb":5884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":444},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4394","reported_seq":4551,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.176812+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:31.176812+0000","last_peered":"2024-09-10T15:53:31.176812+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:31.176812+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17965056,"num_objects":4386,"num_object_clones":0,"num_object_copies":8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":812,"num_write_kb":3248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":469},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4325","reported_seq":4458,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.205017+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:31.205017+0000","last_peered":"2024-09-10T15:53:31.205017+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:31.205017+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4325,"log_dups_size":0,"ondisk_log_size":4325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17698816,"num_objects":4321,"num_object_clones":0,"num_object_copies":8642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":931,"num_write_kb":3724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":444},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4439","reported_seq":4572,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:31.150130+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:31.150130+0000","last_peered":"2024-09-10T15:53:31.150130+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:31.150130+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4439,"log_dups_size":0,"ondisk_log_size":4439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18133475,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1258,"num_write_kb":5032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":425},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":286400995,"num_objects":69923,"num_object_clones":0,"num_object_copies":139846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3600,"num_objects_unfound":0,"num_objects_dirty":69923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31525,"num_write_kb":126100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":574918656,"data_stored":574828331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70004,"ondisk_log_size":70004,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626437,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69284,"kb_used_data":23764,"kb_used_omap":4,"kb_used_meta":45499,"kb_avail":94302556,"statfs":{"total":96636764160,"available":96565817344,"internally_reserved":0,"allocated":24334336,"data_stored":23999192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":46591662},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,8,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149720,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404716,"kb_used_data":268956,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93967124,"statfs":{"total":96636764160,"available":96222334976,"internally_reserved":0,"allocated":275410944,"data_stored":275155286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":138997421},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,17,6,6,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406244,"kb_used_data":266324,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93965596,"statfs":{"total":96636764160,"available":96220770304,"internally_reserved":0,"allocated":272715776,"data_stored":272460118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,0,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":276590592,"data_stored":276586979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25448448,"data_stored":25365349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":272879616,"data_stored":272876003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:35.805 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 58 20 72019 71999 4.84837 2.57422 0.0599105 0.0161041 2024-09-10T15:53:36.805 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 59 20 72692 72672 4.81075 2.62891 0.0146097 0.0162327 2024-09-10T15:53:37.805 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:2024-09-10T15:53:37.804702+0000 min lat: 0.00198913 max lat: 2.66872 avg lat: 0.0163565 2024-09-10T15:53:37.805 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:53:37.805 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 60 20 73357 73337 4.77386 2.59766 0.0122263 0.0163565 2024-09-10T15:53:38.018 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:38.578 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:38.609 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-10T15:53:38.447927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":294105590,"num_objects":71806,"num_object_clones":0,"num_object_copies":143612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3600,"num_objects_unfound":0,"num_objects_dirty":71806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33408,"num_write_kb":133626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":1605651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71888,"ondisk_log_size":71888,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":952072,"kb_used_data":578132,"kb_used_omap":12,"kb_used_meta":373875,"kb_avail":282163448,"statfs":{"total":289910292480,"available":288935370752,"internally_reserved":0,"allocated":592007168,"data_stored":591160708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":382848005},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,7,6,10,12,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25395200,"num_objects":6200,"num_object_clones":0,"num_object_copies":12400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-281,"num_objects_unfound":0,"num_objects_dirty":6200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6200,"num_write_kb":24800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":1150976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004096"},"pg_stats":[{"pgid":"2.f","version":"44'4611","reported_seq":4681,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.198717+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:36.198717+0000","last_peered":"2024-09-10T15:53:36.198717+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:36.198717+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4611,"log_dups_size":0,"ondisk_log_size":4611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18886656,"num_objects":4611,"num_object_clones":0,"num_object_copies":9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4611,"num_write_kb":18444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":638},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4515","reported_seq":4616,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:34.289557+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:34.289557+0000","last_peered":"2024-09-10T15:53:34.289557+0000","last_clean":"2024-09-10T15:53:34.289557+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:34.289557+0000","last_undegraded":"2024-09-10T15:53:34.289557+0000","last_fullsized":"2024-09-10T15:53:34.289557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4515,"log_dups_size":0,"ondisk_log_size":4515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18468864,"num_objects":4509,"num_object_clones":0,"num_object_copies":9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2851,"num_write_kb":11404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4354","reported_seq":4472,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.144079+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:34.144079+0000","last_peered":"2024-09-10T15:53:34.144079+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:34.144079+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":4354,"log_dups_size":0,"ondisk_log_size":4354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17817600,"num_objects":4350,"num_object_clones":0,"num_object_copies":8700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2622,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":548}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4460","reported_seq":4534,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:34.260396+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:34.260396+0000","last_peered":"2024-09-10T15:53:34.260396+0000","last_clean":"2024-09-10T15:53:34.260396+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:34.260396+0000","last_undegraded":"2024-09-10T15:53:34.260396+0000","last_fullsized":"2024-09-10T15:53:34.260396+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18268160,"num_objects":4460,"num_object_clones":0,"num_object_copies":8920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4460,"num_write_kb":17840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4458","reported_seq":4578,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:36.215124+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:36.215124+0000","last_peered":"2024-09-10T15:53:36.215124+0000","last_clean":"2024-09-10T15:53:36.215124+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:36.215124+0000","last_undegraded":"2024-09-10T15:53:36.215124+0000","last_fullsized":"2024-09-10T15:53:36.215124+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4458,"log_dups_size":0,"ondisk_log_size":4458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1529,"num_write_kb":6116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4486","reported_seq":4614,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.233182+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:36.233182+0000","last_peered":"2024-09-10T15:53:36.233182+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:36.233182+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4486,"log_dups_size":0,"ondisk_log_size":4486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18354176,"num_objects":4481,"num_object_clones":0,"num_object_copies":8962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1204,"num_write_kb":4816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":613},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4456","reported_seq":4591,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.199134+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:36.199134+0000","last_peered":"2024-09-10T15:53:36.199134+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:36.199134+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4456,"log_dups_size":0,"ondisk_log_size":4456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18227200,"num_objects":4450,"num_object_clones":0,"num_object_copies":8900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1106,"num_write_kb":4424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":616},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4565","reported_seq":4691,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:36.225670+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:36.225670+0000","last_peered":"2024-09-10T15:53:36.225670+0000","last_clean":"2024-09-10T15:53:36.225670+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:36.225670+0000","last_undegraded":"2024-09-10T15:53:36.225670+0000","last_fullsized":"2024-09-10T15:53:36.225670+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4565,"log_dups_size":0,"ondisk_log_size":4565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18690048,"num_objects":4563,"num_object_clones":0,"num_object_copies":9126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1324,"num_write_kb":5296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4542","reported_seq":5446,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.301107+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:34.301107+0000","last_peered":"2024-09-10T15:53:34.301107+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:34.301107+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4542,"log_dups_size":0,"ondisk_log_size":4542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18575360,"num_objects":4535,"num_object_clones":0,"num_object_copies":9070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3600,"num_objects_unfound":0,"num_objects_dirty":4535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2519,"num_write_kb":10076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":1597440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3600},{"shards":"2","objects":933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4503","reported_seq":4624,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.233175+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:36.233175+0000","last_peered":"2024-09-10T15:53:36.233175+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:36.233175+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4503,"log_dups_size":0,"ondisk_log_size":4503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18411520,"num_objects":4495,"num_object_clones":0,"num_object_copies":8990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1367,"num_write_kb":5468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":632},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4368","reported_seq":4487,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.228530+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:34.228530+0000","last_peered":"2024-09-10T15:53:34.228530+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:34.228530+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":4368,"log_dups_size":0,"ondisk_log_size":4368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17870848,"num_objects":4363,"num_object_clones":0,"num_object_copies":8726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":9368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":531}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4379","reported_seq":4500,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.284987+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:34.284987+0000","last_peered":"2024-09-10T15:53:34.284987+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:34.284987+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":4379,"log_dups_size":0,"ondisk_log_size":4379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17911808,"num_objects":4373,"num_object_clones":0,"num_object_copies":8746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2248,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":544}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4475","reported_seq":4596,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.236503+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:36.236503+0000","last_peered":"2024-09-10T15:53:36.236503+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:36.236503+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4475,"log_dups_size":0,"ondisk_log_size":4475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1667,"num_write_kb":6668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":640},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4587","reported_seq":4745,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.224326+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:36.224326+0000","last_peered":"2024-09-10T15:53:36.224326+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:36.224326+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18755584,"num_objects":4579,"num_object_clones":0,"num_object_copies":9158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1005,"num_write_kb":4020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":662},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4509","reported_seq":4643,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.116244+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:36.116244+0000","last_peered":"2024-09-10T15:53:36.116244+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:36.116244+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4509,"log_dups_size":0,"ondisk_log_size":4509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18452480,"num_objects":4505,"num_object_clones":0,"num_object_copies":9010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1115,"num_write_kb":4460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":628},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4617","reported_seq":4750,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.161036+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:36.161036+0000","last_peered":"2024-09-10T15:53:36.161036+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:36.161036+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4617,"log_dups_size":0,"ondisk_log_size":4617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18862563,"num_objects":4606,"num_object_clones":0,"num_object_copies":9212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1436,"num_write_kb":5744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":603},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":294105571,"num_objects":71804,"num_object_clones":0,"num_object_copies":143608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3600,"num_objects_unfound":0,"num_objects_dirty":71804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33406,"num_write_kb":133624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":594640896,"data_stored":594550571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71885,"ondisk_log_size":71885,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626438,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123492,"kb_used_data":33556,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94248348,"statfs":{"total":96636764160,"available":96510308352,"internally_reserved":0,"allocated":34361344,"data_stored":34026200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92073646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149721,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422336,"kb_used_data":278252,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93949504,"statfs":{"total":96636764160,"available":96204292096,"internally_reserved":0,"allocated":284930048,"data_stored":284674390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":147517101},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406244,"kb_used_data":266324,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93965596,"statfs":{"total":96636764160,"available":96220770304,"internally_reserved":0,"allocated":272715776,"data_stored":272460118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,0,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285868032,"data_stored":285864419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35893248,"data_stored":35810149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":272879616,"data_stored":272876003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:38.611 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:38.805 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 61 19 73968 73949 4.73479 2.39062 0.0652753 0.0164906 2024-09-10T15:53:39.150 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:39.156 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:39.177 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-10T15:53:38.447927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":294105590,"num_objects":71806,"num_object_clones":0,"num_object_copies":143612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3600,"num_objects_unfound":0,"num_objects_dirty":71806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33408,"num_write_kb":133626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":1605651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71888,"ondisk_log_size":71888,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":952072,"kb_used_data":578132,"kb_used_omap":12,"kb_used_meta":373875,"kb_avail":282163448,"statfs":{"total":289910292480,"available":288935370752,"internally_reserved":0,"allocated":592007168,"data_stored":591160708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":382848005},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,7,6,10,12,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25395200,"num_objects":6200,"num_object_clones":0,"num_object_copies":12400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-281,"num_objects_unfound":0,"num_objects_dirty":6200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6200,"num_write_kb":24800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":1150976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004096"},"pg_stats":[{"pgid":"2.f","version":"44'4611","reported_seq":4681,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.198717+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:36.198717+0000","last_peered":"2024-09-10T15:53:36.198717+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:36.198717+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4611,"log_dups_size":0,"ondisk_log_size":4611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18886656,"num_objects":4611,"num_object_clones":0,"num_object_copies":9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4611,"num_write_kb":18444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":638},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4515","reported_seq":4616,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:34.289557+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:34.289557+0000","last_peered":"2024-09-10T15:53:34.289557+0000","last_clean":"2024-09-10T15:53:34.289557+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:34.289557+0000","last_undegraded":"2024-09-10T15:53:34.289557+0000","last_fullsized":"2024-09-10T15:53:34.289557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4515,"log_dups_size":0,"ondisk_log_size":4515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18468864,"num_objects":4509,"num_object_clones":0,"num_object_copies":9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2851,"num_write_kb":11404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4354","reported_seq":4472,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.144079+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:34.144079+0000","last_peered":"2024-09-10T15:53:34.144079+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:34.144079+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":4354,"log_dups_size":0,"ondisk_log_size":4354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17817600,"num_objects":4350,"num_object_clones":0,"num_object_copies":8700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2622,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":548}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4460","reported_seq":4534,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:34.260396+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:34.260396+0000","last_peered":"2024-09-10T15:53:34.260396+0000","last_clean":"2024-09-10T15:53:34.260396+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:34.260396+0000","last_undegraded":"2024-09-10T15:53:34.260396+0000","last_fullsized":"2024-09-10T15:53:34.260396+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18268160,"num_objects":4460,"num_object_clones":0,"num_object_copies":8920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4460,"num_write_kb":17840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4458","reported_seq":4578,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:36.215124+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:36.215124+0000","last_peered":"2024-09-10T15:53:36.215124+0000","last_clean":"2024-09-10T15:53:36.215124+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:36.215124+0000","last_undegraded":"2024-09-10T15:53:36.215124+0000","last_fullsized":"2024-09-10T15:53:36.215124+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4458,"log_dups_size":0,"ondisk_log_size":4458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1529,"num_write_kb":6116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4486","reported_seq":4614,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.233182+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:36.233182+0000","last_peered":"2024-09-10T15:53:36.233182+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:36.233182+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4486,"log_dups_size":0,"ondisk_log_size":4486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18354176,"num_objects":4481,"num_object_clones":0,"num_object_copies":8962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1204,"num_write_kb":4816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":613},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4456","reported_seq":4591,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.199134+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:36.199134+0000","last_peered":"2024-09-10T15:53:36.199134+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:36.199134+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4456,"log_dups_size":0,"ondisk_log_size":4456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18227200,"num_objects":4450,"num_object_clones":0,"num_object_copies":8900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1106,"num_write_kb":4424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":616},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4565","reported_seq":4691,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:36.225670+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:36.225670+0000","last_peered":"2024-09-10T15:53:36.225670+0000","last_clean":"2024-09-10T15:53:36.225670+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:36.225670+0000","last_undegraded":"2024-09-10T15:53:36.225670+0000","last_fullsized":"2024-09-10T15:53:36.225670+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4565,"log_dups_size":0,"ondisk_log_size":4565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18690048,"num_objects":4563,"num_object_clones":0,"num_object_copies":9126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1324,"num_write_kb":5296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4542","reported_seq":5446,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.301107+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:34.301107+0000","last_peered":"2024-09-10T15:53:34.301107+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:34.301107+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4542,"log_dups_size":0,"ondisk_log_size":4542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18575360,"num_objects":4535,"num_object_clones":0,"num_object_copies":9070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3600,"num_objects_unfound":0,"num_objects_dirty":4535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2519,"num_write_kb":10076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":1597440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3600},{"shards":"2","objects":933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4503","reported_seq":4624,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.233175+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:36.233175+0000","last_peered":"2024-09-10T15:53:36.233175+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:36.233175+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4503,"log_dups_size":0,"ondisk_log_size":4503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18411520,"num_objects":4495,"num_object_clones":0,"num_object_copies":8990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1367,"num_write_kb":5468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":632},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4368","reported_seq":4487,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.228530+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:34.228530+0000","last_peered":"2024-09-10T15:53:34.228530+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:34.228530+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":4368,"log_dups_size":0,"ondisk_log_size":4368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17870848,"num_objects":4363,"num_object_clones":0,"num_object_copies":8726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":9368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":531}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4379","reported_seq":4500,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.284987+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:34.284987+0000","last_peered":"2024-09-10T15:53:34.284987+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:34.284987+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":4379,"log_dups_size":0,"ondisk_log_size":4379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17911808,"num_objects":4373,"num_object_clones":0,"num_object_copies":8746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2248,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":544}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4475","reported_seq":4596,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.236503+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:36.236503+0000","last_peered":"2024-09-10T15:53:36.236503+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:36.236503+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4475,"log_dups_size":0,"ondisk_log_size":4475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1667,"num_write_kb":6668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":640},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4587","reported_seq":4745,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.224326+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:36.224326+0000","last_peered":"2024-09-10T15:53:36.224326+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:36.224326+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18755584,"num_objects":4579,"num_object_clones":0,"num_object_copies":9158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1005,"num_write_kb":4020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":662},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4509","reported_seq":4643,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.116244+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:36.116244+0000","last_peered":"2024-09-10T15:53:36.116244+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:36.116244+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4509,"log_dups_size":0,"ondisk_log_size":4509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18452480,"num_objects":4505,"num_object_clones":0,"num_object_copies":9010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1115,"num_write_kb":4460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":628},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4617","reported_seq":4750,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.161036+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:36.161036+0000","last_peered":"2024-09-10T15:53:36.161036+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:36.161036+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4617,"log_dups_size":0,"ondisk_log_size":4617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18862563,"num_objects":4606,"num_object_clones":0,"num_object_copies":9212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1436,"num_write_kb":5744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":603},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":294105571,"num_objects":71804,"num_object_clones":0,"num_object_copies":143608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3600,"num_objects_unfound":0,"num_objects_dirty":71804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33406,"num_write_kb":133624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":594640896,"data_stored":594550571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71885,"ondisk_log_size":71885,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626438,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123492,"kb_used_data":33556,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94248348,"statfs":{"total":96636764160,"available":96510308352,"internally_reserved":0,"allocated":34361344,"data_stored":34026200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92073646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149721,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422336,"kb_used_data":278252,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93949504,"statfs":{"total":96636764160,"available":96204292096,"internally_reserved":0,"allocated":284930048,"data_stored":284674390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":147517101},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406244,"kb_used_data":266324,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93965596,"statfs":{"total":96636764160,"available":96220770304,"internally_reserved":0,"allocated":272715776,"data_stored":272460118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,0,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285868032,"data_stored":285864419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35893248,"data_stored":35810149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":272879616,"data_stored":272876003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:39.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:53:39.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'4611", 'reported_seq': 4681, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:36.198717+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:53:36.198717+0000', 'last_peered': '2024-09-10T15:53:36.198717+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:53:36.198717+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 4611, 'log_dups_size': 0, 'ondisk_log_size': 4611, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18886656, 'num_objects': 4611, 'num_object_clones': 0, 'num_object_copies': 9222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4611, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4611, 'num_write_kb': 18444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 638}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:39.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:53:39.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'4354", 'reported_seq': 4472, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:34.144079+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:53:34.144079+0000', 'last_peered': '2024-09-10T15:53:34.144079+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:53:34.144079+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 4354, 'log_dups_size': 0, 'ondisk_log_size': 4354, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17817600, 'num_objects': 4350, 'num_object_clones': 0, 'num_object_copies': 8700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4350, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2622, 'num_write_kb': 10488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 548}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:39.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:53:39.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'4486", 'reported_seq': 4614, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:36.233182+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:53:36.233182+0000', 'last_peered': '2024-09-10T15:53:36.233182+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:53:36.233182+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 4486, 'log_dups_size': 0, 'ondisk_log_size': 4486, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18354176, 'num_objects': 4481, 'num_object_clones': 0, 'num_object_copies': 8962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4481, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1204, 'num_write_kb': 4816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 613}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:39.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:53:39.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'4456", 'reported_seq': 4591, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:36.199134+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:53:36.199134+0000', 'last_peered': '2024-09-10T15:53:36.199134+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:53:36.199134+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 4456, 'log_dups_size': 0, 'ondisk_log_size': 4456, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18227200, 'num_objects': 4450, 'num_object_clones': 0, 'num_object_copies': 8900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4450, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1106, 'num_write_kb': 4424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 616}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:39.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:53:39.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'4542", 'reported_seq': 5446, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:34.301107+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:53:34.301107+0000', 'last_peered': '2024-09-10T15:53:34.301107+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:53:34.301107+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 4542, 'log_dups_size': 0, 'ondisk_log_size': 4542, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18575360, 'num_objects': 4535, 'num_object_clones': 0, 'num_object_copies': 9070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3600, 'num_objects_unfound': 0, 'num_objects_dirty': 4535, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2519, 'num_write_kb': 10076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 390, 'num_bytes_recovered': 1597440, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3600}, {'shards': '2', 'objects': 933}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:39.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:53:39.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'4503", 'reported_seq': 4624, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:36.233175+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:53:36.233175+0000', 'last_peered': '2024-09-10T15:53:36.233175+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:53:36.233175+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 4503, 'log_dups_size': 0, 'ondisk_log_size': 4503, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18411520, 'num_objects': 4495, 'num_object_clones': 0, 'num_object_copies': 8990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4495, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1367, 'num_write_kb': 5468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 632}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:39.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:53:39.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'4368", 'reported_seq': 4487, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:34.228530+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:53:34.228530+0000', 'last_peered': '2024-09-10T15:53:34.228530+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:53:34.228530+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 4368, 'log_dups_size': 0, 'ondisk_log_size': 4368, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17870848, 'num_objects': 4363, 'num_object_clones': 0, 'num_object_copies': 8726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4363, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2342, 'num_write_kb': 9368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 531}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:39.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:53:39.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'4379", 'reported_seq': 4500, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:34.284987+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:53:34.284987+0000', 'last_peered': '2024-09-10T15:53:34.284987+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:53:34.284987+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 4379, 'log_dups_size': 0, 'ondisk_log_size': 4379, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17911808, 'num_objects': 4373, 'num_object_clones': 0, 'num_object_copies': 8746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4373, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2248, 'num_write_kb': 8992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 544}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:39.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:53:39.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'4475", 'reported_seq': 4596, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:36.236503+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:53:36.236503+0000', 'last_peered': '2024-09-10T15:53:36.236503+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:53:36.236503+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 4475, 'log_dups_size': 0, 'ondisk_log_size': 4475, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18321408, 'num_objects': 4473, 'num_object_clones': 0, 'num_object_copies': 8946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1667, 'num_write_kb': 6668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 640}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:39.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:53:39.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'4587", 'reported_seq': 4745, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:36.224326+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:53:36.224326+0000', 'last_peered': '2024-09-10T15:53:36.224326+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:53:36.224326+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 4587, 'log_dups_size': 0, 'ondisk_log_size': 4587, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18755584, 'num_objects': 4579, 'num_object_clones': 0, 'num_object_copies': 9158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4579, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1005, 'num_write_kb': 4020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 662}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:39.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:53:39.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'4509", 'reported_seq': 4643, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:36.116244+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:53:36.116244+0000', 'last_peered': '2024-09-10T15:53:36.116244+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:53:36.116244+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 4509, 'log_dups_size': 0, 'ondisk_log_size': 4509, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18452480, 'num_objects': 4505, 'num_object_clones': 0, 'num_object_copies': 9010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4505, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1115, 'num_write_kb': 4460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 628}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:39.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:53:39.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'4617", 'reported_seq': 4750, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:36.161036+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:53:36.161036+0000', 'last_peered': '2024-09-10T15:53:36.161036+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:53:36.161036+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 4617, 'log_dups_size': 0, 'ondisk_log_size': 4617, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18862563, 'num_objects': 4606, 'num_object_clones': 0, 'num_object_copies': 9212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4606, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1436, 'num_write_kb': 5744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 603}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:39.181 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:53:39.805 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 62 20 74541 74521 4.69446 2.23438 0.0260226 0.0166306 2024-09-10T15:53:39.879 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:39.880 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/143612 objects degraded (29.700%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":145,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":71806,"data_bytes":294105590,"bytes_used":974921728,"bytes_avail":288935370752,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":143612,"degraded_ratio":0.29700164331671447,"misplaced_objects":3600,"misplaced_total":143612,"misplaced_ratio":0.025067543102247722,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":95881,"recovering_keys_per_sec":0,"num_objects_recovered":281,"num_bytes_recovered":1150976,"num_keys_recovered":0,"write_bytes_sec":2115544,"read_op_per_sec":0,"write_op_per_sec":516},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-10T15:53:24.443468+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:53:39.898 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:53:39.898 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:40.436 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:40.437 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:40.464 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-10T15:53:38.447927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":294105590,"num_objects":71806,"num_object_clones":0,"num_object_copies":143612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3600,"num_objects_unfound":0,"num_objects_dirty":71806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33408,"num_write_kb":133626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":1605651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71888,"ondisk_log_size":71888,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":952072,"kb_used_data":578132,"kb_used_omap":12,"kb_used_meta":373875,"kb_avail":282163448,"statfs":{"total":289910292480,"available":288935370752,"internally_reserved":0,"allocated":592007168,"data_stored":591160708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":382848005},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,7,6,10,12,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25395200,"num_objects":6200,"num_object_clones":0,"num_object_copies":12400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-281,"num_objects_unfound":0,"num_objects_dirty":6200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6200,"num_write_kb":24800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":1150976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004096"},"pg_stats":[{"pgid":"2.f","version":"44'4611","reported_seq":4681,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.198717+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:36.198717+0000","last_peered":"2024-09-10T15:53:36.198717+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:36.198717+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4611,"log_dups_size":0,"ondisk_log_size":4611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18886656,"num_objects":4611,"num_object_clones":0,"num_object_copies":9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4611,"num_write_kb":18444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":638},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4515","reported_seq":4616,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:34.289557+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:34.289557+0000","last_peered":"2024-09-10T15:53:34.289557+0000","last_clean":"2024-09-10T15:53:34.289557+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:34.289557+0000","last_undegraded":"2024-09-10T15:53:34.289557+0000","last_fullsized":"2024-09-10T15:53:34.289557+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4515,"log_dups_size":0,"ondisk_log_size":4515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18468864,"num_objects":4509,"num_object_clones":0,"num_object_copies":9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2851,"num_write_kb":11404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4354","reported_seq":4472,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.144079+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:34.144079+0000","last_peered":"2024-09-10T15:53:34.144079+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:34.144079+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":4354,"log_dups_size":0,"ondisk_log_size":4354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17817600,"num_objects":4350,"num_object_clones":0,"num_object_copies":8700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2622,"num_write_kb":10488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":548}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4460","reported_seq":4534,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:34.260396+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:34.260396+0000","last_peered":"2024-09-10T15:53:34.260396+0000","last_clean":"2024-09-10T15:53:34.260396+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:34.260396+0000","last_undegraded":"2024-09-10T15:53:34.260396+0000","last_fullsized":"2024-09-10T15:53:34.260396+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18268160,"num_objects":4460,"num_object_clones":0,"num_object_copies":8920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4460,"num_write_kb":17840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4458","reported_seq":4578,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:36.215124+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:36.215124+0000","last_peered":"2024-09-10T15:53:36.215124+0000","last_clean":"2024-09-10T15:53:36.215124+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:36.215124+0000","last_undegraded":"2024-09-10T15:53:36.215124+0000","last_fullsized":"2024-09-10T15:53:36.215124+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4458,"log_dups_size":0,"ondisk_log_size":4458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1529,"num_write_kb":6116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4486","reported_seq":4614,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.233182+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:36.233182+0000","last_peered":"2024-09-10T15:53:36.233182+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:36.233182+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4486,"log_dups_size":0,"ondisk_log_size":4486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18354176,"num_objects":4481,"num_object_clones":0,"num_object_copies":8962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1204,"num_write_kb":4816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":613},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4456","reported_seq":4591,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.199134+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:36.199134+0000","last_peered":"2024-09-10T15:53:36.199134+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:36.199134+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4456,"log_dups_size":0,"ondisk_log_size":4456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18227200,"num_objects":4450,"num_object_clones":0,"num_object_copies":8900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1106,"num_write_kb":4424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":616},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4565","reported_seq":4691,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:36.225670+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:36.225670+0000","last_peered":"2024-09-10T15:53:36.225670+0000","last_clean":"2024-09-10T15:53:36.225670+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:36.225670+0000","last_undegraded":"2024-09-10T15:53:36.225670+0000","last_fullsized":"2024-09-10T15:53:36.225670+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4565,"log_dups_size":0,"ondisk_log_size":4565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18690048,"num_objects":4563,"num_object_clones":0,"num_object_copies":9126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1324,"num_write_kb":5296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4542","reported_seq":5446,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.301107+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:34.301107+0000","last_peered":"2024-09-10T15:53:34.301107+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:34.301107+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4542,"log_dups_size":0,"ondisk_log_size":4542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18575360,"num_objects":4535,"num_object_clones":0,"num_object_copies":9070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3600,"num_objects_unfound":0,"num_objects_dirty":4535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2519,"num_write_kb":10076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":1597440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3600},{"shards":"2","objects":933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4503","reported_seq":4624,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.233175+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:36.233175+0000","last_peered":"2024-09-10T15:53:36.233175+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:36.233175+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4503,"log_dups_size":0,"ondisk_log_size":4503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18411520,"num_objects":4495,"num_object_clones":0,"num_object_copies":8990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1367,"num_write_kb":5468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":632},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4368","reported_seq":4487,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.228530+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:34.228530+0000","last_peered":"2024-09-10T15:53:34.228530+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:34.228530+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":4368,"log_dups_size":0,"ondisk_log_size":4368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17870848,"num_objects":4363,"num_object_clones":0,"num_object_copies":8726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":9368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":531}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4379","reported_seq":4500,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:34.284987+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:34.284987+0000","last_peered":"2024-09-10T15:53:34.284987+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:34.284987+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":4379,"log_dups_size":0,"ondisk_log_size":4379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17911808,"num_objects":4373,"num_object_clones":0,"num_object_copies":8746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2248,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":544}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4475","reported_seq":4596,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.236503+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:36.236503+0000","last_peered":"2024-09-10T15:53:36.236503+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:36.236503+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4475,"log_dups_size":0,"ondisk_log_size":4475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1667,"num_write_kb":6668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":640},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4587","reported_seq":4745,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.224326+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:36.224326+0000","last_peered":"2024-09-10T15:53:36.224326+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:36.224326+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18755584,"num_objects":4579,"num_object_clones":0,"num_object_copies":9158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1005,"num_write_kb":4020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":662},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4509","reported_seq":4643,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.116244+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:36.116244+0000","last_peered":"2024-09-10T15:53:36.116244+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:36.116244+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4509,"log_dups_size":0,"ondisk_log_size":4509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18452480,"num_objects":4505,"num_object_clones":0,"num_object_copies":9010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1115,"num_write_kb":4460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":628},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4617","reported_seq":4750,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:36.161036+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:36.161036+0000","last_peered":"2024-09-10T15:53:36.161036+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:36.161036+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4617,"log_dups_size":0,"ondisk_log_size":4617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18862563,"num_objects":4606,"num_object_clones":0,"num_object_copies":9212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1436,"num_write_kb":5744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":603},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":294105571,"num_objects":71804,"num_object_clones":0,"num_object_copies":143608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3600,"num_objects_unfound":0,"num_objects_dirty":71804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33406,"num_write_kb":133624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":594640896,"data_stored":594550571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71885,"ondisk_log_size":71885,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626438,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123492,"kb_used_data":33556,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94248348,"statfs":{"total":96636764160,"available":96510308352,"internally_reserved":0,"allocated":34361344,"data_stored":34026200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92073646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149721,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422336,"kb_used_data":278252,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93949504,"statfs":{"total":96636764160,"available":96204292096,"internally_reserved":0,"allocated":284930048,"data_stored":284674390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":147517101},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406244,"kb_used_data":266324,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93965596,"statfs":{"total":96636764160,"available":96220770304,"internally_reserved":0,"allocated":272715776,"data_stored":272460118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,0,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285868032,"data_stored":285864419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35893248,"data_stored":35810149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":272879616,"data_stored":272876003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:40.805 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 63 20 75147 75127 4.65752 2.36719 0.0134376 0.0167642 2024-09-10T15:53:41.808 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 64 20 75816 75796 4.62537 2.61328 0.0118887 0.0168821 2024-09-10T15:53:42.808 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 65 20 76475 76455 4.59381 2.57422 0.0145807 0.0169999 2024-09-10T15:53:43.466 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:43.808 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 66 20 77096 77076 4.56096 2.42578 0.054806 0.0171202 2024-09-10T15:53:44.034 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:44.035 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:44.057 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-10T15:53:42.448753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":307085814,"num_objects":74975,"num_object_clones":0,"num_object_copies":149950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3478,"num_objects_unfound":0,"num_objects_dirty":74975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36577,"num_write_kb":146302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":2105363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75057,"ondisk_log_size":75057,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":976140,"kb_used_data":593884,"kb_used_omap":12,"kb_used_meta":382195,"kb_avail":282139380,"statfs":{"total":289910292480,"available":288910725120,"internally_reserved":0,"allocated":608137216,"data_stored":607290756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":391367685},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,6,8,13,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33550336,"num_objects":8191,"num_object_clones":0,"num_object_copies":16382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-263,"num_objects_unfound":0,"num_objects_dirty":8191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8191,"num_write_kb":32764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":1077248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002529"},"pg_stats":[{"pgid":"2.f","version":"44'4797","reported_seq":4868,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.238655+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:41.238655+0000","last_peered":"2024-09-10T15:53:41.238655+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:41.238655+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4797,"log_dups_size":0,"ondisk_log_size":4797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19648512,"num_objects":4797,"num_object_clones":0,"num_object_copies":9594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4797,"num_write_kb":19188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":824},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4697","reported_seq":4799,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:39.267331+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:39.267331+0000","last_peered":"2024-09-10T15:53:39.267331+0000","last_clean":"2024-09-10T15:53:39.267331+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:39.267331+0000","last_undegraded":"2024-09-10T15:53:39.267331+0000","last_fullsized":"2024-09-10T15:53:39.267331+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4697,"log_dups_size":0,"ondisk_log_size":4697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19214336,"num_objects":4691,"num_object_clones":0,"num_object_copies":9382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3033,"num_write_kb":12132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4558","reported_seq":4676,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:39.280320+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:39.280320+0000","last_peered":"2024-09-10T15:53:39.280320+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:39.280320+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":4558,"log_dups_size":0,"ondisk_log_size":4558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18653184,"num_objects":4554,"num_object_clones":0,"num_object_copies":9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2826,"num_write_kb":11304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":752}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4625","reported_seq":4699,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:39.267436+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:39.267436+0000","last_peered":"2024-09-10T15:53:39.267436+0000","last_clean":"2024-09-10T15:53:39.267436+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:39.267436+0000","last_undegraded":"2024-09-10T15:53:39.267436+0000","last_fullsized":"2024-09-10T15:53:39.267436+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4625,"log_dups_size":0,"ondisk_log_size":4625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18944000,"num_objects":4625,"num_object_clones":0,"num_object_copies":9250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4625,"num_write_kb":18500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4661","reported_seq":4782,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:41.258595+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:41.258595+0000","last_peered":"2024-09-10T15:53:41.258595+0000","last_clean":"2024-09-10T15:53:41.258595+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:41.258595+0000","last_undegraded":"2024-09-10T15:53:41.258595+0000","last_fullsized":"2024-09-10T15:53:41.258595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4661,"log_dups_size":0,"ondisk_log_size":4661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19062784,"num_objects":4654,"num_object_clones":0,"num_object_copies":9308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1732,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4664","reported_seq":4790,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.224094+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:41.224094+0000","last_peered":"2024-09-10T15:53:41.224094+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:41.224094+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4664,"log_dups_size":0,"ondisk_log_size":4664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19083264,"num_objects":4659,"num_object_clones":0,"num_object_copies":9318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1382,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":791},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4652","reported_seq":4786,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.207456+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:41.207456+0000","last_peered":"2024-09-10T15:53:41.207456+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:41.207456+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4652,"log_dups_size":0,"ondisk_log_size":4652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19030016,"num_objects":4646,"num_object_clones":0,"num_object_copies":9292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1302,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":812},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4760","reported_seq":4886,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:41.249113+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:41.249113+0000","last_peered":"2024-09-10T15:53:41.249113+0000","last_clean":"2024-09-10T15:53:41.249113+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:41.249113+0000","last_undegraded":"2024-09-10T15:53:41.249113+0000","last_fullsized":"2024-09-10T15:53:41.249113+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4760,"log_dups_size":0,"ondisk_log_size":4760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19488768,"num_objects":4758,"num_object_clones":0,"num_object_copies":9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1519,"num_write_kb":6076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4768","reported_seq":5916,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:39.248703+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:39.248703+0000","last_peered":"2024-09-10T15:53:39.248703+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:39.248703+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4768,"log_dups_size":0,"ondisk_log_size":4768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19501056,"num_objects":4761,"num_object_clones":0,"num_object_copies":9522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3478,"num_objects_unfound":0,"num_objects_dirty":4761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2745,"num_write_kb":10980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":512,"num_bytes_recovered":2097152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3478},{"shards":"2","objects":1281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4696","reported_seq":4817,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.179850+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:41.179850+0000","last_peered":"2024-09-10T15:53:41.179850+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:41.179850+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19202048,"num_objects":4688,"num_object_clones":0,"num_object_copies":9376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1560,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":825},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4587","reported_seq":4706,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:39.226942+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:39.226942+0000","last_peered":"2024-09-10T15:53:39.226942+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:39.226942+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18767872,"num_objects":4582,"num_object_clones":0,"num_object_copies":9164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2561,"num_write_kb":10244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":750}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4582","reported_seq":4703,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:39.189660+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:39.189660+0000","last_peered":"2024-09-10T15:53:39.189660+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:39.189660+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":4582,"log_dups_size":0,"ondisk_log_size":4582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18743296,"num_objects":4576,"num_object_clones":0,"num_object_copies":9152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2451,"num_write_kb":9804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":747}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4675","reported_seq":4797,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.230230+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:41.230230+0000","last_peered":"2024-09-10T15:53:41.230230+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:41.230230+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4675,"log_dups_size":0,"ondisk_log_size":4675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19140608,"num_objects":4673,"num_object_clones":0,"num_object_copies":9346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1867,"num_write_kb":7468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":840},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4787","reported_seq":4945,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.156752+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:41.156752+0000","last_peered":"2024-09-10T15:53:41.156752+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:41.156752+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4787,"log_dups_size":0,"ondisk_log_size":4787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19574784,"num_objects":4779,"num_object_clones":0,"num_object_copies":9558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1205,"num_write_kb":4820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":862},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4708","reported_seq":4842,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.265749+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:41.265749+0000","last_peered":"2024-09-10T15:53:41.265749+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:41.265749+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1314,"num_write_kb":5256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":827},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4837","reported_seq":4970,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.260436+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:41.260436+0000","last_peered":"2024-09-10T15:53:41.260436+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:41.260436+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4837,"log_dups_size":0,"ondisk_log_size":4837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19763683,"num_objects":4826,"num_object_clones":0,"num_object_copies":9652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1656,"num_write_kb":6624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":823},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":307085795,"num_objects":74973,"num_object_clones":0,"num_object_copies":149946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3478,"num_objects_unfound":0,"num_objects_dirty":74973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36575,"num_write_kb":146300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":2105344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":610811904,"data_stored":610721579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75054,"ondisk_log_size":75054,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626438,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123492,"kb_used_data":33556,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94248348,"statfs":{"total":96636764160,"available":96510308352,"internally_reserved":0,"allocated":34361344,"data_stored":34026200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92073646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149722,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435784,"kb_used_data":287544,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93936056,"statfs":{"total":96636764160,"available":96190521344,"internally_reserved":0,"allocated":294445056,"data_stored":294189398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":151776941},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953492,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416864,"kb_used_data":272784,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93954976,"statfs":{"total":96636764160,"available":96209895424,"internally_reserved":0,"allocated":279330816,"data_stored":279075158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":295387136,"data_stored":295383523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35893248,"data_stored":35810149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":279531520,"data_stored":279527907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:44.059 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:44.602 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:44.602 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:44.626 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-10T15:53:44.449206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":311833078,"num_objects":76134,"num_object_clones":0,"num_object_copies":152268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3341,"num_objects_unfound":0,"num_objects_dirty":76134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37736,"num_write_kb":150938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":653,"num_bytes_recovered":2666515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76216,"ondisk_log_size":76216,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":994188,"kb_used_data":611932,"kb_used_omap":12,"kb_used_meta":382195,"kb_avail":282121332,"statfs":{"total":289910292480,"available":288892243968,"internally_reserved":0,"allocated":626618368,"data_stored":625771908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":391367685},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,11,4,12,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-400,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7377,"num_write_kb":29508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":1638400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002580"},"pg_stats":[{"pgid":"2.f","version":"44'4797","reported_seq":4868,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.238655+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:41.238655+0000","last_peered":"2024-09-10T15:53:41.238655+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:41.238655+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4797,"log_dups_size":0,"ondisk_log_size":4797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19648512,"num_objects":4797,"num_object_clones":0,"num_object_copies":9594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4797,"num_write_kb":19188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":824},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4917","reported_seq":5019,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:44.220414+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:44.220414+0000","last_peered":"2024-09-10T15:53:44.220414+0000","last_clean":"2024-09-10T15:53:44.220414+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:44.220414+0000","last_undegraded":"2024-09-10T15:53:44.220414+0000","last_fullsized":"2024-09-10T15:53:44.220414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4917,"log_dups_size":0,"ondisk_log_size":4917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20115456,"num_objects":4911,"num_object_clones":0,"num_object_copies":9822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3253,"num_write_kb":13012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4755","reported_seq":4873,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.319505+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:44.319505+0000","last_peered":"2024-09-10T15:53:44.319505+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:44.319505+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":4755,"log_dups_size":0,"ondisk_log_size":4755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19460096,"num_objects":4751,"num_object_clones":0,"num_object_copies":9502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3023,"num_write_kb":12092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":949}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4802","reported_seq":4876,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:44.263236+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:44.263236+0000","last_peered":"2024-09-10T15:53:44.263236+0000","last_clean":"2024-09-10T15:53:44.263236+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:44.263236+0000","last_undegraded":"2024-09-10T15:53:44.263236+0000","last_fullsized":"2024-09-10T15:53:44.263236+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19668992,"num_objects":4802,"num_object_clones":0,"num_object_copies":9604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4802,"num_write_kb":19208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4661","reported_seq":4782,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:41.258595+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:41.258595+0000","last_peered":"2024-09-10T15:53:41.258595+0000","last_clean":"2024-09-10T15:53:41.258595+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:41.258595+0000","last_undegraded":"2024-09-10T15:53:41.258595+0000","last_fullsized":"2024-09-10T15:53:41.258595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4661,"log_dups_size":0,"ondisk_log_size":4661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19062784,"num_objects":4654,"num_object_clones":0,"num_object_copies":9308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1732,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4664","reported_seq":4790,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.224094+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:41.224094+0000","last_peered":"2024-09-10T15:53:41.224094+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:41.224094+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4664,"log_dups_size":0,"ondisk_log_size":4664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19083264,"num_objects":4659,"num_object_clones":0,"num_object_copies":9318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1382,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":791},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4652","reported_seq":4786,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.207456+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:41.207456+0000","last_peered":"2024-09-10T15:53:41.207456+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:41.207456+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4652,"log_dups_size":0,"ondisk_log_size":4652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19030016,"num_objects":4646,"num_object_clones":0,"num_object_copies":9292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1302,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":812},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4760","reported_seq":4886,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:41.249113+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:41.249113+0000","last_peered":"2024-09-10T15:53:41.249113+0000","last_clean":"2024-09-10T15:53:41.249113+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:41.249113+0000","last_undegraded":"2024-09-10T15:53:41.249113+0000","last_fullsized":"2024-09-10T15:53:41.249113+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4760,"log_dups_size":0,"ondisk_log_size":4760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19488768,"num_objects":4758,"num_object_clones":0,"num_object_copies":9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1519,"num_write_kb":6076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4958","reported_seq":6380,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.304176+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:44.304176+0000","last_peered":"2024-09-10T15:53:44.304176+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:44.304176+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4958,"log_dups_size":0,"ondisk_log_size":4958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20279296,"num_objects":4951,"num_object_clones":0,"num_object_copies":9902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3341,"num_objects_unfound":0,"num_objects_dirty":4951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2935,"num_write_kb":11740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":649,"num_bytes_recovered":2658304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3341},{"shards":"2","objects":1608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4696","reported_seq":4817,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.179850+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:41.179850+0000","last_peered":"2024-09-10T15:53:41.179850+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:41.179850+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19202048,"num_objects":4688,"num_object_clones":0,"num_object_copies":9376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1560,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":825},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4776","reported_seq":4895,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.256585+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:44.256585+0000","last_peered":"2024-09-10T15:53:44.256585+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:44.256585+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19542016,"num_objects":4771,"num_object_clones":0,"num_object_copies":9542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2750,"num_write_kb":11000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4768","reported_seq":4889,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.200173+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:44.200173+0000","last_peered":"2024-09-10T15:53:44.200173+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:44.200173+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":4768,"log_dups_size":0,"ondisk_log_size":4768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19505152,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2637,"num_write_kb":10548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":933}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4675","reported_seq":4797,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.230230+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:41.230230+0000","last_peered":"2024-09-10T15:53:41.230230+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:41.230230+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4675,"log_dups_size":0,"ondisk_log_size":4675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19140608,"num_objects":4673,"num_object_clones":0,"num_object_copies":9346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1867,"num_write_kb":7468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":840},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4787","reported_seq":4945,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.156752+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:41.156752+0000","last_peered":"2024-09-10T15:53:41.156752+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:41.156752+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4787,"log_dups_size":0,"ondisk_log_size":4787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19574784,"num_objects":4779,"num_object_clones":0,"num_object_copies":9558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1205,"num_write_kb":4820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":862},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4708","reported_seq":4842,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.265749+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:41.265749+0000","last_peered":"2024-09-10T15:53:41.265749+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:41.265749+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1314,"num_write_kb":5256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":827},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4837","reported_seq":4970,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.260436+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:41.260436+0000","last_peered":"2024-09-10T15:53:41.260436+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:41.260436+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4837,"log_dups_size":0,"ondisk_log_size":4837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19763683,"num_objects":4826,"num_object_clones":0,"num_object_copies":9652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1656,"num_write_kb":6624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":823},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":311833059,"num_objects":76132,"num_object_clones":0,"num_object_copies":152264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3341,"num_objects_unfound":0,"num_objects_dirty":76132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37734,"num_write_kb":150936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":651,"num_bytes_recovered":2666496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":627445760,"data_stored":627355435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76213,"ondisk_log_size":76213,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626439,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131284,"kb_used_data":45508,"kb_used_omap":4,"kb_used_meta":85755,"kb_avail":94240556,"statfs":{"total":96636764160,"available":96502329344,"internally_reserved":0,"allocated":46600192,"data_stored":46265048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":87813806},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,6,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149722,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435784,"kb_used_data":287544,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93936056,"statfs":{"total":96636764160,"available":96190521344,"internally_reserved":0,"allocated":294445056,"data_stored":294189398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":151776941},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427120,"kb_used_data":278880,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93944720,"statfs":{"total":96636764160,"available":96199393280,"internally_reserved":0,"allocated":285573120,"data_stored":285317462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":295387136,"data_stored":295383523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":46260224,"data_stored":46177125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":285798400,"data_stored":285794787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:44.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:53:44.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'4797", 'reported_seq': 4868, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:41.238655+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:53:41.238655+0000', 'last_peered': '2024-09-10T15:53:41.238655+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:53:41.238655+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 4797, 'log_dups_size': 0, 'ondisk_log_size': 4797, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19648512, 'num_objects': 4797, 'num_object_clones': 0, 'num_object_copies': 9594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4797, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4797, 'num_write_kb': 19188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 824}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:44.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:53:44.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'4755", 'reported_seq': 4873, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:44.319505+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:53:44.319505+0000', 'last_peered': '2024-09-10T15:53:44.319505+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:53:44.319505+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 4755, 'log_dups_size': 0, 'ondisk_log_size': 4755, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19460096, 'num_objects': 4751, 'num_object_clones': 0, 'num_object_copies': 9502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4751, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3023, 'num_write_kb': 12092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 949}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:44.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:53:44.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'4664", 'reported_seq': 4790, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:41.224094+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:53:41.224094+0000', 'last_peered': '2024-09-10T15:53:41.224094+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:53:41.224094+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 4664, 'log_dups_size': 0, 'ondisk_log_size': 4664, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19083264, 'num_objects': 4659, 'num_object_clones': 0, 'num_object_copies': 9318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4659, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1382, 'num_write_kb': 5528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 791}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:44.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:53:44.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'4652", 'reported_seq': 4786, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:41.207456+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:53:41.207456+0000', 'last_peered': '2024-09-10T15:53:41.207456+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:53:41.207456+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 4652, 'log_dups_size': 0, 'ondisk_log_size': 4652, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19030016, 'num_objects': 4646, 'num_object_clones': 0, 'num_object_copies': 9292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4646, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1302, 'num_write_kb': 5208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 812}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:44.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:53:44.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'4958", 'reported_seq': 6380, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:44.304176+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:53:44.304176+0000', 'last_peered': '2024-09-10T15:53:44.304176+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:53:44.304176+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 4958, 'log_dups_size': 0, 'ondisk_log_size': 4958, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20279296, 'num_objects': 4951, 'num_object_clones': 0, 'num_object_copies': 9902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3341, 'num_objects_unfound': 0, 'num_objects_dirty': 4951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2935, 'num_write_kb': 11740, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 649, 'num_bytes_recovered': 2658304, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3341}, {'shards': '2', 'objects': 1608}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:44.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:53:44.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'4696", 'reported_seq': 4817, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:41.179850+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:53:41.179850+0000', 'last_peered': '2024-09-10T15:53:41.179850+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:53:41.179850+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 4696, 'log_dups_size': 0, 'ondisk_log_size': 4696, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19202048, 'num_objects': 4688, 'num_object_clones': 0, 'num_object_copies': 9376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4688, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1560, 'num_write_kb': 6240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 825}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:44.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:53:44.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'4776", 'reported_seq': 4895, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:44.256585+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:53:44.256585+0000', 'last_peered': '2024-09-10T15:53:44.256585+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:53:44.256585+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 4776, 'log_dups_size': 0, 'ondisk_log_size': 4776, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19542016, 'num_objects': 4771, 'num_object_clones': 0, 'num_object_copies': 9542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4771, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2750, 'num_write_kb': 11000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 939}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:44.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:53:44.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'4768", 'reported_seq': 4889, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:44.200173+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:53:44.200173+0000', 'last_peered': '2024-09-10T15:53:44.200173+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:53:44.200173+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 4768, 'log_dups_size': 0, 'ondisk_log_size': 4768, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19505152, 'num_objects': 4762, 'num_object_clones': 0, 'num_object_copies': 9524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2637, 'num_write_kb': 10548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 933}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:44.629 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:53:44.629 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'4675", 'reported_seq': 4797, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:41.230230+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:53:41.230230+0000', 'last_peered': '2024-09-10T15:53:41.230230+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:53:41.230230+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 4675, 'log_dups_size': 0, 'ondisk_log_size': 4675, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19140608, 'num_objects': 4673, 'num_object_clones': 0, 'num_object_copies': 9346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4673, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1867, 'num_write_kb': 7468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 840}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:44.629 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:53:44.629 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'4787", 'reported_seq': 4945, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:41.156752+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:53:41.156752+0000', 'last_peered': '2024-09-10T15:53:41.156752+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:53:41.156752+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 4787, 'log_dups_size': 0, 'ondisk_log_size': 4787, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19574784, 'num_objects': 4779, 'num_object_clones': 0, 'num_object_copies': 9558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4779, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1205, 'num_write_kb': 4820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 862}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:44.629 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:53:44.629 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'4708", 'reported_seq': 4842, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:41.265749+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:53:41.265749+0000', 'last_peered': '2024-09-10T15:53:41.265749+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:53:41.265749+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 4708, 'log_dups_size': 0, 'ondisk_log_size': 4708, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19267584, 'num_objects': 4704, 'num_object_clones': 0, 'num_object_copies': 9408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4704, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1314, 'num_write_kb': 5256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 827}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:44.629 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:53:44.629 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'4837", 'reported_seq': 4970, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:41.260436+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:53:41.260436+0000', 'last_peered': '2024-09-10T15:53:41.260436+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:53:41.260436+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 4837, 'log_dups_size': 0, 'ondisk_log_size': 4837, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19763683, 'num_objects': 4826, 'num_object_clones': 0, 'num_object_copies': 9652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1656, 'num_write_kb': 6624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 823}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:44.630 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:53:44.809 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 67 20 77664 77644 4.526 2.21875 0.068591 0.0172522 2024-09-10T15:53:45.323 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:45.323 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/152268 objects degraded (28.012%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":151,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":76134,"data_bytes":311833078,"bytes_used":1018048512,"bytes_avail":288892243968,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":152268,"degraded_ratio":0.28011794993038591,"misplaced_objects":3341,"misplaced_total":152268,"misplaced_ratio":0.021941576693724225,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":136503,"recovering_keys_per_sec":0,"num_objects_recovered":400,"num_bytes_recovered":1638400,"num_keys_recovered":0,"write_bytes_sec":2517474,"read_op_per_sec":0,"write_op_per_sec":614},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:53:42.448843+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:53:45.354 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:53:45.354 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:45.808 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 68 20 78246 78226 4.49288 2.27344 0.0447972 0.0173781 2024-09-10T15:53:45.910 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:45.911 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:45.941 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-10T15:53:44.449206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":311833078,"num_objects":76134,"num_object_clones":0,"num_object_copies":152268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3341,"num_objects_unfound":0,"num_objects_dirty":76134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37736,"num_write_kb":150938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":653,"num_bytes_recovered":2666515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76216,"ondisk_log_size":76216,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":994188,"kb_used_data":611932,"kb_used_omap":12,"kb_used_meta":382195,"kb_avail":282121332,"statfs":{"total":289910292480,"available":288892243968,"internally_reserved":0,"allocated":626618368,"data_stored":625771908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":391367685},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,11,4,12,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-400,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7377,"num_write_kb":29508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":1638400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002580"},"pg_stats":[{"pgid":"2.f","version":"44'4797","reported_seq":4868,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.238655+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:41.238655+0000","last_peered":"2024-09-10T15:53:41.238655+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:41.238655+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4797,"log_dups_size":0,"ondisk_log_size":4797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19648512,"num_objects":4797,"num_object_clones":0,"num_object_copies":9594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4797,"num_write_kb":19188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":824},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4917","reported_seq":5019,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:44.220414+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:44.220414+0000","last_peered":"2024-09-10T15:53:44.220414+0000","last_clean":"2024-09-10T15:53:44.220414+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:44.220414+0000","last_undegraded":"2024-09-10T15:53:44.220414+0000","last_fullsized":"2024-09-10T15:53:44.220414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4917,"log_dups_size":0,"ondisk_log_size":4917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20115456,"num_objects":4911,"num_object_clones":0,"num_object_copies":9822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3253,"num_write_kb":13012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4755","reported_seq":4873,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.319505+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:44.319505+0000","last_peered":"2024-09-10T15:53:44.319505+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:44.319505+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":4755,"log_dups_size":0,"ondisk_log_size":4755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19460096,"num_objects":4751,"num_object_clones":0,"num_object_copies":9502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3023,"num_write_kb":12092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":949}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4802","reported_seq":4876,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:44.263236+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:44.263236+0000","last_peered":"2024-09-10T15:53:44.263236+0000","last_clean":"2024-09-10T15:53:44.263236+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:44.263236+0000","last_undegraded":"2024-09-10T15:53:44.263236+0000","last_fullsized":"2024-09-10T15:53:44.263236+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19668992,"num_objects":4802,"num_object_clones":0,"num_object_copies":9604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4802,"num_write_kb":19208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4661","reported_seq":4782,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:41.258595+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:41.258595+0000","last_peered":"2024-09-10T15:53:41.258595+0000","last_clean":"2024-09-10T15:53:41.258595+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:41.258595+0000","last_undegraded":"2024-09-10T15:53:41.258595+0000","last_fullsized":"2024-09-10T15:53:41.258595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4661,"log_dups_size":0,"ondisk_log_size":4661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19062784,"num_objects":4654,"num_object_clones":0,"num_object_copies":9308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1732,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4664","reported_seq":4790,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.224094+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:41.224094+0000","last_peered":"2024-09-10T15:53:41.224094+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:41.224094+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4664,"log_dups_size":0,"ondisk_log_size":4664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19083264,"num_objects":4659,"num_object_clones":0,"num_object_copies":9318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1382,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":791},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4652","reported_seq":4786,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.207456+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:41.207456+0000","last_peered":"2024-09-10T15:53:41.207456+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:41.207456+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4652,"log_dups_size":0,"ondisk_log_size":4652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19030016,"num_objects":4646,"num_object_clones":0,"num_object_copies":9292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1302,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":812},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4760","reported_seq":4886,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:41.249113+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:41.249113+0000","last_peered":"2024-09-10T15:53:41.249113+0000","last_clean":"2024-09-10T15:53:41.249113+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:41.249113+0000","last_undegraded":"2024-09-10T15:53:41.249113+0000","last_fullsized":"2024-09-10T15:53:41.249113+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4760,"log_dups_size":0,"ondisk_log_size":4760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19488768,"num_objects":4758,"num_object_clones":0,"num_object_copies":9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1519,"num_write_kb":6076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4958","reported_seq":6380,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.304176+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:44.304176+0000","last_peered":"2024-09-10T15:53:44.304176+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:44.304176+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4958,"log_dups_size":0,"ondisk_log_size":4958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20279296,"num_objects":4951,"num_object_clones":0,"num_object_copies":9902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3341,"num_objects_unfound":0,"num_objects_dirty":4951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2935,"num_write_kb":11740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":649,"num_bytes_recovered":2658304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3341},{"shards":"2","objects":1608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4696","reported_seq":4817,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.179850+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:41.179850+0000","last_peered":"2024-09-10T15:53:41.179850+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:41.179850+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19202048,"num_objects":4688,"num_object_clones":0,"num_object_copies":9376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1560,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":825},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4776","reported_seq":4895,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.256585+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:44.256585+0000","last_peered":"2024-09-10T15:53:44.256585+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:44.256585+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19542016,"num_objects":4771,"num_object_clones":0,"num_object_copies":9542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2750,"num_write_kb":11000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4768","reported_seq":4889,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.200173+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:44.200173+0000","last_peered":"2024-09-10T15:53:44.200173+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:44.200173+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":4768,"log_dups_size":0,"ondisk_log_size":4768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19505152,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2637,"num_write_kb":10548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":933}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4675","reported_seq":4797,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.230230+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:41.230230+0000","last_peered":"2024-09-10T15:53:41.230230+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:41.230230+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4675,"log_dups_size":0,"ondisk_log_size":4675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19140608,"num_objects":4673,"num_object_clones":0,"num_object_copies":9346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1867,"num_write_kb":7468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":840},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4787","reported_seq":4945,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.156752+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:41.156752+0000","last_peered":"2024-09-10T15:53:41.156752+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:41.156752+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4787,"log_dups_size":0,"ondisk_log_size":4787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19574784,"num_objects":4779,"num_object_clones":0,"num_object_copies":9558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1205,"num_write_kb":4820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":862},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4708","reported_seq":4842,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.265749+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:41.265749+0000","last_peered":"2024-09-10T15:53:41.265749+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:41.265749+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1314,"num_write_kb":5256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":827},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4837","reported_seq":4970,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:41.260436+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:41.260436+0000","last_peered":"2024-09-10T15:53:41.260436+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:41.260436+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":4837,"log_dups_size":0,"ondisk_log_size":4837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19763683,"num_objects":4826,"num_object_clones":0,"num_object_copies":9652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1656,"num_write_kb":6624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":823},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":311833059,"num_objects":76132,"num_object_clones":0,"num_object_copies":152264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3341,"num_objects_unfound":0,"num_objects_dirty":76132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37734,"num_write_kb":150936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":651,"num_bytes_recovered":2666496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":627445760,"data_stored":627355435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76213,"ondisk_log_size":76213,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626439,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131284,"kb_used_data":45508,"kb_used_omap":4,"kb_used_meta":85755,"kb_avail":94240556,"statfs":{"total":96636764160,"available":96502329344,"internally_reserved":0,"allocated":46600192,"data_stored":46265048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":87813806},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,6,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149722,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435784,"kb_used_data":287544,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93936056,"statfs":{"total":96636764160,"available":96190521344,"internally_reserved":0,"allocated":294445056,"data_stored":294189398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":151776941},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427120,"kb_used_data":278880,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93944720,"statfs":{"total":96636764160,"available":96199393280,"internally_reserved":0,"allocated":285573120,"data_stored":285317462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":295387136,"data_stored":295383523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":46260224,"data_stored":46177125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":285798400,"data_stored":285794787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:46.809 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 69 20 78884 78864 4.46388 2.49219 0.0533918 0.0174946 2024-09-10T15:53:47.809 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 70 20 79549 79529 4.43722 2.59766 0.0182185 0.0175991 2024-09-10T15:53:48.809 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 71 20 80190 80170 4.40999 2.50391 0.0161909 0.0177073 2024-09-10T15:53:48.944 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:49.395 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:49.398 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:49.421 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-10T15:53:48.450148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":319939062,"num_objects":78113,"num_object_clones":0,"num_object_copies":156226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3341,"num_objects_unfound":0,"num_objects_dirty":78113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39715,"num_write_kb":158854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":653,"num_bytes_recovered":2666515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78195,"ondisk_log_size":78195,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1022920,"kb_used_data":630936,"kb_used_omap":12,"kb_used_meta":391923,"kb_avail":282092600,"statfs":{"total":289910292480,"available":288862822400,"internally_reserved":0,"allocated":646078464,"data_stored":645232004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":401329157},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,5,4,7,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25833472,"num_objects":6307,"num_object_clones":0,"num_object_copies":12614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-259,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6307,"num_write_kb":25228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":1060864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002665"},"pg_stats":[{"pgid":"2.f","version":"44'4988","reported_seq":5058,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.281235+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:46.281235+0000","last_peered":"2024-09-10T15:53:46.281235+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:46.281235+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4988,"log_dups_size":0,"ondisk_log_size":4988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20430848,"num_objects":4988,"num_object_clones":0,"num_object_copies":9976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4988,"num_write_kb":19952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1015},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4917","reported_seq":5019,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:44.220414+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:44.220414+0000","last_peered":"2024-09-10T15:53:44.220414+0000","last_clean":"2024-09-10T15:53:44.220414+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:44.220414+0000","last_undegraded":"2024-09-10T15:53:44.220414+0000","last_fullsized":"2024-09-10T15:53:44.220414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4917,"log_dups_size":0,"ondisk_log_size":4917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20115456,"num_objects":4911,"num_object_clones":0,"num_object_copies":9822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3253,"num_write_kb":13012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4755","reported_seq":4873,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.319505+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:44.319505+0000","last_peered":"2024-09-10T15:53:44.319505+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:44.319505+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":4755,"log_dups_size":0,"ondisk_log_size":4755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19460096,"num_objects":4751,"num_object_clones":0,"num_object_copies":9502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3023,"num_write_kb":12092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":949}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4802","reported_seq":4876,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:44.263236+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:44.263236+0000","last_peered":"2024-09-10T15:53:44.263236+0000","last_clean":"2024-09-10T15:53:44.263236+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:44.263236+0000","last_undegraded":"2024-09-10T15:53:44.263236+0000","last_fullsized":"2024-09-10T15:53:44.263236+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19668992,"num_objects":4802,"num_object_clones":0,"num_object_copies":9604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4802,"num_write_kb":19208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4831","reported_seq":4952,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:46.267804+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:46.267804+0000","last_peered":"2024-09-10T15:53:46.267804+0000","last_clean":"2024-09-10T15:53:46.267804+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:46.267804+0000","last_undegraded":"2024-09-10T15:53:46.267804+0000","last_fullsized":"2024-09-10T15:53:46.267804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4831,"log_dups_size":0,"ondisk_log_size":4831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19759104,"num_objects":4824,"num_object_clones":0,"num_object_copies":9648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1902,"num_write_kb":7608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4867","reported_seq":4996,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.266590+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:46.266590+0000","last_peered":"2024-09-10T15:53:46.266590+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:46.266590+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4867,"log_dups_size":0,"ondisk_log_size":4867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19914752,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1585,"num_write_kb":6340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":994},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4846","reported_seq":4981,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.294191+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:46.294191+0000","last_peered":"2024-09-10T15:53:46.294191+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:46.294191+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4846,"log_dups_size":0,"ondisk_log_size":4846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19824640,"num_objects":4840,"num_object_clones":0,"num_object_copies":9680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1496,"num_write_kb":5984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1006},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4959","reported_seq":5085,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:46.245198+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:46.245198+0000","last_peered":"2024-09-10T15:53:46.245198+0000","last_clean":"2024-09-10T15:53:46.245198+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:46.245198+0000","last_undegraded":"2024-09-10T15:53:46.245198+0000","last_fullsized":"2024-09-10T15:53:46.245198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4959,"log_dups_size":0,"ondisk_log_size":4959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20303872,"num_objects":4957,"num_object_clones":0,"num_object_copies":9914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1718,"num_write_kb":6872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4958","reported_seq":6380,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.304176+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:44.304176+0000","last_peered":"2024-09-10T15:53:44.304176+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:44.304176+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4958,"log_dups_size":0,"ondisk_log_size":4958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20279296,"num_objects":4951,"num_object_clones":0,"num_object_copies":9902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3341,"num_objects_unfound":0,"num_objects_dirty":4951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2935,"num_write_kb":11740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":649,"num_bytes_recovered":2658304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3341},{"shards":"2","objects":1608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4896","reported_seq":5017,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.222195+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:46.222195+0000","last_peered":"2024-09-10T15:53:46.222195+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:46.222195+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4896,"log_dups_size":0,"ondisk_log_size":4896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021248,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1760,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1025},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4776","reported_seq":4895,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.256585+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:44.256585+0000","last_peered":"2024-09-10T15:53:44.256585+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:44.256585+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19542016,"num_objects":4771,"num_object_clones":0,"num_object_copies":9542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2750,"num_write_kb":11000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4768","reported_seq":4889,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.200173+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:44.200173+0000","last_peered":"2024-09-10T15:53:44.200173+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:44.200173+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":4768,"log_dups_size":0,"ondisk_log_size":4768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19505152,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2637,"num_write_kb":10548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":933}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4902","reported_seq":5023,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.282339+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:46.282339+0000","last_peered":"2024-09-10T15:53:46.282339+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:46.282339+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20070400,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2094,"num_write_kb":8376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1067},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4990","reported_seq":5145,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.295050+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:46.295050+0000","last_peered":"2024-09-10T15:53:46.295050+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:46.295050+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4990,"log_dups_size":0,"ondisk_log_size":4990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20406272,"num_objects":4982,"num_object_clones":0,"num_object_copies":9964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1408,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1065},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4893","reported_seq":5026,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.313582+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:46.313582+0000","last_peered":"2024-09-10T15:53:46.313582+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:46.313582+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4893,"log_dups_size":0,"ondisk_log_size":4893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1499,"num_write_kb":5996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1012},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5044","reported_seq":5177,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.302598+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:46.302598+0000","last_peered":"2024-09-10T15:53:46.302598+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:46.302598+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5044,"log_dups_size":0,"ondisk_log_size":5044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20611555,"num_objects":5033,"num_object_clones":0,"num_object_copies":10066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1863,"num_write_kb":7452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1030},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":319939043,"num_objects":78111,"num_object_clones":0,"num_object_copies":156222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3341,"num_objects_unfound":0,"num_objects_dirty":78111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39713,"num_write_kb":158852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":651,"num_bytes_recovered":2666496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":647249920,"data_stored":647159595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78192,"ondisk_log_size":78192,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144948,"kb_used_data":55012,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94226892,"statfs":{"total":96636764160,"available":96488337408,"internally_reserved":0,"allocated":56332288,"data_stored":55997144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92073646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149723,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450852,"kb_used_data":297044,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93920988,"statfs":{"total":96636764160,"available":96175091712,"internally_reserved":0,"allocated":304173056,"data_stored":303917398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":157478573},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427120,"kb_used_data":278880,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93944720,"statfs":{"total":96636764160,"available":96199393280,"internally_reserved":0,"allocated":285573120,"data_stored":285317462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":305074176,"data_stored":305070563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56377344,"data_stored":56294245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":285798400,"data_stored":285794787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:49.422 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:49.809 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 72 20 80669 80649 4.37473 1.87109 0.0493844 0.0178494 2024-09-10T15:53:49.991 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:49.996 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:50.022 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-10T15:53:48.450148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":319939062,"num_objects":78113,"num_object_clones":0,"num_object_copies":156226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3341,"num_objects_unfound":0,"num_objects_dirty":78113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39715,"num_write_kb":158854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":653,"num_bytes_recovered":2666515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78195,"ondisk_log_size":78195,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1022920,"kb_used_data":630936,"kb_used_omap":12,"kb_used_meta":391923,"kb_avail":282092600,"statfs":{"total":289910292480,"available":288862822400,"internally_reserved":0,"allocated":646078464,"data_stored":645232004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":401329157},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,5,4,7,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25833472,"num_objects":6307,"num_object_clones":0,"num_object_copies":12614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-259,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6307,"num_write_kb":25228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":1060864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002665"},"pg_stats":[{"pgid":"2.f","version":"44'4988","reported_seq":5058,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.281235+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:46.281235+0000","last_peered":"2024-09-10T15:53:46.281235+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:46.281235+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4988,"log_dups_size":0,"ondisk_log_size":4988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20430848,"num_objects":4988,"num_object_clones":0,"num_object_copies":9976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4988,"num_write_kb":19952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1015},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4917","reported_seq":5019,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:44.220414+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:44.220414+0000","last_peered":"2024-09-10T15:53:44.220414+0000","last_clean":"2024-09-10T15:53:44.220414+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:44.220414+0000","last_undegraded":"2024-09-10T15:53:44.220414+0000","last_fullsized":"2024-09-10T15:53:44.220414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":4917,"log_dups_size":0,"ondisk_log_size":4917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20115456,"num_objects":4911,"num_object_clones":0,"num_object_copies":9822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3253,"num_write_kb":13012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4755","reported_seq":4873,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.319505+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:44.319505+0000","last_peered":"2024-09-10T15:53:44.319505+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:44.319505+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":4755,"log_dups_size":0,"ondisk_log_size":4755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19460096,"num_objects":4751,"num_object_clones":0,"num_object_copies":9502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3023,"num_write_kb":12092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":949}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4802","reported_seq":4876,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:44.263236+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:44.263236+0000","last_peered":"2024-09-10T15:53:44.263236+0000","last_clean":"2024-09-10T15:53:44.263236+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:44.263236+0000","last_undegraded":"2024-09-10T15:53:44.263236+0000","last_fullsized":"2024-09-10T15:53:44.263236+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19668992,"num_objects":4802,"num_object_clones":0,"num_object_copies":9604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4802,"num_write_kb":19208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4831","reported_seq":4952,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:46.267804+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:46.267804+0000","last_peered":"2024-09-10T15:53:46.267804+0000","last_clean":"2024-09-10T15:53:46.267804+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:46.267804+0000","last_undegraded":"2024-09-10T15:53:46.267804+0000","last_fullsized":"2024-09-10T15:53:46.267804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4831,"log_dups_size":0,"ondisk_log_size":4831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19759104,"num_objects":4824,"num_object_clones":0,"num_object_copies":9648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1902,"num_write_kb":7608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4867","reported_seq":4996,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.266590+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:46.266590+0000","last_peered":"2024-09-10T15:53:46.266590+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:46.266590+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4867,"log_dups_size":0,"ondisk_log_size":4867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19914752,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1585,"num_write_kb":6340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":994},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4846","reported_seq":4981,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.294191+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:46.294191+0000","last_peered":"2024-09-10T15:53:46.294191+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:46.294191+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4846,"log_dups_size":0,"ondisk_log_size":4846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19824640,"num_objects":4840,"num_object_clones":0,"num_object_copies":9680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1496,"num_write_kb":5984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1006},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4959","reported_seq":5085,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:46.245198+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:46.245198+0000","last_peered":"2024-09-10T15:53:46.245198+0000","last_clean":"2024-09-10T15:53:46.245198+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:46.245198+0000","last_undegraded":"2024-09-10T15:53:46.245198+0000","last_fullsized":"2024-09-10T15:53:46.245198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4959,"log_dups_size":0,"ondisk_log_size":4959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20303872,"num_objects":4957,"num_object_clones":0,"num_object_copies":9914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1718,"num_write_kb":6872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4958","reported_seq":6380,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.304176+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:44.304176+0000","last_peered":"2024-09-10T15:53:44.304176+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:44.304176+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":4958,"log_dups_size":0,"ondisk_log_size":4958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20279296,"num_objects":4951,"num_object_clones":0,"num_object_copies":9902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3341,"num_objects_unfound":0,"num_objects_dirty":4951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2935,"num_write_kb":11740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":649,"num_bytes_recovered":2658304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3341},{"shards":"2","objects":1608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4896","reported_seq":5017,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.222195+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:46.222195+0000","last_peered":"2024-09-10T15:53:46.222195+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:46.222195+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4896,"log_dups_size":0,"ondisk_log_size":4896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021248,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1760,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1025},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4776","reported_seq":4895,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.256585+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:44.256585+0000","last_peered":"2024-09-10T15:53:44.256585+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:44.256585+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19542016,"num_objects":4771,"num_object_clones":0,"num_object_copies":9542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2750,"num_write_kb":11000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4768","reported_seq":4889,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:44.200173+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:44.200173+0000","last_peered":"2024-09-10T15:53:44.200173+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:44.200173+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":4768,"log_dups_size":0,"ondisk_log_size":4768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19505152,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2637,"num_write_kb":10548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":933}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4902","reported_seq":5023,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.282339+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:46.282339+0000","last_peered":"2024-09-10T15:53:46.282339+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:46.282339+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20070400,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2094,"num_write_kb":8376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1067},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4990","reported_seq":5145,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.295050+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:46.295050+0000","last_peered":"2024-09-10T15:53:46.295050+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:46.295050+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4990,"log_dups_size":0,"ondisk_log_size":4990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20406272,"num_objects":4982,"num_object_clones":0,"num_object_copies":9964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1408,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1065},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4893","reported_seq":5026,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.313582+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:46.313582+0000","last_peered":"2024-09-10T15:53:46.313582+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:46.313582+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4893,"log_dups_size":0,"ondisk_log_size":4893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1499,"num_write_kb":5996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1012},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5044","reported_seq":5177,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.302598+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:46.302598+0000","last_peered":"2024-09-10T15:53:46.302598+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:46.302598+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5044,"log_dups_size":0,"ondisk_log_size":5044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20611555,"num_objects":5033,"num_object_clones":0,"num_object_copies":10066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1863,"num_write_kb":7452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1030},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":319939043,"num_objects":78111,"num_object_clones":0,"num_object_copies":156222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3341,"num_objects_unfound":0,"num_objects_dirty":78111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39713,"num_write_kb":158852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":651,"num_bytes_recovered":2666496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":647249920,"data_stored":647159595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78192,"ondisk_log_size":78192,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144948,"kb_used_data":55012,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94226892,"statfs":{"total":96636764160,"available":96488337408,"internally_reserved":0,"allocated":56332288,"data_stored":55997144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92073646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149723,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450852,"kb_used_data":297044,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93920988,"statfs":{"total":96636764160,"available":96175091712,"internally_reserved":0,"allocated":304173056,"data_stored":303917398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":157478573},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427120,"kb_used_data":278880,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93944720,"statfs":{"total":96636764160,"available":96199393280,"internally_reserved":0,"allocated":285573120,"data_stored":285317462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":305074176,"data_stored":305070563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56377344,"data_stored":56294245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":285798400,"data_stored":285794787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:50.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:53:50.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'4988", 'reported_seq': 5058, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:46.281235+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:53:46.281235+0000', 'last_peered': '2024-09-10T15:53:46.281235+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:53:46.281235+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 4988, 'log_dups_size': 0, 'ondisk_log_size': 4988, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20430848, 'num_objects': 4988, 'num_object_clones': 0, 'num_object_copies': 9976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4988, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4988, 'num_write_kb': 19952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1015}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:50.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:53:50.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'4755", 'reported_seq': 4873, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:44.319505+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:53:44.319505+0000', 'last_peered': '2024-09-10T15:53:44.319505+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:53:44.319505+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 4755, 'log_dups_size': 0, 'ondisk_log_size': 4755, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19460096, 'num_objects': 4751, 'num_object_clones': 0, 'num_object_copies': 9502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4751, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3023, 'num_write_kb': 12092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 949}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:50.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:53:50.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'4867", 'reported_seq': 4996, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:46.266590+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:53:46.266590+0000', 'last_peered': '2024-09-10T15:53:46.266590+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:53:46.266590+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 4867, 'log_dups_size': 0, 'ondisk_log_size': 4867, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19914752, 'num_objects': 4862, 'num_object_clones': 0, 'num_object_copies': 9724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1585, 'num_write_kb': 6340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 994}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:50.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:53:50.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'4846", 'reported_seq': 4981, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:46.294191+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:53:46.294191+0000', 'last_peered': '2024-09-10T15:53:46.294191+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:53:46.294191+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 4846, 'log_dups_size': 0, 'ondisk_log_size': 4846, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19824640, 'num_objects': 4840, 'num_object_clones': 0, 'num_object_copies': 9680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4840, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1496, 'num_write_kb': 5984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1006}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:50.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:53:50.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'4958", 'reported_seq': 6380, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:44.304176+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:53:44.304176+0000', 'last_peered': '2024-09-10T15:53:44.304176+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:53:44.304176+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 4958, 'log_dups_size': 0, 'ondisk_log_size': 4958, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20279296, 'num_objects': 4951, 'num_object_clones': 0, 'num_object_copies': 9902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3341, 'num_objects_unfound': 0, 'num_objects_dirty': 4951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2935, 'num_write_kb': 11740, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 649, 'num_bytes_recovered': 2658304, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3341}, {'shards': '2', 'objects': 1608}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:50.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:53:50.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'4896", 'reported_seq': 5017, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:46.222195+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:53:46.222195+0000', 'last_peered': '2024-09-10T15:53:46.222195+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:53:46.222195+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 4896, 'log_dups_size': 0, 'ondisk_log_size': 4896, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20021248, 'num_objects': 4888, 'num_object_clones': 0, 'num_object_copies': 9776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1760, 'num_write_kb': 7040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1025}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:50.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:53:50.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'4776", 'reported_seq': 4895, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:44.256585+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:53:44.256585+0000', 'last_peered': '2024-09-10T15:53:44.256585+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:53:44.256585+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 4776, 'log_dups_size': 0, 'ondisk_log_size': 4776, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19542016, 'num_objects': 4771, 'num_object_clones': 0, 'num_object_copies': 9542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4771, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2750, 'num_write_kb': 11000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 939}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:50.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:53:50.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'4768", 'reported_seq': 4889, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:44.200173+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:53:44.200173+0000', 'last_peered': '2024-09-10T15:53:44.200173+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:53:44.200173+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 4768, 'log_dups_size': 0, 'ondisk_log_size': 4768, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19505152, 'num_objects': 4762, 'num_object_clones': 0, 'num_object_copies': 9524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2637, 'num_write_kb': 10548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 933}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:50.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:53:50.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'4902", 'reported_seq': 5023, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:46.282339+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:53:46.282339+0000', 'last_peered': '2024-09-10T15:53:46.282339+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:53:46.282339+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 4902, 'log_dups_size': 0, 'ondisk_log_size': 4902, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20070400, 'num_objects': 4900, 'num_object_clones': 0, 'num_object_copies': 9800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2094, 'num_write_kb': 8376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1067}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:50.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:53:50.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'4990", 'reported_seq': 5145, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:46.295050+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:53:46.295050+0000', 'last_peered': '2024-09-10T15:53:46.295050+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:53:46.295050+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 4990, 'log_dups_size': 0, 'ondisk_log_size': 4990, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20406272, 'num_objects': 4982, 'num_object_clones': 0, 'num_object_copies': 9964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1408, 'num_write_kb': 5632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1065}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:50.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:53:50.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'4893", 'reported_seq': 5026, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:46.313582+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:53:46.313582+0000', 'last_peered': '2024-09-10T15:53:46.313582+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:53:46.313582+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 4893, 'log_dups_size': 0, 'ondisk_log_size': 4893, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20025344, 'num_objects': 4889, 'num_object_clones': 0, 'num_object_copies': 9778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1499, 'num_write_kb': 5996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1012}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:50.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:53:50.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'5044", 'reported_seq': 5177, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:46.302598+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:53:46.302598+0000', 'last_peered': '2024-09-10T15:53:46.302598+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:53:46.302598+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 5044, 'log_dups_size': 0, 'ondisk_log_size': 5044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20611555, 'num_objects': 5033, 'num_object_clones': 0, 'num_object_copies': 10066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1863, 'num_write_kb': 7452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1030}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:50.027 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:53:50.694 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:50.694 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/158436 objects degraded (26.921%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":156,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":79218,"data_bytes":324465142,"bytes_used":1040896000,"bytes_avail":288869396480,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":158436,"degraded_ratio":0.26921280517054202,"misplaced_objects":3182,"misplaced_total":158436,"misplaced_ratio":0.020083819333989748,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":142645,"recovering_keys_per_sec":0,"num_objects_recovered":418,"num_bytes_recovered":1712128,"num_keys_recovered":0,"write_bytes_sec":2529400,"read_op_per_sec":0,"write_op_per_sec":617},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:53:42.448843+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:53:50.714 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:53:50.714 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:50.809 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 73 20 81228 81208 4.3447 2.18359 0.029407 0.0179728 2024-09-10T15:53:51.247 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:51.249 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:51.270 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-10T15:53:50.450596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":324465142,"num_objects":79218,"num_object_clones":0,"num_object_copies":158436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3182,"num_objects_unfound":0,"num_objects_dirty":79218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40820,"num_write_kb":163274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":812,"num_bytes_recovered":3317779,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79300,"ondisk_log_size":79300,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1016500,"kb_used_data":636996,"kb_used_omap":12,"kb_used_meta":379443,"kb_avail":282099020,"statfs":{"total":289910292480,"available":288869396480,"internally_reserved":0,"allocated":652283904,"data_stored":651437444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":388549637},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,1,12,5,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30359552,"num_objects":7412,"num_object_clones":0,"num_object_copies":14824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-418,"num_objects_unfound":0,"num_objects_dirty":7412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7412,"num_write_kb":29648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":1712128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002668"},"pg_stats":[{"pgid":"2.f","version":"44'4988","reported_seq":5058,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.281235+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:46.281235+0000","last_peered":"2024-09-10T15:53:46.281235+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:46.281235+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4988,"log_dups_size":0,"ondisk_log_size":4988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20430848,"num_objects":4988,"num_object_clones":0,"num_object_copies":9976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4988,"num_write_kb":19952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1015},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5124","reported_seq":5226,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:49.314530+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:49.314530+0000","last_peered":"2024-09-10T15:53:49.314530+0000","last_clean":"2024-09-10T15:53:49.314530+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:49.314530+0000","last_undegraded":"2024-09-10T15:53:49.314530+0000","last_fullsized":"2024-09-10T15:53:49.314530+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5124,"log_dups_size":0,"ondisk_log_size":5124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20963328,"num_objects":5118,"num_object_clones":0,"num_object_copies":10236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3460,"num_write_kb":13840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4953","reported_seq":5071,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:49.370446+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:49.370446+0000","last_peered":"2024-09-10T15:53:49.370446+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:49.370446+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20271104,"num_objects":4949,"num_object_clones":0,"num_object_copies":9898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3221,"num_write_kb":12884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1147}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4971","reported_seq":5045,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:49.345395+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:49.345395+0000","last_peered":"2024-09-10T15:53:49.345395+0000","last_clean":"2024-09-10T15:53:49.345395+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:49.345395+0000","last_undegraded":"2024-09-10T15:53:49.345395+0000","last_fullsized":"2024-09-10T15:53:49.345395+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":4971,"log_dups_size":0,"ondisk_log_size":4971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20361216,"num_objects":4971,"num_object_clones":0,"num_object_copies":9942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4971,"num_write_kb":19884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4831","reported_seq":4952,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:46.267804+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:46.267804+0000","last_peered":"2024-09-10T15:53:46.267804+0000","last_clean":"2024-09-10T15:53:46.267804+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:46.267804+0000","last_undegraded":"2024-09-10T15:53:46.267804+0000","last_fullsized":"2024-09-10T15:53:46.267804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":4831,"log_dups_size":0,"ondisk_log_size":4831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19759104,"num_objects":4824,"num_object_clones":0,"num_object_copies":9648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1902,"num_write_kb":7608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4867","reported_seq":4996,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.266590+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:46.266590+0000","last_peered":"2024-09-10T15:53:46.266590+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:46.266590+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":4867,"log_dups_size":0,"ondisk_log_size":4867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19914752,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1585,"num_write_kb":6340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":994},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4846","reported_seq":4981,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.294191+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:46.294191+0000","last_peered":"2024-09-10T15:53:46.294191+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:46.294191+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":4846,"log_dups_size":0,"ondisk_log_size":4846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19824640,"num_objects":4840,"num_object_clones":0,"num_object_copies":9680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1496,"num_write_kb":5984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1006},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4959","reported_seq":5085,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:46.245198+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:46.245198+0000","last_peered":"2024-09-10T15:53:46.245198+0000","last_clean":"2024-09-10T15:53:46.245198+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:46.245198+0000","last_undegraded":"2024-09-10T15:53:46.245198+0000","last_fullsized":"2024-09-10T15:53:46.245198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":4959,"log_dups_size":0,"ondisk_log_size":4959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20303872,"num_objects":4957,"num_object_clones":0,"num_object_copies":9914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1718,"num_write_kb":6872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5140","reported_seq":6878,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:49.362673+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:49.362673+0000","last_peered":"2024-09-10T15:53:49.362673+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:49.362673+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5140,"log_dups_size":0,"ondisk_log_size":5140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21024768,"num_objects":5133,"num_object_clones":0,"num_object_copies":10266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3182,"num_objects_unfound":0,"num_objects_dirty":5133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3117,"num_write_kb":12468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":808,"num_bytes_recovered":3309568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3182},{"shards":"2","objects":1949}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4896","reported_seq":5017,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.222195+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:46.222195+0000","last_peered":"2024-09-10T15:53:46.222195+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:46.222195+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":4896,"log_dups_size":0,"ondisk_log_size":4896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021248,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1760,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1025},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4972","reported_seq":5091,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:49.222098+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:49.222098+0000","last_peered":"2024-09-10T15:53:49.222098+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:49.222098+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20344832,"num_objects":4967,"num_object_clones":0,"num_object_copies":9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2946,"num_write_kb":11784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1135}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4921","reported_seq":5042,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:49.296349+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:49.296349+0000","last_peered":"2024-09-10T15:53:49.296349+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:49.296349+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":4921,"log_dups_size":0,"ondisk_log_size":4921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20131840,"num_objects":4915,"num_object_clones":0,"num_object_copies":9830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2790,"num_write_kb":11160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1086}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4902","reported_seq":5023,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.282339+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:46.282339+0000","last_peered":"2024-09-10T15:53:46.282339+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:46.282339+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20070400,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2094,"num_write_kb":8376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1067},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4990","reported_seq":5145,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.295050+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:46.295050+0000","last_peered":"2024-09-10T15:53:46.295050+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:46.295050+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":4990,"log_dups_size":0,"ondisk_log_size":4990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20406272,"num_objects":4982,"num_object_clones":0,"num_object_copies":9964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1408,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1065},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4893","reported_seq":5026,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.313582+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:46.313582+0000","last_peered":"2024-09-10T15:53:46.313582+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:46.313582+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":4893,"log_dups_size":0,"ondisk_log_size":4893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1499,"num_write_kb":5996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1012},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5044","reported_seq":5177,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:46.302598+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:46.302598+0000","last_peered":"2024-09-10T15:53:46.302598+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:46.302598+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5044,"log_dups_size":0,"ondisk_log_size":5044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20611555,"num_objects":5033,"num_object_clones":0,"num_object_copies":10066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1863,"num_write_kb":7452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1030},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":324465123,"num_objects":79216,"num_object_clones":0,"num_object_copies":158432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3182,"num_objects_unfound":0,"num_objects_dirty":79216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40818,"num_write_kb":163272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":810,"num_bytes_recovered":3317760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":653287424,"data_stored":653197099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79297,"ondisk_log_size":79297,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144948,"kb_used_data":55012,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94226892,"statfs":{"total":96636764160,"available":96488337408,"internally_reserved":0,"allocated":56332288,"data_stored":55997144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92073646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149723,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450852,"kb_used_data":297044,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93920988,"statfs":{"total":96636764160,"available":96175091712,"internally_reserved":0,"allocated":304173056,"data_stored":303917398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":157478573},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":420700,"kb_used_data":284940,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93951140,"statfs":{"total":96636764160,"available":96205967360,"internally_reserved":0,"allocated":291778560,"data_stored":291522902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,8,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":305074176,"data_stored":305070563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56377344,"data_stored":56294245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":291835904,"data_stored":291832291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:51.809 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 74 20 81833 81813 4.3179 2.36328 0.0398722 0.0180865 2024-09-10T15:53:52.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 75 20 82481 82461 4.29408 2.53125 0.0306 0.0181865 2024-09-10T15:53:53.810 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 76 20 83114 83094 4.27011 2.47266 0.0176169 0.0182867 2024-09-10T15:53:54.272 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:54.810 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 77 20 83731 83711 4.24596 2.41016 0.0355887 0.0183925 2024-09-10T15:53:54.860 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:54.861 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:54.887 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-10T15:53:54.452054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":337031670,"num_objects":82286,"num_object_clones":0,"num_object_copies":164572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3051,"num_objects_unfound":0,"num_objects_dirty":82286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43888,"num_write_kb":175546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":3854355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82368,"ondisk_log_size":82368,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1044992,"kb_used_data":661324,"kb_used_omap":12,"kb_used_meta":383603,"kb_avail":282070528,"statfs":{"total":289910292480,"available":288840220672,"internally_reserved":0,"allocated":677195776,"data_stored":676349316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":392809477},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,5,10,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29945856,"num_objects":7311,"num_object_clones":0,"num_object_copies":14622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-427,"num_objects_unfound":0,"num_objects_dirty":7311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7311,"num_write_kb":29244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":1748992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003300"},"pg_stats":[{"pgid":"2.f","version":"44'5177","reported_seq":5248,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.274189+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:51.274189+0000","last_peered":"2024-09-10T15:53:51.274189+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:51.274189+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5177,"log_dups_size":0,"ondisk_log_size":5177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21204992,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5177,"num_write_kb":20708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1204},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5337","reported_seq":5439,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:54.327216+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:54.327216+0000","last_peered":"2024-09-10T15:53:54.327216+0000","last_clean":"2024-09-10T15:53:54.327216+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:54.327216+0000","last_undegraded":"2024-09-10T15:53:54.327216+0000","last_fullsized":"2024-09-10T15:53:54.327216+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21835776,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3673,"num_write_kb":14692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5124","reported_seq":5242,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.291102+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:54.291102+0000","last_peered":"2024-09-10T15:53:54.291102+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:54.291102+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5124,"log_dups_size":0,"ondisk_log_size":5124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5120,"num_object_clones":0,"num_object_copies":10240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3392,"num_write_kb":13568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1318}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5174","reported_seq":5248,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:54.313090+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:54.313090+0000","last_peered":"2024-09-10T15:53:54.313090+0000","last_clean":"2024-09-10T15:53:54.313090+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:54.313090+0000","last_undegraded":"2024-09-10T15:53:54.313090+0000","last_fullsized":"2024-09-10T15:53:54.313090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21192704,"num_objects":5174,"num_object_clones":0,"num_object_copies":10348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5174,"num_write_kb":20696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5018","reported_seq":5138,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:51.341660+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:51.341660+0000","last_peered":"2024-09-10T15:53:51.341660+0000","last_clean":"2024-09-10T15:53:51.341660+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:51.341660+0000","last_undegraded":"2024-09-10T15:53:51.341660+0000","last_fullsized":"2024-09-10T15:53:51.341660+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5018,"log_dups_size":0,"ondisk_log_size":5018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20525056,"num_objects":5011,"num_object_clones":0,"num_object_copies":10022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2089,"num_write_kb":8356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5077","reported_seq":5206,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.301884+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:51.301884+0000","last_peered":"2024-09-10T15:53:51.301884+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:51.301884+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1795,"num_write_kb":7180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1204},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5027","reported_seq":5162,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.313269+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:51.313269+0000","last_peered":"2024-09-10T15:53:51.313269+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:51.313269+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5027,"log_dups_size":0,"ondisk_log_size":5027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20566016,"num_objects":5021,"num_object_clones":0,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1677,"num_write_kb":6708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1187},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5172","reported_seq":5297,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:51.334428+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:51.334428+0000","last_peered":"2024-09-10T15:53:51.334428+0000","last_clean":"2024-09-10T15:53:51.334428+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:51.334428+0000","last_undegraded":"2024-09-10T15:53:51.334428+0000","last_fullsized":"2024-09-10T15:53:51.334428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5172,"log_dups_size":0,"ondisk_log_size":5172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176320,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1931,"num_write_kb":7724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5344","reported_seq":7346,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.388326+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:54.388326+0000","last_peered":"2024-09-10T15:53:54.388326+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:54.388326+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5344,"log_dups_size":0,"ondisk_log_size":5344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21860352,"num_objects":5337,"num_object_clones":0,"num_object_copies":10674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3051,"num_objects_unfound":0,"num_objects_dirty":5337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3321,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":939,"num_bytes_recovered":3846144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3051},{"shards":"2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5063","reported_seq":5184,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.260663+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:51.260663+0000","last_peered":"2024-09-10T15:53:51.260663+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:51.260663+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5063,"log_dups_size":0,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20705280,"num_objects":5055,"num_object_clones":0,"num_object_copies":10110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1927,"num_write_kb":7708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1192},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5165","reported_seq":5282,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.333898+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:54.333898+0000","last_peered":"2024-09-10T15:53:54.333898+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:54.333898+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5165,"log_dups_size":0,"ondisk_log_size":5165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135360,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3139,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1328}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5128","reported_seq":5249,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.370527+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:54.370527+0000","last_peered":"2024-09-10T15:53:54.370527+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:54.370527+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5128,"log_dups_size":0,"ondisk_log_size":5128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20979712,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2997,"num_write_kb":11988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1293}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5096","reported_seq":5218,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.302695+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:51.302695+0000","last_peered":"2024-09-10T15:53:51.302695+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:51.302695+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5096,"log_dups_size":0,"ondisk_log_size":5096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20865024,"num_objects":5094,"num_object_clones":0,"num_object_copies":10188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2288,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1261},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5163","reported_seq":5321,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.338000+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:51.338000+0000","last_peered":"2024-09-10T15:53:51.338000+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:51.338000+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5163,"log_dups_size":0,"ondisk_log_size":5163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21114880,"num_objects":5155,"num_object_clones":0,"num_object_copies":10310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1581,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1238},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5079","reported_seq":5213,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.310187+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:51.310187+0000","last_peered":"2024-09-10T15:53:51.310187+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:51.310187+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5079,"log_dups_size":0,"ondisk_log_size":5079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20787200,"num_objects":5075,"num_object_clones":0,"num_object_copies":10150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1685,"num_write_kb":6740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1198},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5221","reported_seq":5354,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.305629+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:51.305629+0000","last_peered":"2024-09-10T15:53:51.305629+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:51.305629+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5221,"log_dups_size":0,"ondisk_log_size":5221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21336547,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2040,"num_write_kb":8160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1207},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":337031651,"num_objects":82284,"num_object_clones":0,"num_object_copies":164568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3051,"num_objects_unfound":0,"num_objects_dirty":82284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43886,"num_write_kb":175544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":941,"num_bytes_recovered":3854336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":678629376,"data_stored":678539051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82365,"ondisk_log_size":82365,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626441,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145880,"kb_used_data":64260,"kb_used_omap":4,"kb_used_meta":81595,"kb_avail":94225960,"statfs":{"total":96636764160,"available":96487383040,"internally_reserved":0,"allocated":65802240,"data_stored":65467096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":83553966},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149724,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468408,"kb_used_data":306280,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93903432,"statfs":{"total":96636764160,"available":96157114368,"internally_reserved":0,"allocated":313630720,"data_stored":313375062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":165998253},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430704,"kb_used_data":290784,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93941136,"statfs":{"total":96636764160,"available":96195723264,"internally_reserved":0,"allocated":297762816,"data_stored":297507158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314294272,"data_stored":314290659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66007040,"data_stored":65923941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298328064,"data_stored":298324451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:54.889 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:55.367 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:55.368 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:55.395 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-10T15:53:54.452054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":337031670,"num_objects":82286,"num_object_clones":0,"num_object_copies":164572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3051,"num_objects_unfound":0,"num_objects_dirty":82286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43888,"num_write_kb":175546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":3854355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82368,"ondisk_log_size":82368,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1044992,"kb_used_data":661324,"kb_used_omap":12,"kb_used_meta":383603,"kb_avail":282070528,"statfs":{"total":289910292480,"available":288840220672,"internally_reserved":0,"allocated":677195776,"data_stored":676349316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":392809477},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,5,10,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29945856,"num_objects":7311,"num_object_clones":0,"num_object_copies":14622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-427,"num_objects_unfound":0,"num_objects_dirty":7311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7311,"num_write_kb":29244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":1748992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003300"},"pg_stats":[{"pgid":"2.f","version":"44'5177","reported_seq":5248,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.274189+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:51.274189+0000","last_peered":"2024-09-10T15:53:51.274189+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:51.274189+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5177,"log_dups_size":0,"ondisk_log_size":5177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21204992,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5177,"num_write_kb":20708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1204},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5337","reported_seq":5439,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:54.327216+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:54.327216+0000","last_peered":"2024-09-10T15:53:54.327216+0000","last_clean":"2024-09-10T15:53:54.327216+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:54.327216+0000","last_undegraded":"2024-09-10T15:53:54.327216+0000","last_fullsized":"2024-09-10T15:53:54.327216+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21835776,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3673,"num_write_kb":14692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5124","reported_seq":5242,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.291102+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:54.291102+0000","last_peered":"2024-09-10T15:53:54.291102+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:54.291102+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5124,"log_dups_size":0,"ondisk_log_size":5124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5120,"num_object_clones":0,"num_object_copies":10240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3392,"num_write_kb":13568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1318}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5174","reported_seq":5248,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:54.313090+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:54.313090+0000","last_peered":"2024-09-10T15:53:54.313090+0000","last_clean":"2024-09-10T15:53:54.313090+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:54.313090+0000","last_undegraded":"2024-09-10T15:53:54.313090+0000","last_fullsized":"2024-09-10T15:53:54.313090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21192704,"num_objects":5174,"num_object_clones":0,"num_object_copies":10348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5174,"num_write_kb":20696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5018","reported_seq":5138,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:51.341660+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:51.341660+0000","last_peered":"2024-09-10T15:53:51.341660+0000","last_clean":"2024-09-10T15:53:51.341660+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:51.341660+0000","last_undegraded":"2024-09-10T15:53:51.341660+0000","last_fullsized":"2024-09-10T15:53:51.341660+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5018,"log_dups_size":0,"ondisk_log_size":5018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20525056,"num_objects":5011,"num_object_clones":0,"num_object_copies":10022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2089,"num_write_kb":8356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5077","reported_seq":5206,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.301884+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:51.301884+0000","last_peered":"2024-09-10T15:53:51.301884+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:51.301884+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1795,"num_write_kb":7180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1204},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5027","reported_seq":5162,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.313269+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:51.313269+0000","last_peered":"2024-09-10T15:53:51.313269+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:51.313269+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5027,"log_dups_size":0,"ondisk_log_size":5027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20566016,"num_objects":5021,"num_object_clones":0,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1677,"num_write_kb":6708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1187},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5172","reported_seq":5297,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:51.334428+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:51.334428+0000","last_peered":"2024-09-10T15:53:51.334428+0000","last_clean":"2024-09-10T15:53:51.334428+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:51.334428+0000","last_undegraded":"2024-09-10T15:53:51.334428+0000","last_fullsized":"2024-09-10T15:53:51.334428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5172,"log_dups_size":0,"ondisk_log_size":5172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176320,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1931,"num_write_kb":7724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5344","reported_seq":7346,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.388326+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:54.388326+0000","last_peered":"2024-09-10T15:53:54.388326+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:54.388326+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5344,"log_dups_size":0,"ondisk_log_size":5344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21860352,"num_objects":5337,"num_object_clones":0,"num_object_copies":10674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3051,"num_objects_unfound":0,"num_objects_dirty":5337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3321,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":939,"num_bytes_recovered":3846144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3051},{"shards":"2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5063","reported_seq":5184,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.260663+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:51.260663+0000","last_peered":"2024-09-10T15:53:51.260663+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:51.260663+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5063,"log_dups_size":0,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20705280,"num_objects":5055,"num_object_clones":0,"num_object_copies":10110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1927,"num_write_kb":7708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1192},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5165","reported_seq":5282,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.333898+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:54.333898+0000","last_peered":"2024-09-10T15:53:54.333898+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:54.333898+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5165,"log_dups_size":0,"ondisk_log_size":5165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135360,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3139,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1328}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5128","reported_seq":5249,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.370527+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:54.370527+0000","last_peered":"2024-09-10T15:53:54.370527+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:54.370527+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5128,"log_dups_size":0,"ondisk_log_size":5128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20979712,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2997,"num_write_kb":11988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1293}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5096","reported_seq":5218,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.302695+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:51.302695+0000","last_peered":"2024-09-10T15:53:51.302695+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:51.302695+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5096,"log_dups_size":0,"ondisk_log_size":5096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20865024,"num_objects":5094,"num_object_clones":0,"num_object_copies":10188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2288,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1261},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5163","reported_seq":5321,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.338000+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:51.338000+0000","last_peered":"2024-09-10T15:53:51.338000+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:51.338000+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5163,"log_dups_size":0,"ondisk_log_size":5163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21114880,"num_objects":5155,"num_object_clones":0,"num_object_copies":10310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1581,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1238},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5079","reported_seq":5213,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.310187+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:51.310187+0000","last_peered":"2024-09-10T15:53:51.310187+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:51.310187+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5079,"log_dups_size":0,"ondisk_log_size":5079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20787200,"num_objects":5075,"num_object_clones":0,"num_object_copies":10150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1685,"num_write_kb":6740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1198},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5221","reported_seq":5354,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.305629+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:51.305629+0000","last_peered":"2024-09-10T15:53:51.305629+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:51.305629+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5221,"log_dups_size":0,"ondisk_log_size":5221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21336547,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2040,"num_write_kb":8160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1207},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":337031651,"num_objects":82284,"num_object_clones":0,"num_object_copies":164568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3051,"num_objects_unfound":0,"num_objects_dirty":82284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43886,"num_write_kb":175544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":941,"num_bytes_recovered":3854336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":678629376,"data_stored":678539051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82365,"ondisk_log_size":82365,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626441,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145880,"kb_used_data":64260,"kb_used_omap":4,"kb_used_meta":81595,"kb_avail":94225960,"statfs":{"total":96636764160,"available":96487383040,"internally_reserved":0,"allocated":65802240,"data_stored":65467096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":83553966},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149724,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468408,"kb_used_data":306280,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93903432,"statfs":{"total":96636764160,"available":96157114368,"internally_reserved":0,"allocated":313630720,"data_stored":313375062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":165998253},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430704,"kb_used_data":290784,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93941136,"statfs":{"total":96636764160,"available":96195723264,"internally_reserved":0,"allocated":297762816,"data_stored":297507158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314294272,"data_stored":314290659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66007040,"data_stored":65923941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298328064,"data_stored":298324451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:55.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:53:55.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'5177", 'reported_seq': 5248, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:51.274189+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:53:51.274189+0000', 'last_peered': '2024-09-10T15:53:51.274189+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:53:51.274189+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 5177, 'log_dups_size': 0, 'ondisk_log_size': 5177, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21204992, 'num_objects': 5177, 'num_object_clones': 0, 'num_object_copies': 10354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5177, 'num_write_kb': 20708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1204}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:55.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:53:55.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'5124", 'reported_seq': 5242, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:54.291102+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:53:54.291102+0000', 'last_peered': '2024-09-10T15:53:54.291102+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:53:54.291102+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 5124, 'log_dups_size': 0, 'ondisk_log_size': 5124, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20971520, 'num_objects': 5120, 'num_object_clones': 0, 'num_object_copies': 10240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5120, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3392, 'num_write_kb': 13568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 1318}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:55.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:53:55.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'5077", 'reported_seq': 5206, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:51.301884+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:53:51.301884+0000', 'last_peered': '2024-09-10T15:53:51.301884+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:53:51.301884+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 5077, 'log_dups_size': 0, 'ondisk_log_size': 5077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20774912, 'num_objects': 5072, 'num_object_clones': 0, 'num_object_copies': 10144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5072, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1795, 'num_write_kb': 7180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1204}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:55.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:53:55.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'5027", 'reported_seq': 5162, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:51.313269+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:53:51.313269+0000', 'last_peered': '2024-09-10T15:53:51.313269+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:53:51.313269+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 5027, 'log_dups_size': 0, 'ondisk_log_size': 5027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20566016, 'num_objects': 5021, 'num_object_clones': 0, 'num_object_copies': 10042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5021, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1677, 'num_write_kb': 6708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1187}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:55.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:53:55.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'5344", 'reported_seq': 7346, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:54.388326+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:53:54.388326+0000', 'last_peered': '2024-09-10T15:53:54.388326+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:53:54.388326+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 5344, 'log_dups_size': 0, 'ondisk_log_size': 5344, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21860352, 'num_objects': 5337, 'num_object_clones': 0, 'num_object_copies': 10674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3051, 'num_objects_unfound': 0, 'num_objects_dirty': 5337, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3321, 'num_write_kb': 13284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 939, 'num_bytes_recovered': 3846144, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3051}, {'shards': '2', 'objects': 2284}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:55.398 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:53:55.398 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'5063", 'reported_seq': 5184, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:51.260663+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:53:51.260663+0000', 'last_peered': '2024-09-10T15:53:51.260663+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:53:51.260663+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 5063, 'log_dups_size': 0, 'ondisk_log_size': 5063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20705280, 'num_objects': 5055, 'num_object_clones': 0, 'num_object_copies': 10110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1927, 'num_write_kb': 7708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1192}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:55.398 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:53:55.398 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'5165", 'reported_seq': 5282, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:54.333898+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:53:54.333898+0000', 'last_peered': '2024-09-10T15:53:54.333898+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:53:54.333898+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 5165, 'log_dups_size': 0, 'ondisk_log_size': 5165, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21135360, 'num_objects': 5160, 'num_object_clones': 0, 'num_object_copies': 10320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3139, 'num_write_kb': 12556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 1328}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:55.398 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:53:55.398 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'5128", 'reported_seq': 5249, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:54.370527+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:53:54.370527+0000', 'last_peered': '2024-09-10T15:53:54.370527+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:53:54.370527+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 5128, 'log_dups_size': 0, 'ondisk_log_size': 5128, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20979712, 'num_objects': 5122, 'num_object_clones': 0, 'num_object_copies': 10244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5122, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2997, 'num_write_kb': 11988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 1293}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:53:55.398 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:53:55.398 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'5096", 'reported_seq': 5218, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:51.302695+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:53:51.302695+0000', 'last_peered': '2024-09-10T15:53:51.302695+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:53:51.302695+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 5096, 'log_dups_size': 0, 'ondisk_log_size': 5096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20865024, 'num_objects': 5094, 'num_object_clones': 0, 'num_object_copies': 10188, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5094, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2288, 'num_write_kb': 9152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1261}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:55.399 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:53:55.399 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'5163", 'reported_seq': 5321, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:51.338000+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:53:51.338000+0000', 'last_peered': '2024-09-10T15:53:51.338000+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:53:51.338000+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 5163, 'log_dups_size': 0, 'ondisk_log_size': 5163, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21114880, 'num_objects': 5155, 'num_object_clones': 0, 'num_object_copies': 10310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5155, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1581, 'num_write_kb': 6324, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1238}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:55.399 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:53:55.399 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'5079", 'reported_seq': 5213, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:51.310187+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:53:51.310187+0000', 'last_peered': '2024-09-10T15:53:51.310187+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:53:51.310187+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 5079, 'log_dups_size': 0, 'ondisk_log_size': 5079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20787200, 'num_objects': 5075, 'num_object_clones': 0, 'num_object_copies': 10150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5075, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1685, 'num_write_kb': 6740, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1198}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:55.399 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:53:55.399 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'5221", 'reported_seq': 5354, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:51.305629+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:53:51.305629+0000', 'last_peered': '2024-09-10T15:53:51.305629+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:53:51.305629+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 5221, 'log_dups_size': 0, 'ondisk_log_size': 5221, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21336547, 'num_objects': 5210, 'num_object_clones': 0, 'num_object_copies': 10420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5210, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2040, 'num_write_kb': 8160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1207}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:53:55.400 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:53:55.810 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 78 20 84206 84186 4.21531 1.85547 0.0285933 0.018525 2024-09-10T15:53:56.125 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:56.125 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/164572 objects degraded (25.918%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":162,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":82286,"data_bytes":337031670,"bytes_used":1070071808,"bytes_avail":288840220672,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":164572,"degraded_ratio":0.25917531536348831,"misplaced_objects":3051,"misplaced_total":164572,"misplaced_ratio":0.018538998128478722,"recovering_objects_per_sec":35,"recovering_bytes_per_sec":145709,"recovering_keys_per_sec":0,"num_objects_recovered":427,"num_bytes_recovered":1748992,"num_keys_recovered":0,"write_bytes_sec":2494801,"read_op_per_sec":0,"write_op_per_sec":609},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:53:42.448843+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:53:56.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:53:56.149 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:56.713 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:53:56.714 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:53:56.738 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-10T15:53:56.452624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":337031670,"num_objects":82286,"num_object_clones":0,"num_object_copies":164572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3051,"num_objects_unfound":0,"num_objects_dirty":82286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43888,"num_write_kb":175546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":3854355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82368,"ondisk_log_size":82368,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1044992,"kb_used_data":661324,"kb_used_omap":12,"kb_used_meta":383603,"kb_avail":282070528,"statfs":{"total":289910292480,"available":288840220672,"internally_reserved":0,"allocated":677195776,"data_stored":676349316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":392809477},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,5,10,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25198592,"num_objects":6152,"num_object_clones":0,"num_object_copies":12304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-290,"num_objects_unfound":0,"num_objects_dirty":6152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6152,"num_write_kb":24608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":1187840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003418"},"pg_stats":[{"pgid":"2.f","version":"44'5177","reported_seq":5248,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.274189+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:51.274189+0000","last_peered":"2024-09-10T15:53:51.274189+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:51.274189+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5177,"log_dups_size":0,"ondisk_log_size":5177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21204992,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5177,"num_write_kb":20708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1204},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5337","reported_seq":5439,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:54.327216+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:54.327216+0000","last_peered":"2024-09-10T15:53:54.327216+0000","last_clean":"2024-09-10T15:53:54.327216+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:54.327216+0000","last_undegraded":"2024-09-10T15:53:54.327216+0000","last_fullsized":"2024-09-10T15:53:54.327216+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21835776,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3673,"num_write_kb":14692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5124","reported_seq":5242,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.291102+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:54.291102+0000","last_peered":"2024-09-10T15:53:54.291102+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:54.291102+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5124,"log_dups_size":0,"ondisk_log_size":5124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5120,"num_object_clones":0,"num_object_copies":10240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3392,"num_write_kb":13568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1318}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5174","reported_seq":5248,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:54.313090+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:54.313090+0000","last_peered":"2024-09-10T15:53:54.313090+0000","last_clean":"2024-09-10T15:53:54.313090+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:54.313090+0000","last_undegraded":"2024-09-10T15:53:54.313090+0000","last_fullsized":"2024-09-10T15:53:54.313090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21192704,"num_objects":5174,"num_object_clones":0,"num_object_copies":10348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5174,"num_write_kb":20696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5018","reported_seq":5138,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:51.341660+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:51.341660+0000","last_peered":"2024-09-10T15:53:51.341660+0000","last_clean":"2024-09-10T15:53:51.341660+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:51.341660+0000","last_undegraded":"2024-09-10T15:53:51.341660+0000","last_fullsized":"2024-09-10T15:53:51.341660+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5018,"log_dups_size":0,"ondisk_log_size":5018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20525056,"num_objects":5011,"num_object_clones":0,"num_object_copies":10022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2089,"num_write_kb":8356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5077","reported_seq":5206,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.301884+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:51.301884+0000","last_peered":"2024-09-10T15:53:51.301884+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:51.301884+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1795,"num_write_kb":7180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1204},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5027","reported_seq":5162,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.313269+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:51.313269+0000","last_peered":"2024-09-10T15:53:51.313269+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:51.313269+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5027,"log_dups_size":0,"ondisk_log_size":5027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20566016,"num_objects":5021,"num_object_clones":0,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1677,"num_write_kb":6708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1187},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5172","reported_seq":5297,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:51.334428+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:51.334428+0000","last_peered":"2024-09-10T15:53:51.334428+0000","last_clean":"2024-09-10T15:53:51.334428+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:51.334428+0000","last_undegraded":"2024-09-10T15:53:51.334428+0000","last_fullsized":"2024-09-10T15:53:51.334428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5172,"log_dups_size":0,"ondisk_log_size":5172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176320,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1931,"num_write_kb":7724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5344","reported_seq":7346,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.388326+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:54.388326+0000","last_peered":"2024-09-10T15:53:54.388326+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:54.388326+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5344,"log_dups_size":0,"ondisk_log_size":5344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21860352,"num_objects":5337,"num_object_clones":0,"num_object_copies":10674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3051,"num_objects_unfound":0,"num_objects_dirty":5337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3321,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":939,"num_bytes_recovered":3846144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3051},{"shards":"2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5063","reported_seq":5184,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.260663+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:51.260663+0000","last_peered":"2024-09-10T15:53:51.260663+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:51.260663+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5063,"log_dups_size":0,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20705280,"num_objects":5055,"num_object_clones":0,"num_object_copies":10110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1927,"num_write_kb":7708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1192},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5165","reported_seq":5282,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.333898+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:54.333898+0000","last_peered":"2024-09-10T15:53:54.333898+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:54.333898+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5165,"log_dups_size":0,"ondisk_log_size":5165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135360,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3139,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1328}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5128","reported_seq":5249,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.370527+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:54.370527+0000","last_peered":"2024-09-10T15:53:54.370527+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:54.370527+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5128,"log_dups_size":0,"ondisk_log_size":5128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20979712,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2997,"num_write_kb":11988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1293}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5096","reported_seq":5218,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.302695+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:51.302695+0000","last_peered":"2024-09-10T15:53:51.302695+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:51.302695+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5096,"log_dups_size":0,"ondisk_log_size":5096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20865024,"num_objects":5094,"num_object_clones":0,"num_object_copies":10188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2288,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1261},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5163","reported_seq":5321,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.338000+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:51.338000+0000","last_peered":"2024-09-10T15:53:51.338000+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:51.338000+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5163,"log_dups_size":0,"ondisk_log_size":5163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21114880,"num_objects":5155,"num_object_clones":0,"num_object_copies":10310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1581,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1238},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5079","reported_seq":5213,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.310187+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:51.310187+0000","last_peered":"2024-09-10T15:53:51.310187+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:51.310187+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5079,"log_dups_size":0,"ondisk_log_size":5079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20787200,"num_objects":5075,"num_object_clones":0,"num_object_copies":10150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1685,"num_write_kb":6740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1198},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5221","reported_seq":5354,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:51.305629+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:51.305629+0000","last_peered":"2024-09-10T15:53:51.305629+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:51.305629+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5221,"log_dups_size":0,"ondisk_log_size":5221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21336547,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2040,"num_write_kb":8160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1207},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":337031651,"num_objects":82284,"num_object_clones":0,"num_object_copies":164568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3051,"num_objects_unfound":0,"num_objects_dirty":82284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43886,"num_write_kb":175544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":941,"num_bytes_recovered":3854336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":678629376,"data_stored":678539051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82365,"ondisk_log_size":82365,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626441,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145880,"kb_used_data":64260,"kb_used_omap":4,"kb_used_meta":81595,"kb_avail":94225960,"statfs":{"total":96636764160,"available":96487383040,"internally_reserved":0,"allocated":65802240,"data_stored":65467096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":83553966},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149724,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468408,"kb_used_data":306280,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93903432,"statfs":{"total":96636764160,"available":96157114368,"internally_reserved":0,"allocated":313630720,"data_stored":313375062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":165998253},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430704,"kb_used_data":290784,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93941136,"statfs":{"total":96636764160,"available":96195723264,"internally_reserved":0,"allocated":297762816,"data_stored":297507158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314294272,"data_stored":314290659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66007040,"data_stored":65923941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298328064,"data_stored":298324451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:53:56.810 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 79 20 84766 84746 4.18964 2.1875 0.0502954 0.018641 2024-09-10T15:53:57.810 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:2024-09-10T15:53:57.809878+0000 min lat: 0.00198913 max lat: 2.66872 avg lat: 0.0187321 2024-09-10T15:53:57.810 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:53:57.810 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 80 20 85413 85393 4.16886 2.52734 0.0259086 0.0187321 2024-09-10T15:53:58.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 81 20 86046 86026 4.14792 2.47266 0.0463647 0.0188264 2024-09-10T15:53:59.741 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:53:59.810 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 82 20 86678 86658 4.12744 2.46875 0.045805 0.018921 2024-09-10T15:54:00.383 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:00.386 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:00.408 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-10T15:53:58.453242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346087926,"num_objects":84497,"num_object_clones":0,"num_object_copies":168994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3051,"num_objects_unfound":0,"num_objects_dirty":84497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46099,"num_write_kb":184390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":3854355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84579,"ondisk_log_size":84579,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1095436,"kb_used_data":681244,"kb_used_omap":12,"kb_used_meta":414131,"kb_avail":282020084,"statfs":{"total":289910292480,"available":288788566016,"internally_reserved":0,"allocated":697593856,"data_stored":696747396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":424070149},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,10,8,10,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26148864,"num_objects":6384,"num_object_clones":0,"num_object_copies":12768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-290,"num_objects_unfound":0,"num_objects_dirty":6384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6384,"num_write_kb":25536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":1187840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003549"},"pg_stats":[{"pgid":"2.f","version":"44'5372","reported_seq":5443,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.374998+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:57.374998+0000","last_peered":"2024-09-10T15:53:57.374998+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:57.374998+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5372,"log_dups_size":0,"ondisk_log_size":5372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5372,"num_write_kb":21488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1399},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5337","reported_seq":5439,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:54.327216+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:54.327216+0000","last_peered":"2024-09-10T15:53:54.327216+0000","last_clean":"2024-09-10T15:53:54.327216+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:54.327216+0000","last_undegraded":"2024-09-10T15:53:54.327216+0000","last_fullsized":"2024-09-10T15:53:54.327216+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21835776,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3673,"num_write_kb":14692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5124","reported_seq":5242,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.291102+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:54.291102+0000","last_peered":"2024-09-10T15:53:54.291102+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:54.291102+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5124,"log_dups_size":0,"ondisk_log_size":5124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5120,"num_object_clones":0,"num_object_copies":10240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3392,"num_write_kb":13568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1318}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5174","reported_seq":5248,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:54.313090+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:54.313090+0000","last_peered":"2024-09-10T15:53:54.313090+0000","last_clean":"2024-09-10T15:53:54.313090+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:54.313090+0000","last_undegraded":"2024-09-10T15:53:54.313090+0000","last_fullsized":"2024-09-10T15:53:54.313090+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21192704,"num_objects":5174,"num_object_clones":0,"num_object_copies":10348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5174,"num_write_kb":20696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5243","reported_seq":5364,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:57.375604+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:57.375604+0000","last_peered":"2024-09-10T15:53:57.375604+0000","last_clean":"2024-09-10T15:53:57.375604+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:57.375604+0000","last_undegraded":"2024-09-10T15:53:57.375604+0000","last_fullsized":"2024-09-10T15:53:57.375604+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5243,"log_dups_size":0,"ondisk_log_size":5243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2314,"num_write_kb":9256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5329","reported_seq":5457,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.334288+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:57.334288+0000","last_peered":"2024-09-10T15:53:57.334288+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:57.334288+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5329,"log_dups_size":0,"ondisk_log_size":5329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807104,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2047,"num_write_kb":8188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1456},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5250","reported_seq":5385,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.336072+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:57.336072+0000","last_peered":"2024-09-10T15:53:57.336072+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:57.336072+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5250,"log_dups_size":0,"ondisk_log_size":5250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21479424,"num_objects":5244,"num_object_clones":0,"num_object_copies":10488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1900,"num_write_kb":7600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1410},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5374","reported_seq":5500,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:57.294414+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:57.294414+0000","last_peered":"2024-09-10T15:53:57.294414+0000","last_clean":"2024-09-10T15:53:57.294414+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:57.294414+0000","last_undegraded":"2024-09-10T15:53:57.294414+0000","last_fullsized":"2024-09-10T15:53:57.294414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5374,"log_dups_size":0,"ondisk_log_size":5374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2133,"num_write_kb":8532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5344","reported_seq":7346,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.388326+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:54.388326+0000","last_peered":"2024-09-10T15:53:54.388326+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:54.388326+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5344,"log_dups_size":0,"ondisk_log_size":5344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21860352,"num_objects":5337,"num_object_clones":0,"num_object_copies":10674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3051,"num_objects_unfound":0,"num_objects_dirty":5337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3321,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":939,"num_bytes_recovered":3846144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3051},{"shards":"2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5288","reported_seq":5409,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.330086+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:57.330086+0000","last_peered":"2024-09-10T15:53:57.330086+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:57.330086+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5288,"log_dups_size":0,"ondisk_log_size":5288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21626880,"num_objects":5280,"num_object_clones":0,"num_object_copies":10560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2152,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1417},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5165","reported_seq":5282,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.333898+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:54.333898+0000","last_peered":"2024-09-10T15:53:54.333898+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:54.333898+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5165,"log_dups_size":0,"ondisk_log_size":5165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135360,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3139,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1328}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5128","reported_seq":5249,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:54.370527+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:54.370527+0000","last_peered":"2024-09-10T15:53:54.370527+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:54.370527+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5128,"log_dups_size":0,"ondisk_log_size":5128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20979712,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2997,"num_write_kb":11988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1293}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5335","reported_seq":5456,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.377198+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:57.377198+0000","last_peered":"2024-09-10T15:53:57.377198+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:57.377198+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2527,"num_write_kb":10108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1500},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5377","reported_seq":5535,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.340922+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:57.340922+0000","last_peered":"2024-09-10T15:53:57.340922+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:57.340922+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5377,"log_dups_size":0,"ondisk_log_size":5377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21991424,"num_objects":5369,"num_object_clones":0,"num_object_copies":10738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1795,"num_write_kb":7180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1452},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5299","reported_seq":5433,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.330376+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:57.330376+0000","last_peered":"2024-09-10T15:53:57.330376+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:57.330376+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21688320,"num_objects":5295,"num_object_clones":0,"num_object_copies":10590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1905,"num_write_kb":7620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1418},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5437","reported_seq":5570,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.371233+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:57.371233+0000","last_peered":"2024-09-10T15:53:57.371233+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:57.371233+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5437,"log_dups_size":0,"ondisk_log_size":5437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22221283,"num_objects":5426,"num_object_clones":0,"num_object_copies":10852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":9024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1423},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":346087907,"num_objects":84495,"num_object_clones":0,"num_object_copies":168990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":3051,"num_objects_unfound":0,"num_objects_dirty":84495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46097,"num_write_kb":184388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":941,"num_bytes_recovered":3854336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":699342848,"data_stored":699252523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84576,"ondisk_log_size":84576,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626442,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163376,"kb_used_data":73440,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94208464,"statfs":{"total":96636764160,"available":96469467136,"internally_reserved":0,"allocated":75202560,"data_stored":74867416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92073646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149726,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501356,"kb_used_data":317020,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93870484,"statfs":{"total":96636764160,"available":96123375616,"internally_reserved":0,"allocated":324628480,"data_stored":324372822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188739245},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,8,3,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430704,"kb_used_data":290784,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93941136,"statfs":{"total":96636764160,"available":96195723264,"internally_reserved":0,"allocated":297762816,"data_stored":297507158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":325337088,"data_stored":325333475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75677696,"data_stored":75594597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298328064,"data_stored":298324451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:00.410 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:00.810 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 83 20 87163 87143 4.10054 1.89453 0.0280691 0.0190421 2024-09-10T15:54:00.944 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:00.947 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:00.975 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-10T15:54:00.453815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":350736886,"num_objects":85632,"num_object_clones":0,"num_object_copies":171264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2901,"num_objects_unfound":0,"num_objects_dirty":85632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47234,"num_write_kb":188930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1093,"num_bytes_recovered":4468755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85714,"ondisk_log_size":85714,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1123648,"kb_used_data":687248,"kb_used_omap":12,"kb_used_meta":436339,"kb_avail":281991872,"statfs":{"total":289910292480,"available":288759676928,"internally_reserved":0,"allocated":703741952,"data_stored":702895492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":446811141},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,9,11,13,8,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30797824,"num_objects":7519,"num_object_clones":0,"num_object_copies":15038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-440,"num_objects_unfound":0,"num_objects_dirty":7519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7519,"num_write_kb":30076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003667"},"pg_stats":[{"pgid":"2.f","version":"44'5372","reported_seq":5443,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.374998+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:57.374998+0000","last_peered":"2024-09-10T15:53:57.374998+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:57.374998+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5372,"log_dups_size":0,"ondisk_log_size":5372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5372,"num_write_kb":21488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1399},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5537","reported_seq":5639,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:59.335858+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:59.335858+0000","last_peered":"2024-09-10T15:53:59.335858+0000","last_clean":"2024-09-10T15:53:59.335858+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:59.335858+0000","last_undegraded":"2024-09-10T15:53:59.335858+0000","last_fullsized":"2024-09-10T15:53:59.335858+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5537,"log_dups_size":0,"ondisk_log_size":5537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22654976,"num_objects":5531,"num_object_clones":0,"num_object_copies":11062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3873,"num_write_kb":15492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5314","reported_seq":5432,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:59.355349+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:59.355349+0000","last_peered":"2024-09-10T15:53:59.355349+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:59.355349+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5314,"log_dups_size":0,"ondisk_log_size":5314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21749760,"num_objects":5310,"num_object_clones":0,"num_object_copies":10620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3582,"num_write_kb":14328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5343","reported_seq":5417,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:59.360508+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:59.360508+0000","last_peered":"2024-09-10T15:53:59.360508+0000","last_clean":"2024-09-10T15:53:59.360508+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:59.360508+0000","last_undegraded":"2024-09-10T15:53:59.360508+0000","last_fullsized":"2024-09-10T15:53:59.360508+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5343,"log_dups_size":0,"ondisk_log_size":5343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21884928,"num_objects":5343,"num_object_clones":0,"num_object_copies":10686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5343,"num_write_kb":21372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5243","reported_seq":5364,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:57.375604+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:57.375604+0000","last_peered":"2024-09-10T15:53:57.375604+0000","last_clean":"2024-09-10T15:53:57.375604+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:57.375604+0000","last_undegraded":"2024-09-10T15:53:57.375604+0000","last_fullsized":"2024-09-10T15:53:57.375604+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5243,"log_dups_size":0,"ondisk_log_size":5243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2314,"num_write_kb":9256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5329","reported_seq":5457,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.334288+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:57.334288+0000","last_peered":"2024-09-10T15:53:57.334288+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:57.334288+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5329,"log_dups_size":0,"ondisk_log_size":5329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807104,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2047,"num_write_kb":8188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1456},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5250","reported_seq":5385,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.336072+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:57.336072+0000","last_peered":"2024-09-10T15:53:57.336072+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:57.336072+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5250,"log_dups_size":0,"ondisk_log_size":5250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21479424,"num_objects":5244,"num_object_clones":0,"num_object_copies":10488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1900,"num_write_kb":7600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1410},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5374","reported_seq":5500,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:57.294414+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:57.294414+0000","last_peered":"2024-09-10T15:53:57.294414+0000","last_clean":"2024-09-10T15:53:57.294414+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:57.294414+0000","last_undegraded":"2024-09-10T15:53:57.294414+0000","last_fullsized":"2024-09-10T15:53:57.294414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5374,"log_dups_size":0,"ondisk_log_size":5374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2133,"num_write_kb":8532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5527","reported_seq":7829,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:59.395767+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:59.395767+0000","last_peered":"2024-09-10T15:53:59.395767+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:59.395767+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5527,"log_dups_size":0,"ondisk_log_size":5527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2901,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3504,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1089,"num_bytes_recovered":4460544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2901},{"shards":"2","objects":2617}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5288","reported_seq":5409,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.330086+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:57.330086+0000","last_peered":"2024-09-10T15:53:57.330086+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:57.330086+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5288,"log_dups_size":0,"ondisk_log_size":5288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21626880,"num_objects":5280,"num_object_clones":0,"num_object_copies":10560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2152,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1417},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5381","reported_seq":5500,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:59.299077+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:59.299077+0000","last_peered":"2024-09-10T15:53:59.299077+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:59.299077+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22020096,"num_objects":5376,"num_object_clones":0,"num_object_copies":10752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3355,"num_write_kb":13420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1544}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5305","reported_seq":5426,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:59.403657+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:59.403657+0000","last_peered":"2024-09-10T15:53:59.403657+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:59.403657+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5305,"log_dups_size":0,"ondisk_log_size":5305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21704704,"num_objects":5299,"num_object_clones":0,"num_object_copies":10598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3174,"num_write_kb":12696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1470}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5335","reported_seq":5456,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.377198+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:57.377198+0000","last_peered":"2024-09-10T15:53:57.377198+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:57.377198+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2527,"num_write_kb":10108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1500},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5377","reported_seq":5535,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.340922+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:57.340922+0000","last_peered":"2024-09-10T15:53:57.340922+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:57.340922+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5377,"log_dups_size":0,"ondisk_log_size":5377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21991424,"num_objects":5369,"num_object_clones":0,"num_object_copies":10738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1795,"num_write_kb":7180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1452},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5299","reported_seq":5433,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.330376+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:57.330376+0000","last_peered":"2024-09-10T15:53:57.330376+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:57.330376+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21688320,"num_objects":5295,"num_object_clones":0,"num_object_copies":10590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1905,"num_write_kb":7620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1418},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5437","reported_seq":5570,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.371233+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:57.371233+0000","last_peered":"2024-09-10T15:53:57.371233+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:57.371233+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5437,"log_dups_size":0,"ondisk_log_size":5437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22221283,"num_objects":5426,"num_object_clones":0,"num_object_copies":10852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":9024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1423},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":350736867,"num_objects":85630,"num_object_clones":0,"num_object_copies":171260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2901,"num_objects_unfound":0,"num_objects_dirty":85630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47232,"num_write_kb":188928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1091,"num_bytes_recovered":4468736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":705478656,"data_stored":705388331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85711,"ondisk_log_size":85711,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626442,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163376,"kb_used_data":73440,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94208464,"statfs":{"total":96636764160,"available":96469467136,"internally_reserved":0,"allocated":75202560,"data_stored":74867416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92073646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149726,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501356,"kb_used_data":317020,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93870484,"statfs":{"total":96636764160,"available":96123375616,"internally_reserved":0,"allocated":324628480,"data_stored":324372822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188739245},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,8,3,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953496,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458916,"kb_used_data":296788,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93912924,"statfs":{"total":96636764160,"available":96166834176,"internally_reserved":0,"allocated":303910912,"data_stored":303655254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,5,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":325337088,"data_stored":325333475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75677696,"data_stored":75594597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":304463872,"data_stored":304460259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:00.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:54:00.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'5372", 'reported_seq': 5443, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:57.374998+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:53:57.374998+0000', 'last_peered': '2024-09-10T15:53:57.374998+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:53:57.374998+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 5372, 'log_dups_size': 0, 'ondisk_log_size': 5372, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22003712, 'num_objects': 5372, 'num_object_clones': 0, 'num_object_copies': 10744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5372, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5372, 'num_write_kb': 21488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1399}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:00.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:54:00.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'5314", 'reported_seq': 5432, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:59.355349+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:53:59.355349+0000', 'last_peered': '2024-09-10T15:53:59.355349+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:53:59.355349+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 5314, 'log_dups_size': 0, 'ondisk_log_size': 5314, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21749760, 'num_objects': 5310, 'num_object_clones': 0, 'num_object_copies': 10620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5310, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3582, 'num_write_kb': 14328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 1508}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:00.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:54:00.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'5329", 'reported_seq': 5457, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:57.334288+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:53:57.334288+0000', 'last_peered': '2024-09-10T15:53:57.334288+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:53:57.334288+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 5329, 'log_dups_size': 0, 'ondisk_log_size': 5329, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21807104, 'num_objects': 5324, 'num_object_clones': 0, 'num_object_copies': 10648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5324, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2047, 'num_write_kb': 8188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1456}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:00.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:54:00.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'5250", 'reported_seq': 5385, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:57.336072+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:53:57.336072+0000', 'last_peered': '2024-09-10T15:53:57.336072+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:53:57.336072+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 5250, 'log_dups_size': 0, 'ondisk_log_size': 5250, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21479424, 'num_objects': 5244, 'num_object_clones': 0, 'num_object_copies': 10488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5244, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1900, 'num_write_kb': 7600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1410}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:00.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:54:00.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'5527", 'reported_seq': 7829, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:59.395767+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:53:59.395767+0000', 'last_peered': '2024-09-10T15:53:59.395767+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:53:59.395767+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 5527, 'log_dups_size': 0, 'ondisk_log_size': 5527, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22609920, 'num_objects': 5520, 'num_object_clones': 0, 'num_object_copies': 11040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2901, 'num_objects_unfound': 0, 'num_objects_dirty': 5520, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3504, 'num_write_kb': 14016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1089, 'num_bytes_recovered': 4460544, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2901}, {'shards': '2', 'objects': 2617}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:00.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:54:00.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'5288", 'reported_seq': 5409, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:57.330086+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:53:57.330086+0000', 'last_peered': '2024-09-10T15:53:57.330086+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:53:57.330086+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 5288, 'log_dups_size': 0, 'ondisk_log_size': 5288, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21626880, 'num_objects': 5280, 'num_object_clones': 0, 'num_object_copies': 10560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5280, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2152, 'num_write_kb': 8608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1417}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:00.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:54:00.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'5381", 'reported_seq': 5500, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:59.299077+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:53:59.299077+0000', 'last_peered': '2024-09-10T15:53:59.299077+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:53:59.299077+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 5381, 'log_dups_size': 0, 'ondisk_log_size': 5381, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22020096, 'num_objects': 5376, 'num_object_clones': 0, 'num_object_copies': 10752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5376, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3355, 'num_write_kb': 13420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 1544}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:00.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:54:00.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'5305", 'reported_seq': 5426, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:59.403657+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:53:59.403657+0000', 'last_peered': '2024-09-10T15:53:59.403657+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:53:59.403657+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 5305, 'log_dups_size': 0, 'ondisk_log_size': 5305, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21704704, 'num_objects': 5299, 'num_object_clones': 0, 'num_object_copies': 10598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5299, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3174, 'num_write_kb': 12696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 1470}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:00.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:54:00.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'5335", 'reported_seq': 5456, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:57.377198+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:53:57.377198+0000', 'last_peered': '2024-09-10T15:53:57.377198+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:53:57.377198+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 5335, 'log_dups_size': 0, 'ondisk_log_size': 5335, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21843968, 'num_objects': 5333, 'num_object_clones': 0, 'num_object_copies': 10666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5333, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2527, 'num_write_kb': 10108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1500}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:00.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:54:00.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'5377", 'reported_seq': 5535, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:57.340922+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:53:57.340922+0000', 'last_peered': '2024-09-10T15:53:57.340922+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:53:57.340922+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 5377, 'log_dups_size': 0, 'ondisk_log_size': 5377, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21991424, 'num_objects': 5369, 'num_object_clones': 0, 'num_object_copies': 10738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5369, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1795, 'num_write_kb': 7180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1452}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:00.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:54:00.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'5299", 'reported_seq': 5433, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:57.330376+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:53:57.330376+0000', 'last_peered': '2024-09-10T15:53:57.330376+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:53:57.330376+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 5299, 'log_dups_size': 0, 'ondisk_log_size': 5299, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21688320, 'num_objects': 5295, 'num_object_clones': 0, 'num_object_copies': 10590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1905, 'num_write_kb': 7620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1418}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:00.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:54:00.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'5437", 'reported_seq': 5570, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:53:57.371233+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:53:57.371233+0000', 'last_peered': '2024-09-10T15:53:57.371233+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:53:57.371233+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 5437, 'log_dups_size': 0, 'ondisk_log_size': 5437, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22221283, 'num_objects': 5426, 'num_object_clones': 0, 'num_object_copies': 10852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2256, 'num_write_kb': 9024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1423}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:00.979 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:54:01.696 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:01.696 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/171264 objects degraded (24.905%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":167,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":85632,"data_bytes":350736886,"bytes_used":1150615552,"bytes_avail":288759676928,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":171264,"degraded_ratio":0.24904825298953662,"misplaced_objects":2901,"misplaced_total":171264,"misplaced_ratio":0.016938761210762331,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":150140,"recovering_keys_per_sec":0,"num_objects_recovered":440,"num_bytes_recovered":1802240,"num_keys_recovered":0,"write_bytes_sec":2565701,"read_op_per_sec":0,"write_op_per_sec":626},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:53:42.448843+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:54:01.725 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:54:01.725 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:01.810 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 84 20 87712 87692 4.07725 2.14453 0.0269021 0.0191537 2024-09-10T15:54:02.354 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:02.355 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:02.380 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-10T15:54:00.453815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":350736886,"num_objects":85632,"num_object_clones":0,"num_object_copies":171264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2901,"num_objects_unfound":0,"num_objects_dirty":85632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47234,"num_write_kb":188930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1093,"num_bytes_recovered":4468755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85714,"ondisk_log_size":85714,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1123648,"kb_used_data":687248,"kb_used_omap":12,"kb_used_meta":436339,"kb_avail":281991872,"statfs":{"total":289910292480,"available":288759676928,"internally_reserved":0,"allocated":703741952,"data_stored":702895492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":446811141},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,9,11,13,8,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30797824,"num_objects":7519,"num_object_clones":0,"num_object_copies":15038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-440,"num_objects_unfound":0,"num_objects_dirty":7519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7519,"num_write_kb":30076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003667"},"pg_stats":[{"pgid":"2.f","version":"44'5372","reported_seq":5443,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.374998+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:53:57.374998+0000","last_peered":"2024-09-10T15:53:57.374998+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:53:57.374998+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5372,"log_dups_size":0,"ondisk_log_size":5372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5372,"num_write_kb":21488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1399},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5537","reported_seq":5639,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:59.335858+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:53:59.335858+0000","last_peered":"2024-09-10T15:53:59.335858+0000","last_clean":"2024-09-10T15:53:59.335858+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:53:59.335858+0000","last_undegraded":"2024-09-10T15:53:59.335858+0000","last_fullsized":"2024-09-10T15:53:59.335858+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5537,"log_dups_size":0,"ondisk_log_size":5537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22654976,"num_objects":5531,"num_object_clones":0,"num_object_copies":11062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3873,"num_write_kb":15492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5314","reported_seq":5432,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:59.355349+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:53:59.355349+0000","last_peered":"2024-09-10T15:53:59.355349+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:53:59.355349+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5314,"log_dups_size":0,"ondisk_log_size":5314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21749760,"num_objects":5310,"num_object_clones":0,"num_object_copies":10620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3582,"num_write_kb":14328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5343","reported_seq":5417,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:59.360508+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:53:59.360508+0000","last_peered":"2024-09-10T15:53:59.360508+0000","last_clean":"2024-09-10T15:53:59.360508+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:53:59.360508+0000","last_undegraded":"2024-09-10T15:53:59.360508+0000","last_fullsized":"2024-09-10T15:53:59.360508+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5343,"log_dups_size":0,"ondisk_log_size":5343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21884928,"num_objects":5343,"num_object_clones":0,"num_object_copies":10686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5343,"num_write_kb":21372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5243","reported_seq":5364,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:57.375604+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:53:57.375604+0000","last_peered":"2024-09-10T15:53:57.375604+0000","last_clean":"2024-09-10T15:53:57.375604+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:53:57.375604+0000","last_undegraded":"2024-09-10T15:53:57.375604+0000","last_fullsized":"2024-09-10T15:53:57.375604+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5243,"log_dups_size":0,"ondisk_log_size":5243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2314,"num_write_kb":9256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5329","reported_seq":5457,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.334288+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:53:57.334288+0000","last_peered":"2024-09-10T15:53:57.334288+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:53:57.334288+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5329,"log_dups_size":0,"ondisk_log_size":5329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807104,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2047,"num_write_kb":8188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1456},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5250","reported_seq":5385,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.336072+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:53:57.336072+0000","last_peered":"2024-09-10T15:53:57.336072+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:53:57.336072+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5250,"log_dups_size":0,"ondisk_log_size":5250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21479424,"num_objects":5244,"num_object_clones":0,"num_object_copies":10488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1900,"num_write_kb":7600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1410},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5374","reported_seq":5500,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:57.294414+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:53:57.294414+0000","last_peered":"2024-09-10T15:53:57.294414+0000","last_clean":"2024-09-10T15:53:57.294414+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:53:57.294414+0000","last_undegraded":"2024-09-10T15:53:57.294414+0000","last_fullsized":"2024-09-10T15:53:57.294414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5374,"log_dups_size":0,"ondisk_log_size":5374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2133,"num_write_kb":8532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5527","reported_seq":7829,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:59.395767+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:53:59.395767+0000","last_peered":"2024-09-10T15:53:59.395767+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:53:59.395767+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5527,"log_dups_size":0,"ondisk_log_size":5527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2901,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3504,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1089,"num_bytes_recovered":4460544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2901},{"shards":"2","objects":2617}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5288","reported_seq":5409,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.330086+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:53:57.330086+0000","last_peered":"2024-09-10T15:53:57.330086+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:53:57.330086+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5288,"log_dups_size":0,"ondisk_log_size":5288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21626880,"num_objects":5280,"num_object_clones":0,"num_object_copies":10560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2152,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1417},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5381","reported_seq":5500,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:59.299077+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:53:59.299077+0000","last_peered":"2024-09-10T15:53:59.299077+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:53:59.299077+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22020096,"num_objects":5376,"num_object_clones":0,"num_object_copies":10752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3355,"num_write_kb":13420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1544}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5305","reported_seq":5426,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:59.403657+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:53:59.403657+0000","last_peered":"2024-09-10T15:53:59.403657+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:53:59.403657+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5305,"log_dups_size":0,"ondisk_log_size":5305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21704704,"num_objects":5299,"num_object_clones":0,"num_object_copies":10598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3174,"num_write_kb":12696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1470}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5335","reported_seq":5456,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.377198+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:53:57.377198+0000","last_peered":"2024-09-10T15:53:57.377198+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:53:57.377198+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2527,"num_write_kb":10108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1500},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5377","reported_seq":5535,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.340922+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:53:57.340922+0000","last_peered":"2024-09-10T15:53:57.340922+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:53:57.340922+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5377,"log_dups_size":0,"ondisk_log_size":5377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21991424,"num_objects":5369,"num_object_clones":0,"num_object_copies":10738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1795,"num_write_kb":7180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1452},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5299","reported_seq":5433,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.330376+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:53:57.330376+0000","last_peered":"2024-09-10T15:53:57.330376+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:53:57.330376+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21688320,"num_objects":5295,"num_object_clones":0,"num_object_copies":10590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1905,"num_write_kb":7620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1418},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5437","reported_seq":5570,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:53:57.371233+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:53:57.371233+0000","last_peered":"2024-09-10T15:53:57.371233+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:53:57.371233+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5437,"log_dups_size":0,"ondisk_log_size":5437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22221283,"num_objects":5426,"num_object_clones":0,"num_object_copies":10852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":9024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1423},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":350736867,"num_objects":85630,"num_object_clones":0,"num_object_copies":171260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2901,"num_objects_unfound":0,"num_objects_dirty":85630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47232,"num_write_kb":188928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1091,"num_bytes_recovered":4468736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":705478656,"data_stored":705388331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85711,"ondisk_log_size":85711,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626442,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163376,"kb_used_data":73440,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94208464,"statfs":{"total":96636764160,"available":96469467136,"internally_reserved":0,"allocated":75202560,"data_stored":74867416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92073646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149726,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501356,"kb_used_data":317020,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93870484,"statfs":{"total":96636764160,"available":96123375616,"internally_reserved":0,"allocated":324628480,"data_stored":324372822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188739245},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,8,3,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953496,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458916,"kb_used_data":296788,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93912924,"statfs":{"total":96636764160,"available":96166834176,"internally_reserved":0,"allocated":303910912,"data_stored":303655254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,5,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":325337088,"data_stored":325333475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75677696,"data_stored":75594597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":304463872,"data_stored":304460259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:02.810 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 85 20 88252 88232 4.0541 2.10938 0.0184228 0.0192616 2024-09-10T15:54:03.811 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 86 20 88859 88839 4.03453 2.37109 0.0161019 0.0193557 2024-09-10T15:54:04.811 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 87 20 89484 89464 4.01622 2.44141 0.0248393 0.0194429 2024-09-10T15:54:05.382 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:05.811 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 88 20 90068 90048 3.9965 2.28125 0.0238831 0.0195385 2024-09-10T15:54:05.926 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:05.927 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:05.955 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-10T15:54:04.455764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":362320374,"num_objects":88460,"num_object_clones":0,"num_object_copies":176920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2746,"num_objects_unfound":0,"num_objects_dirty":88460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50062,"num_write_kb":200242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1248,"num_bytes_recovered":5103635,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88542,"ondisk_log_size":88542,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1165048,"kb_used_data":710600,"kb_used_omap":12,"kb_used_meta":454387,"kb_avail":281950472,"statfs":{"total":289910292480,"available":288717283328,"internally_reserved":0,"allocated":727654400,"data_stored":726807940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":465292293},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,5,8,7,12,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30167040,"num_objects":7365,"num_object_clones":0,"num_object_copies":14730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-436,"num_objects_unfound":0,"num_objects_dirty":7365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7365,"num_write_kb":29460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":436,"num_bytes_recovered":1785856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004475"},"pg_stats":[{"pgid":"2.f","version":"44'5533","reported_seq":5603,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.417586+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:02.417586+0000","last_peered":"2024-09-10T15:54:02.417586+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:02.417586+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5533,"log_dups_size":0,"ondisk_log_size":5533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5533,"num_write_kb":22132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1560},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5734","reported_seq":5836,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:04.358623+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:04.358623+0000","last_peered":"2024-09-10T15:54:04.358623+0000","last_clean":"2024-09-10T15:54:04.358623+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:04.358623+0000","last_undegraded":"2024-09-10T15:54:04.358623+0000","last_fullsized":"2024-09-10T15:54:04.358623+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5734,"log_dups_size":0,"ondisk_log_size":5734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":5728,"num_object_clones":0,"num_object_copies":11456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4070,"num_write_kb":16280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5491","reported_seq":5609,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:04.418606+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:04.418606+0000","last_peered":"2024-09-10T15:54:04.418606+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:04.418606+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5491,"log_dups_size":0,"ondisk_log_size":5491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22474752,"num_objects":5487,"num_object_clones":0,"num_object_copies":10974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3759,"num_write_kb":15036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1685}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5516","reported_seq":5590,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:04.382106+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:04.382106+0000","last_peered":"2024-09-10T15:54:04.382106+0000","last_clean":"2024-09-10T15:54:04.382106+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:04.382106+0000","last_undegraded":"2024-09-10T15:54:04.382106+0000","last_fullsized":"2024-09-10T15:54:04.382106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5516,"log_dups_size":0,"ondisk_log_size":5516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22593536,"num_objects":5516,"num_object_clones":0,"num_object_copies":11032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5516,"num_write_kb":22064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5421","reported_seq":5542,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:02.363549+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:02.363549+0000","last_peered":"2024-09-10T15:54:02.363549+0000","last_clean":"2024-09-10T15:54:02.363549+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:02.363549+0000","last_undegraded":"2024-09-10T15:54:02.363549+0000","last_fullsized":"2024-09-10T15:54:02.363549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5421,"log_dups_size":0,"ondisk_log_size":5421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22175744,"num_objects":5414,"num_object_clones":0,"num_object_copies":10828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2492,"num_write_kb":9968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5508","reported_seq":5637,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.386565+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:02.386565+0000","last_peered":"2024-09-10T15:54:02.386565+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:02.386565+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5508,"log_dups_size":0,"ondisk_log_size":5508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540288,"num_objects":5503,"num_object_clones":0,"num_object_copies":11006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2226,"num_write_kb":8904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1635},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5396","reported_seq":5531,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.320349+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:02.320349+0000","last_peered":"2024-09-10T15:54:02.320349+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:02.320349+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5396,"log_dups_size":0,"ondisk_log_size":5396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22077440,"num_objects":5390,"num_object_clones":0,"num_object_copies":10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2046,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1556},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5545","reported_seq":5671,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:02.320147+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:02.320147+0000","last_peered":"2024-09-10T15:54:02.320147+0000","last_clean":"2024-09-10T15:54:02.320147+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:02.320147+0000","last_undegraded":"2024-09-10T15:54:02.320147+0000","last_fullsized":"2024-09-10T15:54:02.320147+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5545,"log_dups_size":0,"ondisk_log_size":5545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2304,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5686","reported_seq":8298,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:04.426981+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:04.426981+0000","last_peered":"2024-09-10T15:54:04.426981+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:04.426981+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5686,"log_dups_size":0,"ondisk_log_size":5686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23261184,"num_objects":5679,"num_object_clones":0,"num_object_copies":11358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2746,"num_objects_unfound":0,"num_objects_dirty":5679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3663,"num_write_kb":14652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1244,"num_bytes_recovered":5095424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2746},{"shards":"2","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5468","reported_seq":5586,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.426073+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:02.426073+0000","last_peered":"2024-09-10T15:54:02.426073+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:02.426073+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22364160,"num_objects":5460,"num_object_clones":0,"num_object_copies":10920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2332,"num_write_kb":9328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1597},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5560","reported_seq":5679,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:04.370717+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:04.370717+0000","last_peered":"2024-09-10T15:54:04.370717+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:04.370717+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5560,"log_dups_size":0,"ondisk_log_size":5560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22753280,"num_objects":5555,"num_object_clones":0,"num_object_copies":11110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3534,"num_write_kb":14136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1723}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5473","reported_seq":5594,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:04.379473+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:04.379473+0000","last_peered":"2024-09-10T15:54:04.379473+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:04.379473+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5473,"log_dups_size":0,"ondisk_log_size":5473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22392832,"num_objects":5467,"num_object_clones":0,"num_object_copies":10934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3342,"num_write_kb":13368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1638}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5538","reported_seq":5660,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.334207+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:02.334207+0000","last_peered":"2024-09-10T15:54:02.334207+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:02.334207+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5538,"log_dups_size":0,"ondisk_log_size":5538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22675456,"num_objects":5536,"num_object_clones":0,"num_object_copies":11072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2730,"num_write_kb":10920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1703},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5574","reported_seq":5731,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.398616+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:02.398616+0000","last_peered":"2024-09-10T15:54:02.398616+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:02.398616+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5574,"log_dups_size":0,"ondisk_log_size":5574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22798336,"num_objects":5566,"num_object_clones":0,"num_object_copies":11132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1992,"num_write_kb":7968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1649},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5473","reported_seq":5607,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.381703+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:02.381703+0000","last_peered":"2024-09-10T15:54:02.381703+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:02.381703+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5473,"log_dups_size":0,"ondisk_log_size":5473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22401024,"num_objects":5469,"num_object_clones":0,"num_object_copies":10938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2079,"num_write_kb":8316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1592},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5623","reported_seq":5756,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.381152+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:02.381152+0000","last_peered":"2024-09-10T15:54:02.381152+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:02.381152+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5623,"log_dups_size":0,"ondisk_log_size":5623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22983139,"num_objects":5612,"num_object_clones":0,"num_object_copies":11224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2442,"num_write_kb":9768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1609},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":362320355,"num_objects":88458,"num_object_clones":0,"num_object_copies":176916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2746,"num_objects_unfound":0,"num_objects_dirty":88458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50060,"num_write_kb":200240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1246,"num_bytes_recovered":5103616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":729223168,"data_stored":729132843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88539,"ondisk_log_size":88539,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626443,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":194696,"kb_used_data":82552,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94177144,"statfs":{"total":96636764160,"available":96437395456,"internally_reserved":0,"allocated":84533248,"data_stored":84198104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":114814638},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149727,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501796,"kb_used_data":325780,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93870044,"statfs":{"total":96636764160,"available":96122925056,"internally_reserved":0,"allocated":333598720,"data_stored":333343062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180219565},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,3,2,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468556,"kb_used_data":302268,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93903284,"statfs":{"total":96636764160,"available":96156962816,"internally_reserved":0,"allocated":309522432,"data_stored":309266774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":334163968,"data_stored":334160355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":85020672,"data_stored":84937573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310038528,"data_stored":310034915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:05.957 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:06.534 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:06.536 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:06.566 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-10T15:54:06.456371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":362320374,"num_objects":88460,"num_object_clones":0,"num_object_copies":176920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2746,"num_objects_unfound":0,"num_objects_dirty":88460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50062,"num_write_kb":200242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1248,"num_bytes_recovered":5103635,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88542,"ondisk_log_size":88542,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1165048,"kb_used_data":710600,"kb_used_omap":12,"kb_used_meta":454387,"kb_avail":281950472,"statfs":{"total":289910292480,"available":288717283328,"internally_reserved":0,"allocated":727654400,"data_stored":726807940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":465292293},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,5,8,7,12,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25288704,"num_objects":6174,"num_object_clones":0,"num_object_copies":12348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-305,"num_objects_unfound":0,"num_objects_dirty":6174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6174,"num_write_kb":24696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004316"},"pg_stats":[{"pgid":"2.f","version":"44'5533","reported_seq":5603,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.417586+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:02.417586+0000","last_peered":"2024-09-10T15:54:02.417586+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:02.417586+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5533,"log_dups_size":0,"ondisk_log_size":5533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5533,"num_write_kb":22132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1560},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5734","reported_seq":5836,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:04.358623+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:04.358623+0000","last_peered":"2024-09-10T15:54:04.358623+0000","last_clean":"2024-09-10T15:54:04.358623+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:04.358623+0000","last_undegraded":"2024-09-10T15:54:04.358623+0000","last_fullsized":"2024-09-10T15:54:04.358623+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5734,"log_dups_size":0,"ondisk_log_size":5734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":5728,"num_object_clones":0,"num_object_copies":11456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4070,"num_write_kb":16280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5491","reported_seq":5609,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:04.418606+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:04.418606+0000","last_peered":"2024-09-10T15:54:04.418606+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:04.418606+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5491,"log_dups_size":0,"ondisk_log_size":5491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22474752,"num_objects":5487,"num_object_clones":0,"num_object_copies":10974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3759,"num_write_kb":15036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1685}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5516","reported_seq":5590,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:04.382106+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:04.382106+0000","last_peered":"2024-09-10T15:54:04.382106+0000","last_clean":"2024-09-10T15:54:04.382106+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:04.382106+0000","last_undegraded":"2024-09-10T15:54:04.382106+0000","last_fullsized":"2024-09-10T15:54:04.382106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5516,"log_dups_size":0,"ondisk_log_size":5516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22593536,"num_objects":5516,"num_object_clones":0,"num_object_copies":11032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5516,"num_write_kb":22064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5421","reported_seq":5542,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:02.363549+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:02.363549+0000","last_peered":"2024-09-10T15:54:02.363549+0000","last_clean":"2024-09-10T15:54:02.363549+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:02.363549+0000","last_undegraded":"2024-09-10T15:54:02.363549+0000","last_fullsized":"2024-09-10T15:54:02.363549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5421,"log_dups_size":0,"ondisk_log_size":5421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22175744,"num_objects":5414,"num_object_clones":0,"num_object_copies":10828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2492,"num_write_kb":9968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5508","reported_seq":5637,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.386565+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:02.386565+0000","last_peered":"2024-09-10T15:54:02.386565+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:02.386565+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5508,"log_dups_size":0,"ondisk_log_size":5508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540288,"num_objects":5503,"num_object_clones":0,"num_object_copies":11006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2226,"num_write_kb":8904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1635},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5396","reported_seq":5531,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.320349+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:02.320349+0000","last_peered":"2024-09-10T15:54:02.320349+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:02.320349+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5396,"log_dups_size":0,"ondisk_log_size":5396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22077440,"num_objects":5390,"num_object_clones":0,"num_object_copies":10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2046,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1556},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5545","reported_seq":5671,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:02.320147+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:02.320147+0000","last_peered":"2024-09-10T15:54:02.320147+0000","last_clean":"2024-09-10T15:54:02.320147+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:02.320147+0000","last_undegraded":"2024-09-10T15:54:02.320147+0000","last_fullsized":"2024-09-10T15:54:02.320147+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5545,"log_dups_size":0,"ondisk_log_size":5545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2304,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5686","reported_seq":8298,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:04.426981+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:04.426981+0000","last_peered":"2024-09-10T15:54:04.426981+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:04.426981+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5686,"log_dups_size":0,"ondisk_log_size":5686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23261184,"num_objects":5679,"num_object_clones":0,"num_object_copies":11358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2746,"num_objects_unfound":0,"num_objects_dirty":5679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3663,"num_write_kb":14652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1244,"num_bytes_recovered":5095424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2746},{"shards":"2","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5468","reported_seq":5586,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.426073+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:02.426073+0000","last_peered":"2024-09-10T15:54:02.426073+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:02.426073+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22364160,"num_objects":5460,"num_object_clones":0,"num_object_copies":10920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2332,"num_write_kb":9328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1597},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5560","reported_seq":5679,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:04.370717+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:04.370717+0000","last_peered":"2024-09-10T15:54:04.370717+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:04.370717+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5560,"log_dups_size":0,"ondisk_log_size":5560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22753280,"num_objects":5555,"num_object_clones":0,"num_object_copies":11110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3534,"num_write_kb":14136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1723}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5473","reported_seq":5594,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:04.379473+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:04.379473+0000","last_peered":"2024-09-10T15:54:04.379473+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:04.379473+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5473,"log_dups_size":0,"ondisk_log_size":5473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22392832,"num_objects":5467,"num_object_clones":0,"num_object_copies":10934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3342,"num_write_kb":13368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1638}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5538","reported_seq":5660,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.334207+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:02.334207+0000","last_peered":"2024-09-10T15:54:02.334207+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:02.334207+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5538,"log_dups_size":0,"ondisk_log_size":5538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22675456,"num_objects":5536,"num_object_clones":0,"num_object_copies":11072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2730,"num_write_kb":10920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1703},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5574","reported_seq":5731,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.398616+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:02.398616+0000","last_peered":"2024-09-10T15:54:02.398616+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:02.398616+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5574,"log_dups_size":0,"ondisk_log_size":5574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22798336,"num_objects":5566,"num_object_clones":0,"num_object_copies":11132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1992,"num_write_kb":7968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1649},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5473","reported_seq":5607,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.381703+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:02.381703+0000","last_peered":"2024-09-10T15:54:02.381703+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:02.381703+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5473,"log_dups_size":0,"ondisk_log_size":5473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22401024,"num_objects":5469,"num_object_clones":0,"num_object_copies":10938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2079,"num_write_kb":8316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1592},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5623","reported_seq":5756,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.381152+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:02.381152+0000","last_peered":"2024-09-10T15:54:02.381152+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:02.381152+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5623,"log_dups_size":0,"ondisk_log_size":5623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22983139,"num_objects":5612,"num_object_clones":0,"num_object_copies":11224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2442,"num_write_kb":9768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1609},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":362320355,"num_objects":88458,"num_object_clones":0,"num_object_copies":176916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2746,"num_objects_unfound":0,"num_objects_dirty":88458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50060,"num_write_kb":200240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1246,"num_bytes_recovered":5103616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":729223168,"data_stored":729132843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88539,"ondisk_log_size":88539,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626443,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":194696,"kb_used_data":82552,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94177144,"statfs":{"total":96636764160,"available":96437395456,"internally_reserved":0,"allocated":84533248,"data_stored":84198104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":114814638},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149727,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501796,"kb_used_data":325780,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93870044,"statfs":{"total":96636764160,"available":96122925056,"internally_reserved":0,"allocated":333598720,"data_stored":333343062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180219565},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,3,2,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468556,"kb_used_data":302268,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93903284,"statfs":{"total":96636764160,"available":96156962816,"internally_reserved":0,"allocated":309522432,"data_stored":309266774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":334163968,"data_stored":334160355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":85020672,"data_stored":84937573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310038528,"data_stored":310034915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:06.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:54:06.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'5533", 'reported_seq': 5603, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:02.417586+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:54:02.417586+0000', 'last_peered': '2024-09-10T15:54:02.417586+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:54:02.417586+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 5533, 'log_dups_size': 0, 'ondisk_log_size': 5533, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22663168, 'num_objects': 5533, 'num_object_clones': 0, 'num_object_copies': 11066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5533, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5533, 'num_write_kb': 22132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1560}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:06.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:54:06.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'5491", 'reported_seq': 5609, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:04.418606+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:54:04.418606+0000', 'last_peered': '2024-09-10T15:54:04.418606+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:54:04.418606+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 5491, 'log_dups_size': 0, 'ondisk_log_size': 5491, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22474752, 'num_objects': 5487, 'num_object_clones': 0, 'num_object_copies': 10974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5487, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3759, 'num_write_kb': 15036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 1685}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:06.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:54:06.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'5508", 'reported_seq': 5637, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:02.386565+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:54:02.386565+0000', 'last_peered': '2024-09-10T15:54:02.386565+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:54:02.386565+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 5508, 'log_dups_size': 0, 'ondisk_log_size': 5508, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22540288, 'num_objects': 5503, 'num_object_clones': 0, 'num_object_copies': 11006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5503, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2226, 'num_write_kb': 8904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1635}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:06.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:54:06.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'5396", 'reported_seq': 5531, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:02.320349+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:54:02.320349+0000', 'last_peered': '2024-09-10T15:54:02.320349+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:54:02.320349+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 5396, 'log_dups_size': 0, 'ondisk_log_size': 5396, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22077440, 'num_objects': 5390, 'num_object_clones': 0, 'num_object_copies': 10780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5390, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2046, 'num_write_kb': 8184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1556}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:06.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:54:06.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'5686", 'reported_seq': 8298, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:04.426981+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:54:04.426981+0000', 'last_peered': '2024-09-10T15:54:04.426981+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:54:04.426981+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 5686, 'log_dups_size': 0, 'ondisk_log_size': 5686, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23261184, 'num_objects': 5679, 'num_object_clones': 0, 'num_object_copies': 11358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2746, 'num_objects_unfound': 0, 'num_objects_dirty': 5679, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3663, 'num_write_kb': 14652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1244, 'num_bytes_recovered': 5095424, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2746}, {'shards': '2', 'objects': 2931}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:06.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:54:06.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'5468", 'reported_seq': 5586, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:02.426073+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:54:02.426073+0000', 'last_peered': '2024-09-10T15:54:02.426073+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:54:02.426073+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 5468, 'log_dups_size': 0, 'ondisk_log_size': 5468, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22364160, 'num_objects': 5460, 'num_object_clones': 0, 'num_object_copies': 10920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5460, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2332, 'num_write_kb': 9328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1597}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:06.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:54:06.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'5560", 'reported_seq': 5679, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:04.370717+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:54:04.370717+0000', 'last_peered': '2024-09-10T15:54:04.370717+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:54:04.370717+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 5560, 'log_dups_size': 0, 'ondisk_log_size': 5560, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22753280, 'num_objects': 5555, 'num_object_clones': 0, 'num_object_copies': 11110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5555, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3534, 'num_write_kb': 14136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 1723}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:06.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:54:06.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'5473", 'reported_seq': 5594, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:04.379473+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:54:04.379473+0000', 'last_peered': '2024-09-10T15:54:04.379473+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:54:04.379473+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 5473, 'log_dups_size': 0, 'ondisk_log_size': 5473, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22392832, 'num_objects': 5467, 'num_object_clones': 0, 'num_object_copies': 10934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3342, 'num_write_kb': 13368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 1638}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:06.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:54:06.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'5538", 'reported_seq': 5660, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:02.334207+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:54:02.334207+0000', 'last_peered': '2024-09-10T15:54:02.334207+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:54:02.334207+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 5538, 'log_dups_size': 0, 'ondisk_log_size': 5538, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22675456, 'num_objects': 5536, 'num_object_clones': 0, 'num_object_copies': 11072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5536, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2730, 'num_write_kb': 10920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1703}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:06.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:54:06.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'5574", 'reported_seq': 5731, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:02.398616+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:54:02.398616+0000', 'last_peered': '2024-09-10T15:54:02.398616+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:54:02.398616+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 5574, 'log_dups_size': 0, 'ondisk_log_size': 5574, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22798336, 'num_objects': 5566, 'num_object_clones': 0, 'num_object_copies': 11132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1992, 'num_write_kb': 7968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1649}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:06.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:54:06.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'5473", 'reported_seq': 5607, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:02.381703+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:54:02.381703+0000', 'last_peered': '2024-09-10T15:54:02.381703+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:54:02.381703+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 5473, 'log_dups_size': 0, 'ondisk_log_size': 5473, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22401024, 'num_objects': 5469, 'num_object_clones': 0, 'num_object_copies': 10938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5469, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2079, 'num_write_kb': 8316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1592}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:06.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:54:06.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'5623", 'reported_seq': 5756, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:02.381152+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:54:02.381152+0000', 'last_peered': '2024-09-10T15:54:02.381152+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:54:02.381152+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 5623, 'log_dups_size': 0, 'ondisk_log_size': 5623, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22983139, 'num_objects': 5612, 'num_object_clones': 0, 'num_object_copies': 11224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2442, 'num_write_kb': 9768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1609}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:06.570 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:54:06.811 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 89 20 90618 90598 3.97574 2.14844 0.025126 0.0196424 2024-09-10T15:54:07.260 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:07.260 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/176920 objects degraded (24.109%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":173,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":88460,"data_bytes":362320374,"bytes_used":1193009152,"bytes_avail":288717283328,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":176920,"degraded_ratio":0.24108636671942121,"misplaced_objects":2746,"misplaced_total":176920,"misplaced_ratio":0.015521139498078228,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":104069,"recovering_keys_per_sec":0,"num_objects_recovered":305,"num_bytes_recovered":1249280,"num_keys_recovered":0,"write_bytes_sec":2106634,"read_op_per_sec":0,"write_op_per_sec":514},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:53:42.448843+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:54:07.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:54:07.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:07.811 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 90 20 91166 91146 3.95535 2.14062 0.0621496 0.019745 2024-09-10T15:54:07.844 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:07.844 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:07.866 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-10T15:54:06.456371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":362320374,"num_objects":88460,"num_object_clones":0,"num_object_copies":176920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2746,"num_objects_unfound":0,"num_objects_dirty":88460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50062,"num_write_kb":200242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1248,"num_bytes_recovered":5103635,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88542,"ondisk_log_size":88542,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1165048,"kb_used_data":710600,"kb_used_omap":12,"kb_used_meta":454387,"kb_avail":281950472,"statfs":{"total":289910292480,"available":288717283328,"internally_reserved":0,"allocated":727654400,"data_stored":726807940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":465292293},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,5,8,7,12,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25288704,"num_objects":6174,"num_object_clones":0,"num_object_copies":12348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-305,"num_objects_unfound":0,"num_objects_dirty":6174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6174,"num_write_kb":24696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004316"},"pg_stats":[{"pgid":"2.f","version":"44'5533","reported_seq":5603,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.417586+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:02.417586+0000","last_peered":"2024-09-10T15:54:02.417586+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:02.417586+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5533,"log_dups_size":0,"ondisk_log_size":5533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5533,"num_write_kb":22132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1560},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5734","reported_seq":5836,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:04.358623+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:04.358623+0000","last_peered":"2024-09-10T15:54:04.358623+0000","last_clean":"2024-09-10T15:54:04.358623+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:04.358623+0000","last_undegraded":"2024-09-10T15:54:04.358623+0000","last_fullsized":"2024-09-10T15:54:04.358623+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5734,"log_dups_size":0,"ondisk_log_size":5734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":5728,"num_object_clones":0,"num_object_copies":11456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4070,"num_write_kb":16280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5491","reported_seq":5609,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:04.418606+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:04.418606+0000","last_peered":"2024-09-10T15:54:04.418606+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:04.418606+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5491,"log_dups_size":0,"ondisk_log_size":5491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22474752,"num_objects":5487,"num_object_clones":0,"num_object_copies":10974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3759,"num_write_kb":15036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1685}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5516","reported_seq":5590,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:04.382106+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:04.382106+0000","last_peered":"2024-09-10T15:54:04.382106+0000","last_clean":"2024-09-10T15:54:04.382106+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:04.382106+0000","last_undegraded":"2024-09-10T15:54:04.382106+0000","last_fullsized":"2024-09-10T15:54:04.382106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5516,"log_dups_size":0,"ondisk_log_size":5516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22593536,"num_objects":5516,"num_object_clones":0,"num_object_copies":11032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5516,"num_write_kb":22064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5421","reported_seq":5542,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:02.363549+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:02.363549+0000","last_peered":"2024-09-10T15:54:02.363549+0000","last_clean":"2024-09-10T15:54:02.363549+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:02.363549+0000","last_undegraded":"2024-09-10T15:54:02.363549+0000","last_fullsized":"2024-09-10T15:54:02.363549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5421,"log_dups_size":0,"ondisk_log_size":5421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22175744,"num_objects":5414,"num_object_clones":0,"num_object_copies":10828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2492,"num_write_kb":9968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5508","reported_seq":5637,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.386565+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:02.386565+0000","last_peered":"2024-09-10T15:54:02.386565+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:02.386565+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5508,"log_dups_size":0,"ondisk_log_size":5508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540288,"num_objects":5503,"num_object_clones":0,"num_object_copies":11006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2226,"num_write_kb":8904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1635},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5396","reported_seq":5531,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.320349+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:02.320349+0000","last_peered":"2024-09-10T15:54:02.320349+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:02.320349+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5396,"log_dups_size":0,"ondisk_log_size":5396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22077440,"num_objects":5390,"num_object_clones":0,"num_object_copies":10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2046,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1556},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5545","reported_seq":5671,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:02.320147+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:02.320147+0000","last_peered":"2024-09-10T15:54:02.320147+0000","last_clean":"2024-09-10T15:54:02.320147+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:02.320147+0000","last_undegraded":"2024-09-10T15:54:02.320147+0000","last_fullsized":"2024-09-10T15:54:02.320147+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5545,"log_dups_size":0,"ondisk_log_size":5545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2304,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5686","reported_seq":8298,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:04.426981+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:04.426981+0000","last_peered":"2024-09-10T15:54:04.426981+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:04.426981+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5686,"log_dups_size":0,"ondisk_log_size":5686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23261184,"num_objects":5679,"num_object_clones":0,"num_object_copies":11358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2746,"num_objects_unfound":0,"num_objects_dirty":5679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3663,"num_write_kb":14652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1244,"num_bytes_recovered":5095424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2746},{"shards":"2","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5468","reported_seq":5586,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.426073+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:02.426073+0000","last_peered":"2024-09-10T15:54:02.426073+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:02.426073+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22364160,"num_objects":5460,"num_object_clones":0,"num_object_copies":10920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2332,"num_write_kb":9328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1597},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5560","reported_seq":5679,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:04.370717+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:04.370717+0000","last_peered":"2024-09-10T15:54:04.370717+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:04.370717+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5560,"log_dups_size":0,"ondisk_log_size":5560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22753280,"num_objects":5555,"num_object_clones":0,"num_object_copies":11110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3534,"num_write_kb":14136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1723}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5473","reported_seq":5594,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:04.379473+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:04.379473+0000","last_peered":"2024-09-10T15:54:04.379473+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:04.379473+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5473,"log_dups_size":0,"ondisk_log_size":5473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22392832,"num_objects":5467,"num_object_clones":0,"num_object_copies":10934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3342,"num_write_kb":13368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1638}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5538","reported_seq":5660,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.334207+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:02.334207+0000","last_peered":"2024-09-10T15:54:02.334207+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:02.334207+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5538,"log_dups_size":0,"ondisk_log_size":5538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22675456,"num_objects":5536,"num_object_clones":0,"num_object_copies":11072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2730,"num_write_kb":10920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1703},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5574","reported_seq":5731,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.398616+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:02.398616+0000","last_peered":"2024-09-10T15:54:02.398616+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:02.398616+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5574,"log_dups_size":0,"ondisk_log_size":5574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22798336,"num_objects":5566,"num_object_clones":0,"num_object_copies":11132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1992,"num_write_kb":7968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1649},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5473","reported_seq":5607,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.381703+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:02.381703+0000","last_peered":"2024-09-10T15:54:02.381703+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:02.381703+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5473,"log_dups_size":0,"ondisk_log_size":5473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22401024,"num_objects":5469,"num_object_clones":0,"num_object_copies":10938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2079,"num_write_kb":8316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1592},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5623","reported_seq":5756,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:02.381152+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:02.381152+0000","last_peered":"2024-09-10T15:54:02.381152+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:02.381152+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5623,"log_dups_size":0,"ondisk_log_size":5623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22983139,"num_objects":5612,"num_object_clones":0,"num_object_copies":11224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2442,"num_write_kb":9768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1609},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":362320355,"num_objects":88458,"num_object_clones":0,"num_object_copies":176916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2746,"num_objects_unfound":0,"num_objects_dirty":88458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50060,"num_write_kb":200240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1246,"num_bytes_recovered":5103616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":729223168,"data_stored":729132843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88539,"ondisk_log_size":88539,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626443,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":194696,"kb_used_data":82552,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94177144,"statfs":{"total":96636764160,"available":96437395456,"internally_reserved":0,"allocated":84533248,"data_stored":84198104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":114814638},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149727,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501796,"kb_used_data":325780,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93870044,"statfs":{"total":96636764160,"available":96122925056,"internally_reserved":0,"allocated":333598720,"data_stored":333343062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180219565},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,3,2,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468556,"kb_used_data":302268,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93903284,"statfs":{"total":96636764160,"available":96156962816,"internally_reserved":0,"allocated":309522432,"data_stored":309266774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":334163968,"data_stored":334160355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":85020672,"data_stored":84937573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310038528,"data_stored":310034915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:08.811 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 91 20 91774 91754 3.93798 2.375 0.0410268 0.0198313 2024-09-10T15:54:09.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 92 20 92410 92390 3.92216 2.48438 0.0179381 0.0199105 2024-09-10T15:54:10.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 93 20 93027 93007 3.9059 2.41016 0.0103011 0.0199921 2024-09-10T15:54:10.868 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:11.420 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:11.424 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:11.445 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-10T15:54:10.457286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374477302,"num_objects":91428,"num_object_clones":0,"num_object_copies":182856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2601,"num_objects_unfound":0,"num_objects_dirty":91428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53030,"num_write_kb":212114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1393,"num_bytes_recovered":5697555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91510,"ondisk_log_size":91510,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1172912,"kb_used_data":735100,"kb_used_omap":12,"kb_used_meta":437747,"kb_avail":281942608,"statfs":{"total":289910292480,"available":288709230592,"internally_reserved":0,"allocated":752742400,"data_stored":751895940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":448252933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,8,5,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28389376,"num_objects":6931,"num_object_clones":0,"num_object_copies":13862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-450,"num_objects_unfound":0,"num_objects_dirty":6931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6931,"num_write_kb":27724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":1843200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004044"},"pg_stats":[{"pgid":"2.f","version":"44'5740","reported_seq":5810,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.424857+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:07.424857+0000","last_peered":"2024-09-10T15:54:07.424857+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:07.424857+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5740,"log_dups_size":0,"ondisk_log_size":5740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23511040,"num_objects":5740,"num_object_clones":0,"num_object_copies":11480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5740,"num_write_kb":22960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1767},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5919","reported_seq":6021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:09.444064+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:09.444064+0000","last_peered":"2024-09-10T15:54:09.444064+0000","last_clean":"2024-09-10T15:54:09.444064+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:09.444064+0000","last_undegraded":"2024-09-10T15:54:09.444064+0000","last_fullsized":"2024-09-10T15:54:09.444064+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5919,"log_dups_size":0,"ondisk_log_size":5919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24219648,"num_objects":5913,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4255,"num_write_kb":17020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5695","reported_seq":5813,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:09.347844+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:09.347844+0000","last_peered":"2024-09-10T15:54:09.347844+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:09.347844+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5695,"log_dups_size":0,"ondisk_log_size":5695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23310336,"num_objects":5691,"num_object_clones":0,"num_object_copies":11382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3963,"num_write_kb":15852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1889}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5702","reported_seq":5775,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:09.444231+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:09.444231+0000","last_peered":"2024-09-10T15:54:09.444231+0000","last_clean":"2024-09-10T15:54:09.444231+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:09.444231+0000","last_undegraded":"2024-09-10T15:54:09.444231+0000","last_fullsized":"2024-09-10T15:54:09.444231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23355392,"num_objects":5702,"num_object_clones":0,"num_object_copies":11404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5702,"num_write_kb":22808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5588","reported_seq":5708,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:07.454791+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:07.454791+0000","last_peered":"2024-09-10T15:54:07.454791+0000","last_clean":"2024-09-10T15:54:07.454791+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:07.454791+0000","last_undegraded":"2024-09-10T15:54:07.454791+0000","last_fullsized":"2024-09-10T15:54:07.454791+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5588,"log_dups_size":0,"ondisk_log_size":5588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2659,"num_write_kb":10636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5702","reported_seq":5828,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.459397+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:07.459397+0000","last_peered":"2024-09-10T15:54:07.459397+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:07.459397+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23334912,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2420,"num_write_kb":9680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1829},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5576","reported_seq":5711,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.414573+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:07.414573+0000","last_peered":"2024-09-10T15:54:07.414573+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:07.414573+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5576,"log_dups_size":0,"ondisk_log_size":5576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22814720,"num_objects":5570,"num_object_clones":0,"num_object_copies":11140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2226,"num_write_kb":8904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1736},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5738","reported_seq":5864,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:07.422353+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:07.422353+0000","last_peered":"2024-09-10T15:54:07.422353+0000","last_clean":"2024-09-10T15:54:07.422353+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:07.422353+0000","last_undegraded":"2024-09-10T15:54:07.422353+0000","last_fullsized":"2024-09-10T15:54:07.422353+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23494656,"num_objects":5736,"num_object_clones":0,"num_object_copies":11472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2497,"num_write_kb":9988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5855","reported_seq":8756,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:09.434059+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:09.434059+0000","last_peered":"2024-09-10T15:54:09.434059+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:09.434059+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5855,"log_dups_size":0,"ondisk_log_size":5855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23953408,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2601,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3832,"num_write_kb":15328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1389,"num_bytes_recovered":5689344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2601},{"shards":"2","objects":3245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5643","reported_seq":5764,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.344792+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:07.344792+0000","last_peered":"2024-09-10T15:54:07.344792+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:07.344792+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23080960,"num_objects":5635,"num_object_clones":0,"num_object_copies":11270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2507,"num_write_kb":10028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1772},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5754","reported_seq":5873,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:09.414173+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:09.414173+0000","last_peered":"2024-09-10T15:54:09.414173+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:09.414173+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5754,"log_dups_size":0,"ondisk_log_size":5754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23547904,"num_objects":5749,"num_object_clones":0,"num_object_copies":11498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3728,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1917}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5649","reported_seq":5770,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:09.395655+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:09.395655+0000","last_peered":"2024-09-10T15:54:09.395655+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:09.395655+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5649,"log_dups_size":0,"ondisk_log_size":5649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3518,"num_write_kb":14072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1814}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5723","reported_seq":5845,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.444148+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:07.444148+0000","last_peered":"2024-09-10T15:54:07.444148+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:07.444148+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5723,"log_dups_size":0,"ondisk_log_size":5723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23433216,"num_objects":5721,"num_object_clones":0,"num_object_copies":11442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2915,"num_write_kb":11660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5771","reported_seq":5929,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.433050+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:07.433050+0000","last_peered":"2024-09-10T15:54:07.433050+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:07.433050+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5771,"log_dups_size":0,"ondisk_log_size":5771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23605248,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2189,"num_write_kb":8756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1846},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5628","reported_seq":5761,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.445714+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:07.445714+0000","last_peered":"2024-09-10T15:54:07.445714+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:07.445714+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5628,"log_dups_size":0,"ondisk_log_size":5628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23035904,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2234,"num_write_kb":8936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1747},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5824","reported_seq":5957,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.439956+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:07.439956+0000","last_peered":"2024-09-10T15:54:07.439956+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:07.439956+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5824,"log_dups_size":0,"ondisk_log_size":5824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23806435,"num_objects":5813,"num_object_clones":0,"num_object_copies":11626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2643,"num_write_kb":10572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1810},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374477283,"num_objects":91426,"num_object_clones":0,"num_object_copies":182852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2601,"num_objects_unfound":0,"num_objects_dirty":91426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53028,"num_write_kb":212112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1391,"num_bytes_recovered":5697536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":754167808,"data_stored":754077483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91507,"ondisk_log_size":91507,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626444,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":195920,"kb_used_data":92096,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94175920,"statfs":{"total":96636764160,"available":96436142080,"internally_reserved":0,"allocated":94306304,"data_stored":93971160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":106294958},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149728,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515024,"kb_used_data":334848,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93856816,"statfs":{"total":96636764160,"available":96109379584,"internally_reserved":0,"allocated":342884352,"data_stored":342628694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184479405},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461968,"kb_used_data":308156,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93909872,"statfs":{"total":96636764160,"available":96163708928,"internally_reserved":0,"allocated":315551744,"data_stored":315296086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157478570},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343257088,"data_stored":343253475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94752768,"data_stored":94669669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":316157952,"data_stored":316154339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:11.446 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:11.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 94 20 93572 93552 3.88699 2.12891 0.0156232 0.0200887 2024-09-10T15:54:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:11.996 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:12.019 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-10T15:54:10.457286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374477302,"num_objects":91428,"num_object_clones":0,"num_object_copies":182856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2601,"num_objects_unfound":0,"num_objects_dirty":91428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53030,"num_write_kb":212114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1393,"num_bytes_recovered":5697555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91510,"ondisk_log_size":91510,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1172912,"kb_used_data":735100,"kb_used_omap":12,"kb_used_meta":437747,"kb_avail":281942608,"statfs":{"total":289910292480,"available":288709230592,"internally_reserved":0,"allocated":752742400,"data_stored":751895940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":448252933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,8,5,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28389376,"num_objects":6931,"num_object_clones":0,"num_object_copies":13862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-450,"num_objects_unfound":0,"num_objects_dirty":6931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6931,"num_write_kb":27724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":1843200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004044"},"pg_stats":[{"pgid":"2.f","version":"44'5740","reported_seq":5810,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.424857+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:07.424857+0000","last_peered":"2024-09-10T15:54:07.424857+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:07.424857+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5740,"log_dups_size":0,"ondisk_log_size":5740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23511040,"num_objects":5740,"num_object_clones":0,"num_object_copies":11480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5740,"num_write_kb":22960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1767},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5919","reported_seq":6021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:09.444064+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:09.444064+0000","last_peered":"2024-09-10T15:54:09.444064+0000","last_clean":"2024-09-10T15:54:09.444064+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:09.444064+0000","last_undegraded":"2024-09-10T15:54:09.444064+0000","last_fullsized":"2024-09-10T15:54:09.444064+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5919,"log_dups_size":0,"ondisk_log_size":5919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24219648,"num_objects":5913,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4255,"num_write_kb":17020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5695","reported_seq":5813,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:09.347844+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:09.347844+0000","last_peered":"2024-09-10T15:54:09.347844+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:09.347844+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5695,"log_dups_size":0,"ondisk_log_size":5695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23310336,"num_objects":5691,"num_object_clones":0,"num_object_copies":11382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3963,"num_write_kb":15852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1889}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5702","reported_seq":5775,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:09.444231+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:09.444231+0000","last_peered":"2024-09-10T15:54:09.444231+0000","last_clean":"2024-09-10T15:54:09.444231+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:09.444231+0000","last_undegraded":"2024-09-10T15:54:09.444231+0000","last_fullsized":"2024-09-10T15:54:09.444231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23355392,"num_objects":5702,"num_object_clones":0,"num_object_copies":11404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5702,"num_write_kb":22808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5588","reported_seq":5708,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:07.454791+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:07.454791+0000","last_peered":"2024-09-10T15:54:07.454791+0000","last_clean":"2024-09-10T15:54:07.454791+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:07.454791+0000","last_undegraded":"2024-09-10T15:54:07.454791+0000","last_fullsized":"2024-09-10T15:54:07.454791+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5588,"log_dups_size":0,"ondisk_log_size":5588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2659,"num_write_kb":10636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5702","reported_seq":5828,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.459397+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:07.459397+0000","last_peered":"2024-09-10T15:54:07.459397+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:07.459397+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23334912,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2420,"num_write_kb":9680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1829},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5576","reported_seq":5711,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.414573+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:07.414573+0000","last_peered":"2024-09-10T15:54:07.414573+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:07.414573+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5576,"log_dups_size":0,"ondisk_log_size":5576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22814720,"num_objects":5570,"num_object_clones":0,"num_object_copies":11140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2226,"num_write_kb":8904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1736},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5738","reported_seq":5864,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:07.422353+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:07.422353+0000","last_peered":"2024-09-10T15:54:07.422353+0000","last_clean":"2024-09-10T15:54:07.422353+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:07.422353+0000","last_undegraded":"2024-09-10T15:54:07.422353+0000","last_fullsized":"2024-09-10T15:54:07.422353+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23494656,"num_objects":5736,"num_object_clones":0,"num_object_copies":11472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2497,"num_write_kb":9988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5855","reported_seq":8756,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:09.434059+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:09.434059+0000","last_peered":"2024-09-10T15:54:09.434059+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:09.434059+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5855,"log_dups_size":0,"ondisk_log_size":5855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23953408,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2601,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3832,"num_write_kb":15328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1389,"num_bytes_recovered":5689344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2601},{"shards":"2","objects":3245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5643","reported_seq":5764,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.344792+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:07.344792+0000","last_peered":"2024-09-10T15:54:07.344792+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:07.344792+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23080960,"num_objects":5635,"num_object_clones":0,"num_object_copies":11270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2507,"num_write_kb":10028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1772},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5754","reported_seq":5873,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:09.414173+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:09.414173+0000","last_peered":"2024-09-10T15:54:09.414173+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:09.414173+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5754,"log_dups_size":0,"ondisk_log_size":5754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23547904,"num_objects":5749,"num_object_clones":0,"num_object_copies":11498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3728,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1917}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5649","reported_seq":5770,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:09.395655+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:09.395655+0000","last_peered":"2024-09-10T15:54:09.395655+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:09.395655+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5649,"log_dups_size":0,"ondisk_log_size":5649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3518,"num_write_kb":14072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1814}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5723","reported_seq":5845,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.444148+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:07.444148+0000","last_peered":"2024-09-10T15:54:07.444148+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:07.444148+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5723,"log_dups_size":0,"ondisk_log_size":5723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23433216,"num_objects":5721,"num_object_clones":0,"num_object_copies":11442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2915,"num_write_kb":11660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5771","reported_seq":5929,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.433050+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:07.433050+0000","last_peered":"2024-09-10T15:54:07.433050+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:07.433050+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5771,"log_dups_size":0,"ondisk_log_size":5771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23605248,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2189,"num_write_kb":8756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1846},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5628","reported_seq":5761,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.445714+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:07.445714+0000","last_peered":"2024-09-10T15:54:07.445714+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:07.445714+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5628,"log_dups_size":0,"ondisk_log_size":5628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23035904,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2234,"num_write_kb":8936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1747},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5824","reported_seq":5957,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.439956+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:07.439956+0000","last_peered":"2024-09-10T15:54:07.439956+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:07.439956+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5824,"log_dups_size":0,"ondisk_log_size":5824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23806435,"num_objects":5813,"num_object_clones":0,"num_object_copies":11626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2643,"num_write_kb":10572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1810},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374477283,"num_objects":91426,"num_object_clones":0,"num_object_copies":182852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2601,"num_objects_unfound":0,"num_objects_dirty":91426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53028,"num_write_kb":212112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1391,"num_bytes_recovered":5697536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":754167808,"data_stored":754077483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91507,"ondisk_log_size":91507,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626444,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":195920,"kb_used_data":92096,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94175920,"statfs":{"total":96636764160,"available":96436142080,"internally_reserved":0,"allocated":94306304,"data_stored":93971160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":106294958},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149728,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515024,"kb_used_data":334848,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93856816,"statfs":{"total":96636764160,"available":96109379584,"internally_reserved":0,"allocated":342884352,"data_stored":342628694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184479405},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461968,"kb_used_data":308156,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93909872,"statfs":{"total":96636764160,"available":96163708928,"internally_reserved":0,"allocated":315551744,"data_stored":315296086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157478570},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343257088,"data_stored":343253475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94752768,"data_stored":94669669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":316157952,"data_stored":316154339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:12.020 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:54:12.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'5740", 'reported_seq': 5810, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:07.424857+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:54:07.424857+0000', 'last_peered': '2024-09-10T15:54:07.424857+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:54:07.424857+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 5740, 'log_dups_size': 0, 'ondisk_log_size': 5740, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23511040, 'num_objects': 5740, 'num_object_clones': 0, 'num_object_copies': 11480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5740, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5740, 'num_write_kb': 22960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1767}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:12.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:54:12.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'5695", 'reported_seq': 5813, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:09.347844+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:54:09.347844+0000', 'last_peered': '2024-09-10T15:54:09.347844+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:54:09.347844+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 5695, 'log_dups_size': 0, 'ondisk_log_size': 5695, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23310336, 'num_objects': 5691, 'num_object_clones': 0, 'num_object_copies': 11382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5691, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3963, 'num_write_kb': 15852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 1889}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:12.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:54:12.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'5702", 'reported_seq': 5828, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:07.459397+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:54:07.459397+0000', 'last_peered': '2024-09-10T15:54:07.459397+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:54:07.459397+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 5702, 'log_dups_size': 0, 'ondisk_log_size': 5702, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23334912, 'num_objects': 5697, 'num_object_clones': 0, 'num_object_copies': 11394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2420, 'num_write_kb': 9680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1829}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:12.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:54:12.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'5576", 'reported_seq': 5711, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:07.414573+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:54:07.414573+0000', 'last_peered': '2024-09-10T15:54:07.414573+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:54:07.414573+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 5576, 'log_dups_size': 0, 'ondisk_log_size': 5576, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22814720, 'num_objects': 5570, 'num_object_clones': 0, 'num_object_copies': 11140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2226, 'num_write_kb': 8904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1736}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:12.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:54:12.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'5855", 'reported_seq': 8756, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:09.434059+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:54:09.434059+0000', 'last_peered': '2024-09-10T15:54:09.434059+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:54:09.434059+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 5855, 'log_dups_size': 0, 'ondisk_log_size': 5855, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23953408, 'num_objects': 5848, 'num_object_clones': 0, 'num_object_copies': 11696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2601, 'num_objects_unfound': 0, 'num_objects_dirty': 5848, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3832, 'num_write_kb': 15328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1389, 'num_bytes_recovered': 5689344, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2601}, {'shards': '2', 'objects': 3245}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:12.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:54:12.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'5643", 'reported_seq': 5764, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:07.344792+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:54:07.344792+0000', 'last_peered': '2024-09-10T15:54:07.344792+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:54:07.344792+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 5643, 'log_dups_size': 0, 'ondisk_log_size': 5643, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23080960, 'num_objects': 5635, 'num_object_clones': 0, 'num_object_copies': 11270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5635, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2507, 'num_write_kb': 10028, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1772}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:12.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:54:12.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'5754", 'reported_seq': 5873, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:09.414173+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:54:09.414173+0000', 'last_peered': '2024-09-10T15:54:09.414173+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:54:09.414173+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 5754, 'log_dups_size': 0, 'ondisk_log_size': 5754, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23547904, 'num_objects': 5749, 'num_object_clones': 0, 'num_object_copies': 11498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5749, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3728, 'num_write_kb': 14912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 1917}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:12.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:54:12.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'5649", 'reported_seq': 5770, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:09.395655+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:54:09.395655+0000', 'last_peered': '2024-09-10T15:54:09.395655+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:54:09.395655+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 5649, 'log_dups_size': 0, 'ondisk_log_size': 5649, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23113728, 'num_objects': 5643, 'num_object_clones': 0, 'num_object_copies': 11286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5643, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3518, 'num_write_kb': 14072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 1814}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:12.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:54:12.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'5723", 'reported_seq': 5845, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:07.444148+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:54:07.444148+0000', 'last_peered': '2024-09-10T15:54:07.444148+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:54:07.444148+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 5723, 'log_dups_size': 0, 'ondisk_log_size': 5723, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23433216, 'num_objects': 5721, 'num_object_clones': 0, 'num_object_copies': 11442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2915, 'num_write_kb': 11660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1888}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:12.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:54:12.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'5771", 'reported_seq': 5929, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:07.433050+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:54:07.433050+0000', 'last_peered': '2024-09-10T15:54:07.433050+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:54:07.433050+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 5771, 'log_dups_size': 0, 'ondisk_log_size': 5771, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23605248, 'num_objects': 5763, 'num_object_clones': 0, 'num_object_copies': 11526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5763, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2189, 'num_write_kb': 8756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1846}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:12.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:54:12.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'5628", 'reported_seq': 5761, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:07.445714+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:54:07.445714+0000', 'last_peered': '2024-09-10T15:54:07.445714+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:54:07.445714+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 5628, 'log_dups_size': 0, 'ondisk_log_size': 5628, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23035904, 'num_objects': 5624, 'num_object_clones': 0, 'num_object_copies': 11248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2234, 'num_write_kb': 8936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1747}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:12.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:54:12.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'5824", 'reported_seq': 5957, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:07.439956+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:54:07.439956+0000', 'last_peered': '2024-09-10T15:54:07.439956+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:54:07.439956+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 5824, 'log_dups_size': 0, 'ondisk_log_size': 5824, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23806435, 'num_objects': 5813, 'num_object_clones': 0, 'num_object_copies': 11626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5813, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2643, 'num_write_kb': 10572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1810}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:12.023 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:54:12.729 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:12.730 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/182856 objects degraded (23.326%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":178,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91428,"data_bytes":374477302,"bytes_used":1201061888,"bytes_avail":288709230592,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":182856,"degraded_ratio":0.23326005162532265,"misplaced_objects":2601,"misplaced_total":182856,"misplaced_ratio":0.014224307651922825,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":153548,"recovering_keys_per_sec":0,"num_objects_recovered":450,"num_bytes_recovered":1843200,"num_keys_recovered":0,"write_bytes_sec":2364984,"read_op_per_sec":0,"write_op_per_sec":577},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:53:42.448843+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:54:12.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:54:12.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:12.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 95 20 94127 94107 3.8689 2.16797 0.0703242 0.0201863 2024-09-10T15:54:13.341 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:13.342 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:13.365 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-10T15:54:12.457711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374477302,"num_objects":91428,"num_object_clones":0,"num_object_copies":182856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2601,"num_objects_unfound":0,"num_objects_dirty":91428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53030,"num_write_kb":212114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1393,"num_bytes_recovered":5697555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91510,"ondisk_log_size":91510,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1172912,"kb_used_data":735100,"kb_used_omap":12,"kb_used_meta":437747,"kb_avail":281942608,"statfs":{"total":289910292480,"available":288709230592,"internally_reserved":0,"allocated":752742400,"data_stored":751895940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":448252933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,8,5,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23740416,"num_objects":5796,"num_object_clones":0,"num_object_copies":11592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-300,"num_objects_unfound":0,"num_objects_dirty":5796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5796,"num_write_kb":23184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1228800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003896"},"pg_stats":[{"pgid":"2.f","version":"44'5740","reported_seq":5810,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.424857+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:07.424857+0000","last_peered":"2024-09-10T15:54:07.424857+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:07.424857+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5740,"log_dups_size":0,"ondisk_log_size":5740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23511040,"num_objects":5740,"num_object_clones":0,"num_object_copies":11480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5740,"num_write_kb":22960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1767},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5919","reported_seq":6021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:09.444064+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:09.444064+0000","last_peered":"2024-09-10T15:54:09.444064+0000","last_clean":"2024-09-10T15:54:09.444064+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:09.444064+0000","last_undegraded":"2024-09-10T15:54:09.444064+0000","last_fullsized":"2024-09-10T15:54:09.444064+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":5919,"log_dups_size":0,"ondisk_log_size":5919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24219648,"num_objects":5913,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4255,"num_write_kb":17020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5695","reported_seq":5813,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:09.347844+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:09.347844+0000","last_peered":"2024-09-10T15:54:09.347844+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:09.347844+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5695,"log_dups_size":0,"ondisk_log_size":5695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23310336,"num_objects":5691,"num_object_clones":0,"num_object_copies":11382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3963,"num_write_kb":15852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":1889}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5702","reported_seq":5775,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:09.444231+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:09.444231+0000","last_peered":"2024-09-10T15:54:09.444231+0000","last_clean":"2024-09-10T15:54:09.444231+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:09.444231+0000","last_undegraded":"2024-09-10T15:54:09.444231+0000","last_fullsized":"2024-09-10T15:54:09.444231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23355392,"num_objects":5702,"num_object_clones":0,"num_object_copies":11404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5702,"num_write_kb":22808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5588","reported_seq":5708,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:07.454791+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:07.454791+0000","last_peered":"2024-09-10T15:54:07.454791+0000","last_clean":"2024-09-10T15:54:07.454791+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:07.454791+0000","last_undegraded":"2024-09-10T15:54:07.454791+0000","last_fullsized":"2024-09-10T15:54:07.454791+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5588,"log_dups_size":0,"ondisk_log_size":5588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2659,"num_write_kb":10636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5702","reported_seq":5828,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.459397+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:07.459397+0000","last_peered":"2024-09-10T15:54:07.459397+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:07.459397+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23334912,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2420,"num_write_kb":9680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1829},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5576","reported_seq":5711,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.414573+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:07.414573+0000","last_peered":"2024-09-10T15:54:07.414573+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:07.414573+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5576,"log_dups_size":0,"ondisk_log_size":5576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22814720,"num_objects":5570,"num_object_clones":0,"num_object_copies":11140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2226,"num_write_kb":8904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1736},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5738","reported_seq":5864,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:07.422353+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:07.422353+0000","last_peered":"2024-09-10T15:54:07.422353+0000","last_clean":"2024-09-10T15:54:07.422353+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:07.422353+0000","last_undegraded":"2024-09-10T15:54:07.422353+0000","last_fullsized":"2024-09-10T15:54:07.422353+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23494656,"num_objects":5736,"num_object_clones":0,"num_object_copies":11472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2497,"num_write_kb":9988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5855","reported_seq":8756,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:09.434059+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:09.434059+0000","last_peered":"2024-09-10T15:54:09.434059+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:09.434059+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":5855,"log_dups_size":0,"ondisk_log_size":5855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23953408,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2601,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3832,"num_write_kb":15328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1389,"num_bytes_recovered":5689344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2601},{"shards":"2","objects":3245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5643","reported_seq":5764,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.344792+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:07.344792+0000","last_peered":"2024-09-10T15:54:07.344792+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:07.344792+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23080960,"num_objects":5635,"num_object_clones":0,"num_object_copies":11270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2507,"num_write_kb":10028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1772},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.891807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.891807+0000","last_peered":"2024-09-10T15:53:19.891807+0000","last_clean":"2024-09-10T15:53:19.891807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.891807+0000","last_undegraded":"2024-09-10T15:53:19.891807+0000","last_fullsized":"2024-09-10T15:53:19.891807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5754","reported_seq":5873,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:09.414173+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:09.414173+0000","last_peered":"2024-09-10T15:54:09.414173+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:09.414173+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5754,"log_dups_size":0,"ondisk_log_size":5754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23547904,"num_objects":5749,"num_object_clones":0,"num_object_copies":11498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3728,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":1917}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5649","reported_seq":5770,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:09.395655+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:09.395655+0000","last_peered":"2024-09-10T15:54:09.395655+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:09.395655+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5649,"log_dups_size":0,"ondisk_log_size":5649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3518,"num_write_kb":14072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":1814}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5723","reported_seq":5845,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.444148+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:07.444148+0000","last_peered":"2024-09-10T15:54:07.444148+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:07.444148+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5723,"log_dups_size":0,"ondisk_log_size":5723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23433216,"num_objects":5721,"num_object_clones":0,"num_object_copies":11442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2915,"num_write_kb":11660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5771","reported_seq":5929,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.433050+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:07.433050+0000","last_peered":"2024-09-10T15:54:07.433050+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:07.433050+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5771,"log_dups_size":0,"ondisk_log_size":5771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23605248,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2189,"num_write_kb":8756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1846},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5628","reported_seq":5761,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.445714+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:07.445714+0000","last_peered":"2024-09-10T15:54:07.445714+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:07.445714+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5628,"log_dups_size":0,"ondisk_log_size":5628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23035904,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2234,"num_write_kb":8936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1747},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5824","reported_seq":5957,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:07.439956+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:07.439956+0000","last_peered":"2024-09-10T15:54:07.439956+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:07.439956+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":5824,"log_dups_size":0,"ondisk_log_size":5824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23806435,"num_objects":5813,"num_object_clones":0,"num_object_copies":11626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2643,"num_write_kb":10572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1810},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374477283,"num_objects":91426,"num_object_clones":0,"num_object_copies":182852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2601,"num_objects_unfound":0,"num_objects_dirty":91426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53028,"num_write_kb":212112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1391,"num_bytes_recovered":5697536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":754167808,"data_stored":754077483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91507,"ondisk_log_size":91507,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626444,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":195920,"kb_used_data":92096,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94175920,"statfs":{"total":96636764160,"available":96436142080,"internally_reserved":0,"allocated":94306304,"data_stored":93971160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":106294958},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149728,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515024,"kb_used_data":334848,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93856816,"statfs":{"total":96636764160,"available":96109379584,"internally_reserved":0,"allocated":342884352,"data_stored":342628694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184479405},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461968,"kb_used_data":308156,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93909872,"statfs":{"total":96636764160,"available":96163708928,"internally_reserved":0,"allocated":315551744,"data_stored":315296086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157478570},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343257088,"data_stored":343253475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94752768,"data_stored":94669669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":316157952,"data_stored":316154339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:13.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 96 20 94698 94678 3.85183 2.23047 0.0394356 0.0202758 2024-09-10T15:54:14.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 97 20 95299 95279 3.83632 2.34766 0.0146893 0.0203564 2024-09-10T15:54:15.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 98 20 95899 95879 3.82109 2.34375 0.0336157 0.0204387 2024-09-10T15:54:16.368 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:16.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 99 20 96484 96464 3.80558 2.28516 0.0190563 0.02052 2024-09-10T15:54:16.961 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:16.966 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:16.995 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-10T15:54:16.458585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386613750,"num_objects":94391,"num_object_clones":0,"num_object_copies":188782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":94391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55993,"num_write_kb":223966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":6287379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94473,"ondisk_log_size":94473,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1235908,"kb_used_data":759252,"kb_used_omap":12,"kb_used_meta":476595,"kb_avail":281879612,"statfs":{"total":289910292480,"available":288644722688,"internally_reserved":0,"allocated":777474048,"data_stored":776627588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":488033285},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,10,10,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24293376,"num_objects":5931,"num_object_clones":0,"num_object_copies":11862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-289,"num_objects_unfound":0,"num_objects_dirty":5931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5931,"num_write_kb":23724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002821"},"pg_stats":[{"pgid":"2.f","version":"44'5920","reported_seq":5991,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.441022+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:12.441022+0000","last_peered":"2024-09-10T15:54:12.441022+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:12.441022+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5920,"log_dups_size":0,"ondisk_log_size":5920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24248320,"num_objects":5920,"num_object_clones":0,"num_object_copies":11840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5920,"num_write_kb":23680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1947},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6119","reported_seq":6221,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:14.389570+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:14.389570+0000","last_peered":"2024-09-10T15:54:14.389570+0000","last_clean":"2024-09-10T15:54:14.389570+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:14.389570+0000","last_undegraded":"2024-09-10T15:54:14.389570+0000","last_fullsized":"2024-09-10T15:54:14.389570+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6119,"log_dups_size":0,"ondisk_log_size":6119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25038848,"num_objects":6113,"num_object_clones":0,"num_object_copies":12226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4455,"num_write_kb":17820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5876","reported_seq":5992,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:14.432256+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:14.432256+0000","last_peered":"2024-09-10T15:54:14.432256+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:14.432256+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5876,"log_dups_size":0,"ondisk_log_size":5876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24051712,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4144,"num_write_kb":16576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5882","reported_seq":5956,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:14.422900+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:14.422900+0000","last_peered":"2024-09-10T15:54:14.422900+0000","last_clean":"2024-09-10T15:54:14.422900+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:14.422900+0000","last_undegraded":"2024-09-10T15:54:14.422900+0000","last_fullsized":"2024-09-10T15:54:14.422900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5882,"log_dups_size":0,"ondisk_log_size":5882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24092672,"num_objects":5882,"num_object_clones":0,"num_object_copies":11764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5882,"num_write_kb":23528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5782","reported_seq":5903,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:12.467040+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:12.467040+0000","last_peered":"2024-09-10T15:54:12.467040+0000","last_clean":"2024-09-10T15:54:12.467040+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:12.467040+0000","last_undegraded":"2024-09-10T15:54:12.467040+0000","last_fullsized":"2024-09-10T15:54:12.467040+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5782,"log_dups_size":0,"ondisk_log_size":5782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23654400,"num_objects":5775,"num_object_clones":0,"num_object_copies":11550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2853,"num_write_kb":11412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5908","reported_seq":6037,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.473749+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:12.473749+0000","last_peered":"2024-09-10T15:54:12.473749+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:12.473749+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5908,"log_dups_size":0,"ondisk_log_size":5908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24178688,"num_objects":5903,"num_object_clones":0,"num_object_copies":11806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2626,"num_write_kb":10504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2035},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5766","reported_seq":5901,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.409618+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:12.409618+0000","last_peered":"2024-09-10T15:54:12.409618+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:12.409618+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23592960,"num_objects":5760,"num_object_clones":0,"num_object_copies":11520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2416,"num_write_kb":9664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1926},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5919","reported_seq":6043,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:12.409630+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:12.409630+0000","last_peered":"2024-09-10T15:54:12.409630+0000","last_clean":"2024-09-10T15:54:12.409630+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:12.409630+0000","last_undegraded":"2024-09-10T15:54:12.409630+0000","last_fullsized":"2024-09-10T15:54:12.409630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5919,"log_dups_size":0,"ondisk_log_size":5919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24236032,"num_objects":5917,"num_object_clones":0,"num_object_copies":11834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2678,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6038","reported_seq":9228,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:14.453227+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:14.453227+0000","last_peered":"2024-09-10T15:54:14.453227+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:14.453227+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6038,"log_dups_size":0,"ondisk_log_size":6038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24702976,"num_objects":6031,"num_object_clones":0,"num_object_copies":12062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":6031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4015,"num_write_kb":16060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":6279168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2457},{"shards":"2","objects":3572}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5810","reported_seq":5932,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.475801+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:12.475801+0000","last_peered":"2024-09-10T15:54:12.475801+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:12.475801+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5810,"log_dups_size":0,"ondisk_log_size":5810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23764992,"num_objects":5802,"num_object_clones":0,"num_object_copies":11604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2674,"num_write_kb":10696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1939},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5940","reported_seq":6059,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:14.414678+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:14.414678+0000","last_peered":"2024-09-10T15:54:14.414678+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:14.414678+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5940,"log_dups_size":0,"ondisk_log_size":5940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24309760,"num_objects":5935,"num_object_clones":0,"num_object_copies":11870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3914,"num_write_kb":15656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2103}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5842","reported_seq":5963,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:14.407551+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:14.407551+0000","last_peered":"2024-09-10T15:54:14.407551+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:14.407551+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5842,"log_dups_size":0,"ondisk_log_size":5842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23904256,"num_objects":5836,"num_object_clones":0,"num_object_copies":11672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3711,"num_write_kb":14844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2007}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5907","reported_seq":6029,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.401501+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:12.401501+0000","last_peered":"2024-09-10T15:54:12.401501+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:12.401501+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5907,"log_dups_size":0,"ondisk_log_size":5907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24186880,"num_objects":5905,"num_object_clones":0,"num_object_copies":11810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3099,"num_write_kb":12396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2072},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5929","reported_seq":6087,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.487733+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:12.487733+0000","last_peered":"2024-09-10T15:54:12.487733+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:12.487733+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5929,"log_dups_size":0,"ondisk_log_size":5929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24252416,"num_objects":5921,"num_object_clones":0,"num_object_copies":11842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2347,"num_write_kb":9388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2004},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5813","reported_seq":5947,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.430082+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:12.430082+0000","last_peered":"2024-09-10T15:54:12.430082+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:12.430082+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5813,"log_dups_size":0,"ondisk_log_size":5813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23793664,"num_objects":5809,"num_object_clones":0,"num_object_copies":11618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2419,"num_write_kb":9676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1932},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6019","reported_seq":6152,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.417083+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:12.417083+0000","last_peered":"2024-09-10T15:54:12.417083+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:12.417083+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6019,"log_dups_size":0,"ondisk_log_size":6019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24605155,"num_objects":6008,"num_object_clones":0,"num_object_copies":12016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2838,"num_write_kb":11352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2005},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386613731,"num_objects":94389,"num_object_clones":0,"num_object_copies":188778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":94389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55991,"num_write_kb":223964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":6287360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":778997760,"data_stored":778907435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94470,"ondisk_log_size":94470,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626445,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231492,"kb_used_data":101300,"kb_used_omap":4,"kb_used_meta":130171,"kb_avail":94140348,"statfs":{"total":96636764160,"available":96399716352,"internally_reserved":0,"allocated":103731200,"data_stored":103396056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":133295790},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149729,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528100,"kb_used_data":343764,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93843740,"statfs":{"total":96636764160,"available":96095989760,"internally_reserved":0,"allocated":352014336,"data_stored":351758678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188739245},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,4,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476316,"kb_used_data":314188,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93895524,"statfs":{"total":96636764160,"available":96149016576,"internally_reserved":0,"allocated":321728512,"data_stored":321472854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":352362496,"data_stored":352358883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":104378368,"data_stored":104295269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322256896,"data_stored":322253283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:16.997 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:17.549 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:17.549 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:17.580 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-10T15:54:16.458585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386613750,"num_objects":94391,"num_object_clones":0,"num_object_copies":188782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":94391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55993,"num_write_kb":223966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":6287379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94473,"ondisk_log_size":94473,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1235908,"kb_used_data":759252,"kb_used_omap":12,"kb_used_meta":476595,"kb_avail":281879612,"statfs":{"total":289910292480,"available":288644722688,"internally_reserved":0,"allocated":777474048,"data_stored":776627588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":488033285},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,10,10,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24293376,"num_objects":5931,"num_object_clones":0,"num_object_copies":11862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-289,"num_objects_unfound":0,"num_objects_dirty":5931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5931,"num_write_kb":23724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002821"},"pg_stats":[{"pgid":"2.f","version":"44'5920","reported_seq":5991,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.441022+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:12.441022+0000","last_peered":"2024-09-10T15:54:12.441022+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:12.441022+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5920,"log_dups_size":0,"ondisk_log_size":5920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24248320,"num_objects":5920,"num_object_clones":0,"num_object_copies":11840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5920,"num_write_kb":23680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1947},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6119","reported_seq":6221,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:14.389570+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:14.389570+0000","last_peered":"2024-09-10T15:54:14.389570+0000","last_clean":"2024-09-10T15:54:14.389570+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:14.389570+0000","last_undegraded":"2024-09-10T15:54:14.389570+0000","last_fullsized":"2024-09-10T15:54:14.389570+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6119,"log_dups_size":0,"ondisk_log_size":6119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25038848,"num_objects":6113,"num_object_clones":0,"num_object_copies":12226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4455,"num_write_kb":17820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5876","reported_seq":5992,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:14.432256+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:14.432256+0000","last_peered":"2024-09-10T15:54:14.432256+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:14.432256+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5876,"log_dups_size":0,"ondisk_log_size":5876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24051712,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4144,"num_write_kb":16576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5882","reported_seq":5956,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:14.422900+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:14.422900+0000","last_peered":"2024-09-10T15:54:14.422900+0000","last_clean":"2024-09-10T15:54:14.422900+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:14.422900+0000","last_undegraded":"2024-09-10T15:54:14.422900+0000","last_fullsized":"2024-09-10T15:54:14.422900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5882,"log_dups_size":0,"ondisk_log_size":5882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24092672,"num_objects":5882,"num_object_clones":0,"num_object_copies":11764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5882,"num_write_kb":23528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5782","reported_seq":5903,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:12.467040+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:12.467040+0000","last_peered":"2024-09-10T15:54:12.467040+0000","last_clean":"2024-09-10T15:54:12.467040+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:12.467040+0000","last_undegraded":"2024-09-10T15:54:12.467040+0000","last_fullsized":"2024-09-10T15:54:12.467040+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5782,"log_dups_size":0,"ondisk_log_size":5782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23654400,"num_objects":5775,"num_object_clones":0,"num_object_copies":11550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2853,"num_write_kb":11412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5908","reported_seq":6037,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.473749+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:12.473749+0000","last_peered":"2024-09-10T15:54:12.473749+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:12.473749+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":5908,"log_dups_size":0,"ondisk_log_size":5908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24178688,"num_objects":5903,"num_object_clones":0,"num_object_copies":11806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2626,"num_write_kb":10504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2035},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5766","reported_seq":5901,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.409618+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:12.409618+0000","last_peered":"2024-09-10T15:54:12.409618+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:12.409618+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23592960,"num_objects":5760,"num_object_clones":0,"num_object_copies":11520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2416,"num_write_kb":9664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1926},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5919","reported_seq":6043,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:12.409630+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:12.409630+0000","last_peered":"2024-09-10T15:54:12.409630+0000","last_clean":"2024-09-10T15:54:12.409630+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:12.409630+0000","last_undegraded":"2024-09-10T15:54:12.409630+0000","last_fullsized":"2024-09-10T15:54:12.409630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":5919,"log_dups_size":0,"ondisk_log_size":5919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24236032,"num_objects":5917,"num_object_clones":0,"num_object_copies":11834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2678,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6038","reported_seq":9228,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:14.453227+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:14.453227+0000","last_peered":"2024-09-10T15:54:14.453227+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:14.453227+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6038,"log_dups_size":0,"ondisk_log_size":6038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24702976,"num_objects":6031,"num_object_clones":0,"num_object_copies":12062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":6031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4015,"num_write_kb":16060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":6279168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2457},{"shards":"2","objects":3572}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5810","reported_seq":5932,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.475801+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:12.475801+0000","last_peered":"2024-09-10T15:54:12.475801+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:12.475801+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5810,"log_dups_size":0,"ondisk_log_size":5810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23764992,"num_objects":5802,"num_object_clones":0,"num_object_copies":11604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2674,"num_write_kb":10696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1939},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5940","reported_seq":6059,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:14.414678+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:14.414678+0000","last_peered":"2024-09-10T15:54:14.414678+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:14.414678+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5940,"log_dups_size":0,"ondisk_log_size":5940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24309760,"num_objects":5935,"num_object_clones":0,"num_object_copies":11870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3914,"num_write_kb":15656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2103}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5842","reported_seq":5963,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:14.407551+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:14.407551+0000","last_peered":"2024-09-10T15:54:14.407551+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:14.407551+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5842,"log_dups_size":0,"ondisk_log_size":5842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23904256,"num_objects":5836,"num_object_clones":0,"num_object_copies":11672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3711,"num_write_kb":14844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2007}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5907","reported_seq":6029,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.401501+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:12.401501+0000","last_peered":"2024-09-10T15:54:12.401501+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:12.401501+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":5907,"log_dups_size":0,"ondisk_log_size":5907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24186880,"num_objects":5905,"num_object_clones":0,"num_object_copies":11810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3099,"num_write_kb":12396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2072},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5929","reported_seq":6087,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.487733+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:12.487733+0000","last_peered":"2024-09-10T15:54:12.487733+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:12.487733+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":5929,"log_dups_size":0,"ondisk_log_size":5929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24252416,"num_objects":5921,"num_object_clones":0,"num_object_copies":11842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2347,"num_write_kb":9388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2004},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5813","reported_seq":5947,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.430082+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:12.430082+0000","last_peered":"2024-09-10T15:54:12.430082+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:12.430082+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":5813,"log_dups_size":0,"ondisk_log_size":5813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23793664,"num_objects":5809,"num_object_clones":0,"num_object_copies":11618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2419,"num_write_kb":9676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1932},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6019","reported_seq":6152,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:12.417083+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:12.417083+0000","last_peered":"2024-09-10T15:54:12.417083+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:12.417083+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6019,"log_dups_size":0,"ondisk_log_size":6019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24605155,"num_objects":6008,"num_object_clones":0,"num_object_copies":12016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2838,"num_write_kb":11352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2005},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386613731,"num_objects":94389,"num_object_clones":0,"num_object_copies":188778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":94389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55991,"num_write_kb":223964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":6287360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":778997760,"data_stored":778907435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94470,"ondisk_log_size":94470,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626445,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231492,"kb_used_data":101300,"kb_used_omap":4,"kb_used_meta":130171,"kb_avail":94140348,"statfs":{"total":96636764160,"available":96399716352,"internally_reserved":0,"allocated":103731200,"data_stored":103396056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":133295790},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149729,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528100,"kb_used_data":343764,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93843740,"statfs":{"total":96636764160,"available":96095989760,"internally_reserved":0,"allocated":352014336,"data_stored":351758678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188739245},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,4,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476316,"kb_used_data":314188,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93895524,"statfs":{"total":96636764160,"available":96149016576,"internally_reserved":0,"allocated":321728512,"data_stored":321472854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":352362496,"data_stored":352358883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":104378368,"data_stored":104295269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322256896,"data_stored":322253283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:17.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:54:17.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'5920", 'reported_seq': 5991, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:12.441022+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:54:12.441022+0000', 'last_peered': '2024-09-10T15:54:12.441022+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:54:12.441022+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 5920, 'log_dups_size': 0, 'ondisk_log_size': 5920, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24248320, 'num_objects': 5920, 'num_object_clones': 0, 'num_object_copies': 11840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5920, 'num_write_kb': 23680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1947}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:17.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:54:17.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'5876", 'reported_seq': 5992, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:14.432256+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:54:14.432256+0000', 'last_peered': '2024-09-10T15:54:14.432256+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:54:14.432256+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 5876, 'log_dups_size': 0, 'ondisk_log_size': 5876, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24051712, 'num_objects': 5872, 'num_object_clones': 0, 'num_object_copies': 11744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5872, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4144, 'num_write_kb': 16576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 2070}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:17.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:54:17.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'5908", 'reported_seq': 6037, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:12.473749+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:54:12.473749+0000', 'last_peered': '2024-09-10T15:54:12.473749+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:54:12.473749+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 5908, 'log_dups_size': 0, 'ondisk_log_size': 5908, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24178688, 'num_objects': 5903, 'num_object_clones': 0, 'num_object_copies': 11806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2626, 'num_write_kb': 10504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2035}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:17.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:54:17.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'5766", 'reported_seq': 5901, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:12.409618+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:54:12.409618+0000', 'last_peered': '2024-09-10T15:54:12.409618+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:54:12.409618+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 5766, 'log_dups_size': 0, 'ondisk_log_size': 5766, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23592960, 'num_objects': 5760, 'num_object_clones': 0, 'num_object_copies': 11520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5760, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2416, 'num_write_kb': 9664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1926}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:17.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:54:17.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'6038", 'reported_seq': 9228, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:14.453227+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:54:14.453227+0000', 'last_peered': '2024-09-10T15:54:14.453227+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:54:14.453227+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 6038, 'log_dups_size': 0, 'ondisk_log_size': 6038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24702976, 'num_objects': 6031, 'num_object_clones': 0, 'num_object_copies': 12062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2457, 'num_objects_unfound': 0, 'num_objects_dirty': 6031, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4015, 'num_write_kb': 16060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1533, 'num_bytes_recovered': 6279168, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2457}, {'shards': '2', 'objects': 3572}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:17.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:54:17.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'5810", 'reported_seq': 5932, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:12.475801+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:54:12.475801+0000', 'last_peered': '2024-09-10T15:54:12.475801+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:54:12.475801+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 5810, 'log_dups_size': 0, 'ondisk_log_size': 5810, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23764992, 'num_objects': 5802, 'num_object_clones': 0, 'num_object_copies': 11604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5802, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2674, 'num_write_kb': 10696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1939}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:17.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:54:17.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'5940", 'reported_seq': 6059, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:14.414678+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:54:14.414678+0000', 'last_peered': '2024-09-10T15:54:14.414678+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:54:14.414678+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 5940, 'log_dups_size': 0, 'ondisk_log_size': 5940, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24309760, 'num_objects': 5935, 'num_object_clones': 0, 'num_object_copies': 11870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5935, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3914, 'num_write_kb': 15656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 2103}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:17.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:54:17.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'5842", 'reported_seq': 5963, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:14.407551+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:54:14.407551+0000', 'last_peered': '2024-09-10T15:54:14.407551+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:54:14.407551+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 5842, 'log_dups_size': 0, 'ondisk_log_size': 5842, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23904256, 'num_objects': 5836, 'num_object_clones': 0, 'num_object_copies': 11672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5836, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3711, 'num_write_kb': 14844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 2007}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:17.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:54:17.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'5907", 'reported_seq': 6029, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:12.401501+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:54:12.401501+0000', 'last_peered': '2024-09-10T15:54:12.401501+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:54:12.401501+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 5907, 'log_dups_size': 0, 'ondisk_log_size': 5907, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24186880, 'num_objects': 5905, 'num_object_clones': 0, 'num_object_copies': 11810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3099, 'num_write_kb': 12396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2072}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:17.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:54:17.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'5929", 'reported_seq': 6087, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:12.487733+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:54:12.487733+0000', 'last_peered': '2024-09-10T15:54:12.487733+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:54:12.487733+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 5929, 'log_dups_size': 0, 'ondisk_log_size': 5929, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24252416, 'num_objects': 5921, 'num_object_clones': 0, 'num_object_copies': 11842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5921, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2347, 'num_write_kb': 9388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2004}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:17.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:54:17.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'5813", 'reported_seq': 5947, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:12.430082+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:54:12.430082+0000', 'last_peered': '2024-09-10T15:54:12.430082+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:54:12.430082+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 5813, 'log_dups_size': 0, 'ondisk_log_size': 5813, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23793664, 'num_objects': 5809, 'num_object_clones': 0, 'num_object_copies': 11618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5809, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2419, 'num_write_kb': 9676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1932}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:17.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:54:17.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'6019", 'reported_seq': 6152, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:12.417083+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:54:12.417083+0000', 'last_peered': '2024-09-10T15:54:12.417083+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:54:12.417083+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 6019, 'log_dups_size': 0, 'ondisk_log_size': 6019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24605155, 'num_objects': 6008, 'num_object_clones': 0, 'num_object_copies': 12016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6008, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2838, 'num_write_kb': 11352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2005}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:17.585 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:54:17.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:2024-09-10T15:54:17.812083+0000 min lat: 0.00198913 max lat: 2.66872 avg lat: 0.0206035 2024-09-10T15:54:17.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:54:17.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 100 20 96956 96936 3.78596 1.84375 0.0182879 0.0206035 2024-09-10T15:54:18.102 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:18.102 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/188782 objects degraded (22.594%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":184,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":94391,"data_bytes":386613750,"bytes_used":1265569792,"bytes_avail":288644722688,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":188782,"degraded_ratio":0.22593785424457841,"misplaced_objects":2457,"misplaced_total":188782,"misplaced_ratio":0.013015012024451484,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":98622,"recovering_keys_per_sec":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"write_bytes_sec":2023972,"read_op_per_sec":0,"write_op_per_sec":494},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-10T15:53:42.448843+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:54:18.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:54:18.125 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:18.682 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:18.682 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:18.708 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-10T15:54:18.458973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":394027510,"num_objects":96201,"num_object_clones":0,"num_object_copies":192402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":96201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57803,"num_write_kb":231206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":6287379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96283,"ondisk_log_size":96283,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1233308,"kb_used_data":777448,"kb_used_omap":12,"kb_used_meta":455795,"kb_avail":281882212,"statfs":{"total":289910292480,"available":288647385088,"internally_reserved":0,"allocated":796106752,"data_stored":795260292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":466734085},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,4,9,10,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31707136,"num_objects":7741,"num_object_clones":0,"num_object_copies":15482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-289,"num_objects_unfound":0,"num_objects_dirty":7741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7741,"num_write_kb":30964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002601"},"pg_stats":[{"pgid":"2.f","version":"44'6089","reported_seq":6160,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.429701+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:17.429701+0000","last_peered":"2024-09-10T15:54:17.429701+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:17.429701+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6089,"log_dups_size":0,"ondisk_log_size":6089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6089,"num_write_kb":24356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2116},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6119","reported_seq":6221,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:14.389570+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:14.389570+0000","last_peered":"2024-09-10T15:54:14.389570+0000","last_clean":"2024-09-10T15:54:14.389570+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:14.389570+0000","last_undegraded":"2024-09-10T15:54:14.389570+0000","last_fullsized":"2024-09-10T15:54:14.389570+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6119,"log_dups_size":0,"ondisk_log_size":6119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25038848,"num_objects":6113,"num_object_clones":0,"num_object_copies":12226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4455,"num_write_kb":17820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5876","reported_seq":5992,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:14.432256+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:14.432256+0000","last_peered":"2024-09-10T15:54:14.432256+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:14.432256+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":5876,"log_dups_size":0,"ondisk_log_size":5876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24051712,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4144,"num_write_kb":16576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5882","reported_seq":5956,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:14.422900+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:14.422900+0000","last_peered":"2024-09-10T15:54:14.422900+0000","last_clean":"2024-09-10T15:54:14.422900+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:14.422900+0000","last_undegraded":"2024-09-10T15:54:14.422900+0000","last_fullsized":"2024-09-10T15:54:14.422900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":5882,"log_dups_size":0,"ondisk_log_size":5882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24092672,"num_objects":5882,"num_object_clones":0,"num_object_copies":11764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5882,"num_write_kb":23528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5957","reported_seq":6077,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:17.540438+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:17.540438+0000","last_peered":"2024-09-10T15:54:17.540438+0000","last_clean":"2024-09-10T15:54:17.540438+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:17.540438+0000","last_undegraded":"2024-09-10T15:54:17.540438+0000","last_fullsized":"2024-09-10T15:54:17.540438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5957,"log_dups_size":0,"ondisk_log_size":5957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24371200,"num_objects":5950,"num_object_clones":0,"num_object_copies":11900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3028,"num_write_kb":12112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6110","reported_seq":6237,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.515888+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:17.515888+0000","last_peered":"2024-09-10T15:54:17.515888+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:17.515888+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6110,"log_dups_size":0,"ondisk_log_size":6110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2828,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5954","reported_seq":6089,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.478627+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:17.478627+0000","last_peered":"2024-09-10T15:54:17.478627+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:17.478627+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5954,"log_dups_size":0,"ondisk_log_size":5954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24363008,"num_objects":5948,"num_object_clones":0,"num_object_copies":11896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2604,"num_write_kb":10416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2114},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6104","reported_seq":6230,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:17.478230+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:17.478230+0000","last_peered":"2024-09-10T15:54:17.478230+0000","last_clean":"2024-09-10T15:54:17.478230+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:17.478230+0000","last_undegraded":"2024-09-10T15:54:17.478230+0000","last_fullsized":"2024-09-10T15:54:17.478230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6104,"log_dups_size":0,"ondisk_log_size":6104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24993792,"num_objects":6102,"num_object_clones":0,"num_object_copies":12204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2863,"num_write_kb":11452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6038","reported_seq":9228,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:14.453227+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:14.453227+0000","last_peered":"2024-09-10T15:54:14.453227+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:14.453227+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6038,"log_dups_size":0,"ondisk_log_size":6038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24702976,"num_objects":6031,"num_object_clones":0,"num_object_copies":12062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":6031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4015,"num_write_kb":16060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":6279168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2457},{"shards":"2","objects":3572}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5985","reported_seq":6107,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.486777+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:17.486777+0000","last_peered":"2024-09-10T15:54:17.486777+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:17.486777+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5985,"log_dups_size":0,"ondisk_log_size":5985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24481792,"num_objects":5977,"num_object_clones":0,"num_object_copies":11954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2849,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2114},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5940","reported_seq":6059,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:14.414678+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:14.414678+0000","last_peered":"2024-09-10T15:54:14.414678+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:14.414678+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":5940,"log_dups_size":0,"ondisk_log_size":5940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24309760,"num_objects":5935,"num_object_clones":0,"num_object_copies":11870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3914,"num_write_kb":15656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2103}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5842","reported_seq":5963,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:14.407551+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:14.407551+0000","last_peered":"2024-09-10T15:54:14.407551+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:14.407551+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":5842,"log_dups_size":0,"ondisk_log_size":5842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23904256,"num_objects":5836,"num_object_clones":0,"num_object_copies":11672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3711,"num_write_kb":14844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2007}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6090","reported_seq":6211,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.494787+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:17.494787+0000","last_peered":"2024-09-10T15:54:17.494787+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:17.494787+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6090,"log_dups_size":0,"ondisk_log_size":6090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24936448,"num_objects":6088,"num_object_clones":0,"num_object_copies":12176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3282,"num_write_kb":13128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2255},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6092","reported_seq":6250,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.459868+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:17.459868+0000","last_peered":"2024-09-10T15:54:17.459868+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:17.459868+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6092,"log_dups_size":0,"ondisk_log_size":6092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24920064,"num_objects":6084,"num_object_clones":0,"num_object_copies":12168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2510,"num_write_kb":10040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2167},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6018","reported_seq":6151,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.520325+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:17.520325+0000","last_peered":"2024-09-10T15:54:17.520325+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:17.520325+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6018,"log_dups_size":0,"ondisk_log_size":6018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24633344,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2624,"num_write_kb":10496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2137},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6184","reported_seq":6317,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.433320+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:17.433320+0000","last_peered":"2024-09-10T15:54:17.433320+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:17.433320+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6184,"log_dups_size":0,"ondisk_log_size":6184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25280995,"num_objects":6173,"num_object_clones":0,"num_object_copies":12346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3003,"num_write_kb":12012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2170},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":394027491,"num_objects":96199,"num_object_clones":0,"num_object_copies":192398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2457,"num_objects_unfound":0,"num_objects_dirty":96199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57801,"num_write_kb":231204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":6287360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":797237248,"data_stored":797146923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96280,"ondisk_log_size":96280,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626446,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232612,"kb_used_data":110736,"kb_used_omap":4,"kb_used_meta":121851,"kb_avail":94139228,"statfs":{"total":96636764160,"available":96398569472,"internally_reserved":0,"allocated":113393664,"data_stored":113058520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124776110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149730,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524380,"kb_used_data":352524,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93847460,"statfs":{"total":96636764160,"available":96099799040,"internally_reserved":0,"allocated":360984576,"data_stored":360728918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":175959725},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,4,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476316,"kb_used_data":314188,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93895524,"statfs":{"total":96636764160,"available":96149016576,"internally_reserved":0,"allocated":321728512,"data_stored":321472854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":361299968,"data_stored":361296355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113680384,"data_stored":113597285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322256896,"data_stored":322253283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:18.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 101 20 97450 97430 3.76758 1.92969 0.0588212 0.0207282 2024-09-10T15:54:19.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 102 20 98053 98033 3.75373 2.35547 0.0343519 0.0208058 2024-09-10T15:54:20.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 103 20 98545 98525 3.73595 1.92188 0.0237811 0.020905 2024-09-10T15:54:21.710 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:21.812 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 104 20 99131 99111 3.72203 2.28906 0.0283204 0.020982 2024-09-10T15:54:22.234 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:22.234 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:22.260 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-10T15:54:20.459417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398291446,"num_objects":97242,"num_object_clones":0,"num_object_copies":194484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2289,"num_objects_unfound":0,"num_objects_dirty":97242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58844,"num_write_kb":235370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1705,"num_bytes_recovered":6975507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97324,"ondisk_log_size":97324,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1243024,"kb_used_data":783004,"kb_used_omap":12,"kb_used_meta":459955,"kb_avail":281872496,"statfs":{"total":289910292480,"available":288637435904,"internally_reserved":0,"allocated":801796096,"data_stored":800949636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":470993925},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,4,10,8,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28377088,"num_objects":6928,"num_object_clones":0,"num_object_copies":13856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-457,"num_objects_unfound":0,"num_objects_dirty":6928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6928,"num_write_kb":27712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":1871872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002597"},"pg_stats":[{"pgid":"2.f","version":"44'6089","reported_seq":6160,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.429701+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:17.429701+0000","last_peered":"2024-09-10T15:54:17.429701+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:17.429701+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6089,"log_dups_size":0,"ondisk_log_size":6089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6089,"num_write_kb":24356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2116},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6308","reported_seq":6410,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:19.435894+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:19.435894+0000","last_peered":"2024-09-10T15:54:19.435894+0000","last_clean":"2024-09-10T15:54:19.435894+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:19.435894+0000","last_undegraded":"2024-09-10T15:54:19.435894+0000","last_fullsized":"2024-09-10T15:54:19.435894+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6308,"log_dups_size":0,"ondisk_log_size":6308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25812992,"num_objects":6302,"num_object_clones":0,"num_object_copies":12604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4644,"num_write_kb":18576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6051","reported_seq":6167,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:19.474931+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:19.474931+0000","last_peered":"2024-09-10T15:54:19.474931+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:19.474931+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":17276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6049","reported_seq":6123,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:19.487839+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:19.487839+0000","last_peered":"2024-09-10T15:54:19.487839+0000","last_clean":"2024-09-10T15:54:19.487839+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:19.487839+0000","last_undegraded":"2024-09-10T15:54:19.487839+0000","last_fullsized":"2024-09-10T15:54:19.487839+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6049,"log_dups_size":0,"ondisk_log_size":6049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24776704,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6049,"num_write_kb":24196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5957","reported_seq":6077,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:17.540438+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:17.540438+0000","last_peered":"2024-09-10T15:54:17.540438+0000","last_clean":"2024-09-10T15:54:17.540438+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:17.540438+0000","last_undegraded":"2024-09-10T15:54:17.540438+0000","last_fullsized":"2024-09-10T15:54:17.540438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5957,"log_dups_size":0,"ondisk_log_size":5957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24371200,"num_objects":5950,"num_object_clones":0,"num_object_copies":11900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3028,"num_write_kb":12112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6110","reported_seq":6237,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.515888+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:17.515888+0000","last_peered":"2024-09-10T15:54:17.515888+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:17.515888+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6110,"log_dups_size":0,"ondisk_log_size":6110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2828,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5954","reported_seq":6089,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.478627+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:17.478627+0000","last_peered":"2024-09-10T15:54:17.478627+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:17.478627+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5954,"log_dups_size":0,"ondisk_log_size":5954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24363008,"num_objects":5948,"num_object_clones":0,"num_object_copies":11896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2604,"num_write_kb":10416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2114},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6104","reported_seq":6230,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:17.478230+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:17.478230+0000","last_peered":"2024-09-10T15:54:17.478230+0000","last_clean":"2024-09-10T15:54:17.478230+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:17.478230+0000","last_undegraded":"2024-09-10T15:54:17.478230+0000","last_fullsized":"2024-09-10T15:54:17.478230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6104,"log_dups_size":0,"ondisk_log_size":6104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24993792,"num_objects":6102,"num_object_clones":0,"num_object_copies":12204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2863,"num_write_kb":11452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6215","reported_seq":9741,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:19.459976+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:19.459976+0000","last_peered":"2024-09-10T15:54:19.459976+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:19.459976+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6215,"log_dups_size":0,"ondisk_log_size":6215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25427968,"num_objects":6208,"num_object_clones":0,"num_object_copies":12416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2289,"num_objects_unfound":0,"num_objects_dirty":6208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4192,"num_write_kb":16768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":6967296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2289},{"shards":"2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5985","reported_seq":6107,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.486777+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:17.486777+0000","last_peered":"2024-09-10T15:54:17.486777+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:17.486777+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5985,"log_dups_size":0,"ondisk_log_size":5985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24481792,"num_objects":5977,"num_object_clones":0,"num_object_copies":11954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2849,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2114},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6097","reported_seq":6214,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:19.461342+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:19.461342+0000","last_peered":"2024-09-10T15:54:19.461342+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:19.461342+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6097,"log_dups_size":0,"ondisk_log_size":6097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952832,"num_objects":6092,"num_object_clones":0,"num_object_copies":12184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4071,"num_write_kb":16284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6018","reported_seq":6138,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:19.460026+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:19.460026+0000","last_peered":"2024-09-10T15:54:19.460026+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:19.460026+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6018,"log_dups_size":0,"ondisk_log_size":6018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24625152,"num_objects":6012,"num_object_clones":0,"num_object_copies":12024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3887,"num_write_kb":15548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2183}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6090","reported_seq":6211,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.494787+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:17.494787+0000","last_peered":"2024-09-10T15:54:17.494787+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:17.494787+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6090,"log_dups_size":0,"ondisk_log_size":6090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24936448,"num_objects":6088,"num_object_clones":0,"num_object_copies":12176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3282,"num_write_kb":13128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2255},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6092","reported_seq":6250,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.459868+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:17.459868+0000","last_peered":"2024-09-10T15:54:17.459868+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:17.459868+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6092,"log_dups_size":0,"ondisk_log_size":6092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24920064,"num_objects":6084,"num_object_clones":0,"num_object_copies":12168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2510,"num_write_kb":10040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2167},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6018","reported_seq":6151,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.520325+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:17.520325+0000","last_peered":"2024-09-10T15:54:17.520325+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:17.520325+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6018,"log_dups_size":0,"ondisk_log_size":6018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24633344,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2624,"num_write_kb":10496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2137},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6184","reported_seq":6317,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.433320+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:17.433320+0000","last_peered":"2024-09-10T15:54:17.433320+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:17.433320+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6184,"log_dups_size":0,"ondisk_log_size":6184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25280995,"num_objects":6173,"num_object_clones":0,"num_object_copies":12346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3003,"num_write_kb":12012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2170},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398291427,"num_objects":97240,"num_object_clones":0,"num_object_copies":194480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2289,"num_objects_unfound":0,"num_objects_dirty":97240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58842,"num_write_kb":235368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1703,"num_bytes_recovered":6975488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802914304,"data_stored":802823979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97321,"ondisk_log_size":97321,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626446,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232612,"kb_used_data":110736,"kb_used_omap":4,"kb_used_meta":121851,"kb_avail":94139228,"statfs":{"total":96636764160,"available":96398569472,"internally_reserved":0,"allocated":113393664,"data_stored":113058520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124776110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149730,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524380,"kb_used_data":352524,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93847460,"statfs":{"total":96636764160,"available":96099799040,"internally_reserved":0,"allocated":360984576,"data_stored":360728918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":175959725},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,4,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486032,"kb_used_data":319744,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93885808,"statfs":{"total":96636764160,"available":96139067392,"internally_reserved":0,"allocated":327417856,"data_stored":327162198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":361299968,"data_stored":361296355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113680384,"data_stored":113597285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327933952,"data_stored":327930339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:22.261 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:22.813 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 105 20 99649 99629 3.70586 2.02344 0.0379014 0.0210741 2024-09-10T15:54:22.814 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:22.814 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:22.843 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-10T15:54:22.459900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398291446,"num_objects":97242,"num_object_clones":0,"num_object_copies":194484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2289,"num_objects_unfound":0,"num_objects_dirty":97242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58844,"num_write_kb":235370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1705,"num_bytes_recovered":6975507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97324,"ondisk_log_size":97324,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1243024,"kb_used_data":783004,"kb_used_omap":12,"kb_used_meta":459955,"kb_avail":281872496,"statfs":{"total":289910292480,"available":288637435904,"internally_reserved":0,"allocated":801796096,"data_stored":800949636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":470993925},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,4,10,8,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23814144,"num_objects":5814,"num_object_clones":0,"num_object_copies":11628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-312,"num_objects_unfound":0,"num_objects_dirty":5814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5814,"num_write_kb":23256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":1277952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002613"},"pg_stats":[{"pgid":"2.f","version":"44'6089","reported_seq":6160,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.429701+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:17.429701+0000","last_peered":"2024-09-10T15:54:17.429701+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:17.429701+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6089,"log_dups_size":0,"ondisk_log_size":6089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6089,"num_write_kb":24356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2116},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6308","reported_seq":6410,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:19.435894+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:19.435894+0000","last_peered":"2024-09-10T15:54:19.435894+0000","last_clean":"2024-09-10T15:54:19.435894+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:19.435894+0000","last_undegraded":"2024-09-10T15:54:19.435894+0000","last_fullsized":"2024-09-10T15:54:19.435894+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6308,"log_dups_size":0,"ondisk_log_size":6308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25812992,"num_objects":6302,"num_object_clones":0,"num_object_copies":12604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4644,"num_write_kb":18576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6051","reported_seq":6167,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:19.474931+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:19.474931+0000","last_peered":"2024-09-10T15:54:19.474931+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:19.474931+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":17276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6049","reported_seq":6123,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:19.487839+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:19.487839+0000","last_peered":"2024-09-10T15:54:19.487839+0000","last_clean":"2024-09-10T15:54:19.487839+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:19.487839+0000","last_undegraded":"2024-09-10T15:54:19.487839+0000","last_fullsized":"2024-09-10T15:54:19.487839+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6049,"log_dups_size":0,"ondisk_log_size":6049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24776704,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6049,"num_write_kb":24196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5957","reported_seq":6077,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:17.540438+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:17.540438+0000","last_peered":"2024-09-10T15:54:17.540438+0000","last_clean":"2024-09-10T15:54:17.540438+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:17.540438+0000","last_undegraded":"2024-09-10T15:54:17.540438+0000","last_fullsized":"2024-09-10T15:54:17.540438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5957,"log_dups_size":0,"ondisk_log_size":5957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24371200,"num_objects":5950,"num_object_clones":0,"num_object_copies":11900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3028,"num_write_kb":12112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6110","reported_seq":6237,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.515888+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:17.515888+0000","last_peered":"2024-09-10T15:54:17.515888+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:17.515888+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6110,"log_dups_size":0,"ondisk_log_size":6110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2828,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5954","reported_seq":6089,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.478627+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:17.478627+0000","last_peered":"2024-09-10T15:54:17.478627+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:17.478627+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5954,"log_dups_size":0,"ondisk_log_size":5954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24363008,"num_objects":5948,"num_object_clones":0,"num_object_copies":11896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2604,"num_write_kb":10416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2114},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6104","reported_seq":6230,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:17.478230+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:17.478230+0000","last_peered":"2024-09-10T15:54:17.478230+0000","last_clean":"2024-09-10T15:54:17.478230+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:17.478230+0000","last_undegraded":"2024-09-10T15:54:17.478230+0000","last_fullsized":"2024-09-10T15:54:17.478230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6104,"log_dups_size":0,"ondisk_log_size":6104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24993792,"num_objects":6102,"num_object_clones":0,"num_object_copies":12204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2863,"num_write_kb":11452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6215","reported_seq":9741,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:19.459976+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:19.459976+0000","last_peered":"2024-09-10T15:54:19.459976+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:19.459976+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6215,"log_dups_size":0,"ondisk_log_size":6215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25427968,"num_objects":6208,"num_object_clones":0,"num_object_copies":12416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2289,"num_objects_unfound":0,"num_objects_dirty":6208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4192,"num_write_kb":16768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":6967296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2289},{"shards":"2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5985","reported_seq":6107,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.486777+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:17.486777+0000","last_peered":"2024-09-10T15:54:17.486777+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:17.486777+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5985,"log_dups_size":0,"ondisk_log_size":5985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24481792,"num_objects":5977,"num_object_clones":0,"num_object_copies":11954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2849,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2114},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6097","reported_seq":6214,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:19.461342+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:19.461342+0000","last_peered":"2024-09-10T15:54:19.461342+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:19.461342+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6097,"log_dups_size":0,"ondisk_log_size":6097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952832,"num_objects":6092,"num_object_clones":0,"num_object_copies":12184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4071,"num_write_kb":16284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6018","reported_seq":6138,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:19.460026+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:19.460026+0000","last_peered":"2024-09-10T15:54:19.460026+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:19.460026+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6018,"log_dups_size":0,"ondisk_log_size":6018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24625152,"num_objects":6012,"num_object_clones":0,"num_object_copies":12024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3887,"num_write_kb":15548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2183}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6090","reported_seq":6211,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.494787+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:17.494787+0000","last_peered":"2024-09-10T15:54:17.494787+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:17.494787+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6090,"log_dups_size":0,"ondisk_log_size":6090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24936448,"num_objects":6088,"num_object_clones":0,"num_object_copies":12176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3282,"num_write_kb":13128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2255},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6092","reported_seq":6250,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.459868+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:17.459868+0000","last_peered":"2024-09-10T15:54:17.459868+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:17.459868+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6092,"log_dups_size":0,"ondisk_log_size":6092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24920064,"num_objects":6084,"num_object_clones":0,"num_object_copies":12168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2510,"num_write_kb":10040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2167},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6018","reported_seq":6151,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.520325+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:17.520325+0000","last_peered":"2024-09-10T15:54:17.520325+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:17.520325+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6018,"log_dups_size":0,"ondisk_log_size":6018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24633344,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2624,"num_write_kb":10496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2137},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6184","reported_seq":6317,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.433320+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:17.433320+0000","last_peered":"2024-09-10T15:54:17.433320+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:17.433320+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6184,"log_dups_size":0,"ondisk_log_size":6184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25280995,"num_objects":6173,"num_object_clones":0,"num_object_copies":12346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3003,"num_write_kb":12012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2170},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398291427,"num_objects":97240,"num_object_clones":0,"num_object_copies":194480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2289,"num_objects_unfound":0,"num_objects_dirty":97240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58842,"num_write_kb":235368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1703,"num_bytes_recovered":6975488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802914304,"data_stored":802823979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97321,"ondisk_log_size":97321,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626446,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232612,"kb_used_data":110736,"kb_used_omap":4,"kb_used_meta":121851,"kb_avail":94139228,"statfs":{"total":96636764160,"available":96398569472,"internally_reserved":0,"allocated":113393664,"data_stored":113058520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124776110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149730,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524380,"kb_used_data":352524,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93847460,"statfs":{"total":96636764160,"available":96099799040,"internally_reserved":0,"allocated":360984576,"data_stored":360728918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":175959725},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,4,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486032,"kb_used_data":319744,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93885808,"statfs":{"total":96636764160,"available":96139067392,"internally_reserved":0,"allocated":327417856,"data_stored":327162198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":361299968,"data_stored":361296355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113680384,"data_stored":113597285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327933952,"data_stored":327930339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:22.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:54:22.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'6089", 'reported_seq': 6160, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:17.429701+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:54:17.429701+0000', 'last_peered': '2024-09-10T15:54:17.429701+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:54:17.429701+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 6089, 'log_dups_size': 0, 'ondisk_log_size': 6089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24940544, 'num_objects': 6089, 'num_object_clones': 0, 'num_object_copies': 12178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6089, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6089, 'num_write_kb': 24356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2116}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:22.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:54:22.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'6051", 'reported_seq': 6167, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:19.474931+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:54:19.474931+0000', 'last_peered': '2024-09-10T15:54:19.474931+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:54:19.474931+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 6051, 'log_dups_size': 0, 'ondisk_log_size': 6051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24768512, 'num_objects': 6047, 'num_object_clones': 0, 'num_object_copies': 12094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4319, 'num_write_kb': 17276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 2245}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:22.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:54:22.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'6110", 'reported_seq': 6237, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:17.515888+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:54:17.515888+0000', 'last_peered': '2024-09-10T15:54:17.515888+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:54:17.515888+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 6110, 'log_dups_size': 0, 'ondisk_log_size': 6110, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25006080, 'num_objects': 6105, 'num_object_clones': 0, 'num_object_copies': 12210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2828, 'num_write_kb': 11312, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2237}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:22.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:54:22.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'5954", 'reported_seq': 6089, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:17.478627+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:54:17.478627+0000', 'last_peered': '2024-09-10T15:54:17.478627+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:54:17.478627+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 5954, 'log_dups_size': 0, 'ondisk_log_size': 5954, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24363008, 'num_objects': 5948, 'num_object_clones': 0, 'num_object_copies': 11896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5948, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2604, 'num_write_kb': 10416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2114}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:22.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:54:22.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'6215", 'reported_seq': 9741, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:19.459976+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:54:19.459976+0000', 'last_peered': '2024-09-10T15:54:19.459976+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:54:19.459976+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 6215, 'log_dups_size': 0, 'ondisk_log_size': 6215, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25427968, 'num_objects': 6208, 'num_object_clones': 0, 'num_object_copies': 12416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2289, 'num_objects_unfound': 0, 'num_objects_dirty': 6208, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4192, 'num_write_kb': 16768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1701, 'num_bytes_recovered': 6967296, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2289}, {'shards': '2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:22.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:54:22.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'5985", 'reported_seq': 6107, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:17.486777+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:54:17.486777+0000', 'last_peered': '2024-09-10T15:54:17.486777+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:54:17.486777+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 5985, 'log_dups_size': 0, 'ondisk_log_size': 5985, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24481792, 'num_objects': 5977, 'num_object_clones': 0, 'num_object_copies': 11954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5977, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2849, 'num_write_kb': 11396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2114}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:22.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:54:22.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'6097", 'reported_seq': 6214, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:19.461342+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:54:19.461342+0000', 'last_peered': '2024-09-10T15:54:19.461342+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:54:19.461342+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 6097, 'log_dups_size': 0, 'ondisk_log_size': 6097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24952832, 'num_objects': 6092, 'num_object_clones': 0, 'num_object_copies': 12184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6092, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4071, 'num_write_kb': 16284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 2260}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:22.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:54:22.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'6018", 'reported_seq': 6138, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:19.460026+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:54:19.460026+0000', 'last_peered': '2024-09-10T15:54:19.460026+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:54:19.460026+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 6018, 'log_dups_size': 0, 'ondisk_log_size': 6018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24625152, 'num_objects': 6012, 'num_object_clones': 0, 'num_object_copies': 12024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6012, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3887, 'num_write_kb': 15548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 2183}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:22.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:54:22.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'6090", 'reported_seq': 6211, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:17.494787+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:54:17.494787+0000', 'last_peered': '2024-09-10T15:54:17.494787+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:54:17.494787+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 6090, 'log_dups_size': 0, 'ondisk_log_size': 6090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24936448, 'num_objects': 6088, 'num_object_clones': 0, 'num_object_copies': 12176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6088, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3282, 'num_write_kb': 13128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2255}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:22.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:54:22.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'6092", 'reported_seq': 6250, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:17.459868+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:54:17.459868+0000', 'last_peered': '2024-09-10T15:54:17.459868+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:54:17.459868+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 6092, 'log_dups_size': 0, 'ondisk_log_size': 6092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24920064, 'num_objects': 6084, 'num_object_clones': 0, 'num_object_copies': 12168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6084, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2510, 'num_write_kb': 10040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2167}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:22.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:54:22.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'6018", 'reported_seq': 6151, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:17.520325+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:54:17.520325+0000', 'last_peered': '2024-09-10T15:54:17.520325+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:54:17.520325+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 6018, 'log_dups_size': 0, 'ondisk_log_size': 6018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24633344, 'num_objects': 6014, 'num_object_clones': 0, 'num_object_copies': 12028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6014, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2624, 'num_write_kb': 10496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2137}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:22.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:54:22.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'6184", 'reported_seq': 6317, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:17.433320+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:54:17.433320+0000', 'last_peered': '2024-09-10T15:54:17.433320+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:54:17.433320+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 6184, 'log_dups_size': 0, 'ondisk_log_size': 6184, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25280995, 'num_objects': 6173, 'num_object_clones': 0, 'num_object_copies': 12346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3003, 'num_write_kb': 12012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2170}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:22.847 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:54:23.521 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:23.521 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/194484 objects degraded (21.931%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":189,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":97242,"data_bytes":398291446,"bytes_used":1272856576,"bytes_avail":288637435904,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":194484,"degraded_ratio":0.21931367104748978,"misplaced_objects":2289,"misplaced_total":194484,"misplaced_ratio":0.0117696057259209,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":106472,"recovering_keys_per_sec":0,"num_objects_recovered":312,"num_bytes_recovered":1277952,"num_keys_recovered":0,"write_bytes_sec":1984079,"read_op_per_sec":0,"write_op_per_sec":484},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T15:54:18.459081+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:54:23.549 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:54:23.550 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:23.813 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 106 20 100171 100151 3.69013 2.03906 0.0214509 0.0211637 2024-09-10T15:54:24.125 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:24.125 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:24.149 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-10T15:54:22.459900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398291446,"num_objects":97242,"num_object_clones":0,"num_object_copies":194484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2289,"num_objects_unfound":0,"num_objects_dirty":97242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58844,"num_write_kb":235370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1705,"num_bytes_recovered":6975507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97324,"ondisk_log_size":97324,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1243024,"kb_used_data":783004,"kb_used_omap":12,"kb_used_meta":459955,"kb_avail":281872496,"statfs":{"total":289910292480,"available":288637435904,"internally_reserved":0,"allocated":801796096,"data_stored":800949636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":470993925},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,4,10,8,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23814144,"num_objects":5814,"num_object_clones":0,"num_object_copies":11628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-312,"num_objects_unfound":0,"num_objects_dirty":5814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5814,"num_write_kb":23256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":1277952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002613"},"pg_stats":[{"pgid":"2.f","version":"44'6089","reported_seq":6160,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.429701+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:17.429701+0000","last_peered":"2024-09-10T15:54:17.429701+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:17.429701+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6089,"log_dups_size":0,"ondisk_log_size":6089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6089,"num_write_kb":24356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2116},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6308","reported_seq":6410,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:19.435894+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:19.435894+0000","last_peered":"2024-09-10T15:54:19.435894+0000","last_clean":"2024-09-10T15:54:19.435894+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:19.435894+0000","last_undegraded":"2024-09-10T15:54:19.435894+0000","last_fullsized":"2024-09-10T15:54:19.435894+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6308,"log_dups_size":0,"ondisk_log_size":6308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25812992,"num_objects":6302,"num_object_clones":0,"num_object_copies":12604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4644,"num_write_kb":18576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6051","reported_seq":6167,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:19.474931+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:19.474931+0000","last_peered":"2024-09-10T15:54:19.474931+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:19.474931+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24768512,"num_objects":6047,"num_object_clones":0,"num_object_copies":12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":17276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6049","reported_seq":6123,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:19.487839+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:19.487839+0000","last_peered":"2024-09-10T15:54:19.487839+0000","last_clean":"2024-09-10T15:54:19.487839+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:19.487839+0000","last_undegraded":"2024-09-10T15:54:19.487839+0000","last_fullsized":"2024-09-10T15:54:19.487839+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6049,"log_dups_size":0,"ondisk_log_size":6049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24776704,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6049,"num_write_kb":24196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5957","reported_seq":6077,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:17.540438+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:17.540438+0000","last_peered":"2024-09-10T15:54:17.540438+0000","last_clean":"2024-09-10T15:54:17.540438+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:17.540438+0000","last_undegraded":"2024-09-10T15:54:17.540438+0000","last_fullsized":"2024-09-10T15:54:17.540438+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":5957,"log_dups_size":0,"ondisk_log_size":5957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24371200,"num_objects":5950,"num_object_clones":0,"num_object_copies":11900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3028,"num_write_kb":12112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6110","reported_seq":6237,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.515888+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:17.515888+0000","last_peered":"2024-09-10T15:54:17.515888+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:17.515888+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6110,"log_dups_size":0,"ondisk_log_size":6110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2828,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5954","reported_seq":6089,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.478627+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:17.478627+0000","last_peered":"2024-09-10T15:54:17.478627+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:17.478627+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":5954,"log_dups_size":0,"ondisk_log_size":5954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24363008,"num_objects":5948,"num_object_clones":0,"num_object_copies":11896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2604,"num_write_kb":10416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2114},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6104","reported_seq":6230,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:17.478230+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:17.478230+0000","last_peered":"2024-09-10T15:54:17.478230+0000","last_clean":"2024-09-10T15:54:17.478230+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:17.478230+0000","last_undegraded":"2024-09-10T15:54:17.478230+0000","last_fullsized":"2024-09-10T15:54:17.478230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6104,"log_dups_size":0,"ondisk_log_size":6104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24993792,"num_objects":6102,"num_object_clones":0,"num_object_copies":12204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2863,"num_write_kb":11452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6215","reported_seq":9741,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:19.459976+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:19.459976+0000","last_peered":"2024-09-10T15:54:19.459976+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:19.459976+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6215,"log_dups_size":0,"ondisk_log_size":6215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25427968,"num_objects":6208,"num_object_clones":0,"num_object_copies":12416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2289,"num_objects_unfound":0,"num_objects_dirty":6208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4192,"num_write_kb":16768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1701,"num_bytes_recovered":6967296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2289},{"shards":"2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5985","reported_seq":6107,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.486777+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:17.486777+0000","last_peered":"2024-09-10T15:54:17.486777+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:17.486777+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":5985,"log_dups_size":0,"ondisk_log_size":5985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24481792,"num_objects":5977,"num_object_clones":0,"num_object_copies":11954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2849,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2114},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6097","reported_seq":6214,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:19.461342+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:19.461342+0000","last_peered":"2024-09-10T15:54:19.461342+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:19.461342+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6097,"log_dups_size":0,"ondisk_log_size":6097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952832,"num_objects":6092,"num_object_clones":0,"num_object_copies":12184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4071,"num_write_kb":16284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6018","reported_seq":6138,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:19.460026+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:19.460026+0000","last_peered":"2024-09-10T15:54:19.460026+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:19.460026+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6018,"log_dups_size":0,"ondisk_log_size":6018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24625152,"num_objects":6012,"num_object_clones":0,"num_object_copies":12024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3887,"num_write_kb":15548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2183}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6090","reported_seq":6211,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.494787+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:17.494787+0000","last_peered":"2024-09-10T15:54:17.494787+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:17.494787+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6090,"log_dups_size":0,"ondisk_log_size":6090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24936448,"num_objects":6088,"num_object_clones":0,"num_object_copies":12176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3282,"num_write_kb":13128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2255},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6092","reported_seq":6250,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.459868+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:17.459868+0000","last_peered":"2024-09-10T15:54:17.459868+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:17.459868+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6092,"log_dups_size":0,"ondisk_log_size":6092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24920064,"num_objects":6084,"num_object_clones":0,"num_object_copies":12168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2510,"num_write_kb":10040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2167},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6018","reported_seq":6151,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.520325+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:17.520325+0000","last_peered":"2024-09-10T15:54:17.520325+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:17.520325+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6018,"log_dups_size":0,"ondisk_log_size":6018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24633344,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2624,"num_write_kb":10496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2137},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6184","reported_seq":6317,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:17.433320+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:17.433320+0000","last_peered":"2024-09-10T15:54:17.433320+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:17.433320+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6184,"log_dups_size":0,"ondisk_log_size":6184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25280995,"num_objects":6173,"num_object_clones":0,"num_object_copies":12346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3003,"num_write_kb":12012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2170},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398291427,"num_objects":97240,"num_object_clones":0,"num_object_copies":194480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2289,"num_objects_unfound":0,"num_objects_dirty":97240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58842,"num_write_kb":235368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1703,"num_bytes_recovered":6975488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802914304,"data_stored":802823979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97321,"ondisk_log_size":97321,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626446,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232612,"kb_used_data":110736,"kb_used_omap":4,"kb_used_meta":121851,"kb_avail":94139228,"statfs":{"total":96636764160,"available":96398569472,"internally_reserved":0,"allocated":113393664,"data_stored":113058520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124776110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149730,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524380,"kb_used_data":352524,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93847460,"statfs":{"total":96636764160,"available":96099799040,"internally_reserved":0,"allocated":360984576,"data_stored":360728918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":175959725},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,4,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486032,"kb_used_data":319744,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93885808,"statfs":{"total":96636764160,"available":96139067392,"internally_reserved":0,"allocated":327417856,"data_stored":327162198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":361299968,"data_stored":361296355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113680384,"data_stored":113597285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327933952,"data_stored":327930339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:24.813 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 107 20 100744 100724 3.67656 2.23828 0.013443 0.0212419 2024-09-10T15:54:25.813 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 108 20 101341 101321 3.66411 2.33203 0.0254322 0.021315 2024-09-10T15:54:26.813 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 109 20 101864 101844 3.64924 2.04297 0.0233567 0.0213996 2024-09-10T15:54:27.151 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:27.692 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:27.695 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:27.719 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-10T15:54:26.460904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":409219574,"num_objects":99910,"num_object_clones":0,"num_object_copies":199820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2095,"num_objects_unfound":0,"num_objects_dirty":99910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61512,"num_write_kb":246042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1899,"num_bytes_recovered":7770131,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99992,"ondisk_log_size":99992,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1304020,"kb_used_data":805152,"kb_used_omap":12,"kb_used_meta":498803,"kb_avail":281811500,"statfs":{"total":289910292480,"available":288574976000,"internally_reserved":0,"allocated":824475648,"data_stored":823629188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":510774277},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,5,6,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27205632,"num_objects":6642,"num_object_clones":0,"num_object_copies":13284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-506,"num_objects_unfound":0,"num_objects_dirty":6642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6642,"num_write_kb":26568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":2072576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002795"},"pg_stats":[{"pgid":"2.f","version":"44'6250","reported_seq":6321,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.500210+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:22.500210+0000","last_peered":"2024-09-10T15:54:22.500210+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:22.500210+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6250,"log_dups_size":0,"ondisk_log_size":6250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25600000,"num_objects":6250,"num_object_clones":0,"num_object_copies":12500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6250,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2277},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6477","reported_seq":6579,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:24.496495+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:24.496495+0000","last_peered":"2024-09-10T15:54:24.496495+0000","last_clean":"2024-09-10T15:54:24.496495+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:24.496495+0000","last_undegraded":"2024-09-10T15:54:24.496495+0000","last_fullsized":"2024-09-10T15:54:24.496495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6477,"log_dups_size":0,"ondisk_log_size":6477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26505216,"num_objects":6471,"num_object_clones":0,"num_object_copies":12942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4813,"num_write_kb":19252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6211","reported_seq":6329,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:24.501427+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:24.501427+0000","last_peered":"2024-09-10T15:54:24.501427+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:24.501427+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25423872,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4479,"num_write_kb":17916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2405}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6220","reported_seq":6294,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:24.489940+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:24.489940+0000","last_peered":"2024-09-10T15:54:24.489940+0000","last_clean":"2024-09-10T15:54:24.489940+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:24.489940+0000","last_undegraded":"2024-09-10T15:54:24.489940+0000","last_fullsized":"2024-09-10T15:54:24.489940+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6220,"log_dups_size":0,"ondisk_log_size":6220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25477120,"num_objects":6220,"num_object_clones":0,"num_object_copies":12440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6220,"num_write_kb":24880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6123","reported_seq":6245,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:22.477410+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:22.477410+0000","last_peered":"2024-09-10T15:54:22.477410+0000","last_clean":"2024-09-10T15:54:22.477410+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:22.477410+0000","last_undegraded":"2024-09-10T15:54:22.477410+0000","last_fullsized":"2024-09-10T15:54:22.477410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6123,"log_dups_size":0,"ondisk_log_size":6123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051136,"num_objects":6116,"num_object_clones":0,"num_object_copies":12232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3194,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6264","reported_seq":6391,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.529692+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:22.529692+0000","last_peered":"2024-09-10T15:54:22.529692+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:22.529692+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6264,"log_dups_size":0,"ondisk_log_size":6264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25636864,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2982,"num_write_kb":11928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2391},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6134","reported_seq":6268,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.567601+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:22.567601+0000","last_peered":"2024-09-10T15:54:22.567601+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:22.567601+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6134,"log_dups_size":0,"ondisk_log_size":6134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25100288,"num_objects":6128,"num_object_clones":0,"num_object_copies":12256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2784,"num_write_kb":11136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2294},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6276","reported_seq":6402,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:22.553704+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:22.553704+0000","last_peered":"2024-09-10T15:54:22.553704+0000","last_clean":"2024-09-10T15:54:22.553704+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:22.553704+0000","last_undegraded":"2024-09-10T15:54:22.553704+0000","last_fullsized":"2024-09-10T15:54:22.553704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6276,"log_dups_size":0,"ondisk_log_size":6276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25698304,"num_objects":6274,"num_object_clones":0,"num_object_copies":12548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3035,"num_write_kb":12140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6367","reported_seq":10281,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:24.489679+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:24.489679+0000","last_peered":"2024-09-10T15:54:24.489679+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:24.489679+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6367,"log_dups_size":0,"ondisk_log_size":6367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26050560,"num_objects":6360,"num_object_clones":0,"num_object_copies":12720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2095,"num_objects_unfound":0,"num_objects_dirty":6360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4344,"num_write_kb":17376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1895,"num_bytes_recovered":7761920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2095},{"shards":"2","objects":4263}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6173","reported_seq":6295,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.514301+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:22.514301+0000","last_peered":"2024-09-10T15:54:22.514301+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:22.514301+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":6173,"log_dups_size":0,"ondisk_log_size":6173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25251840,"num_objects":6165,"num_object_clones":0,"num_object_copies":12330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3037,"num_write_kb":12148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2302},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6267","reported_seq":6384,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:24.497029+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:24.497029+0000","last_peered":"2024-09-10T15:54:24.497029+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:24.497029+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6267,"log_dups_size":0,"ondisk_log_size":6267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25649152,"num_objects":6262,"num_object_clones":0,"num_object_copies":12524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4241,"num_write_kb":16964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2430}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6174","reported_seq":6295,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:24.456328+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:24.456328+0000","last_peered":"2024-09-10T15:54:24.456328+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:24.456328+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6174,"log_dups_size":0,"ondisk_log_size":6174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25264128,"num_objects":6168,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4043,"num_write_kb":16172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2339}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6258","reported_seq":6378,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.530333+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:22.530333+0000","last_peered":"2024-09-10T15:54:22.530333+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:22.530333+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6258,"log_dups_size":0,"ondisk_log_size":6258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25624576,"num_objects":6256,"num_object_clones":0,"num_object_copies":12512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3450,"num_write_kb":13800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2423},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6248","reported_seq":6406,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.552641+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:22.552641+0000","last_peered":"2024-09-10T15:54:22.552641+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:22.552641+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6248,"log_dups_size":0,"ondisk_log_size":6248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25559040,"num_objects":6240,"num_object_clones":0,"num_object_copies":12480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2666,"num_write_kb":10664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2323},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6189","reported_seq":6323,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.575668+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:22.575668+0000","last_peered":"2024-09-10T15:54:22.575668+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:22.575668+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6189,"log_dups_size":0,"ondisk_log_size":6189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2795,"num_write_kb":11180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2308},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6358","reported_seq":6491,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.496830+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:22.496830+0000","last_peered":"2024-09-10T15:54:22.496830+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:22.496830+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6358,"log_dups_size":0,"ondisk_log_size":6358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25993699,"num_objects":6347,"num_object_clones":0,"num_object_copies":12694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3177,"num_write_kb":12708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2344},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":409219555,"num_objects":99908,"num_object_clones":0,"num_object_copies":199816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2095,"num_objects_unfound":0,"num_objects_dirty":99908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61510,"num_write_kb":246040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1897,"num_bytes_recovered":7770112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":825753600,"data_stored":825663275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99989,"ondisk_log_size":99989,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626447,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":249728,"kb_used_data":119536,"kb_used_omap":4,"kb_used_meta":130171,"kb_avail":94122112,"statfs":{"total":96636764160,"available":96381042688,"internally_reserved":0,"allocated":122404864,"data_stored":122069720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":133295790},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149731,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540772,"kb_used_data":360592,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93831068,"statfs":{"total":96636764160,"available":96083013632,"internally_reserved":0,"allocated":369246208,"data_stored":368990550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184479405},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,1,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513520,"kb_used_data":325024,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93858320,"statfs":{"total":96636764160,"available":96110919680,"internally_reserved":0,"allocated":332824576,"data_stored":332568918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":192999082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":369586176,"data_stored":369582563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122863616,"data_stored":122780517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333303808,"data_stored":333300195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:27.721 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:27.813 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 110 20 102397 102377 3.63499 2.08203 0.0546512 0.0214811 2024-09-10T15:54:28.273 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:28.273 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:28.302 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-10T15:54:26.460904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":409219574,"num_objects":99910,"num_object_clones":0,"num_object_copies":199820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2095,"num_objects_unfound":0,"num_objects_dirty":99910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61512,"num_write_kb":246042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1899,"num_bytes_recovered":7770131,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99992,"ondisk_log_size":99992,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1304020,"kb_used_data":805152,"kb_used_omap":12,"kb_used_meta":498803,"kb_avail":281811500,"statfs":{"total":289910292480,"available":288574976000,"internally_reserved":0,"allocated":824475648,"data_stored":823629188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":510774277},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,5,6,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27205632,"num_objects":6642,"num_object_clones":0,"num_object_copies":13284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-506,"num_objects_unfound":0,"num_objects_dirty":6642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6642,"num_write_kb":26568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":2072576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002795"},"pg_stats":[{"pgid":"2.f","version":"44'6250","reported_seq":6321,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.500210+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:22.500210+0000","last_peered":"2024-09-10T15:54:22.500210+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:22.500210+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6250,"log_dups_size":0,"ondisk_log_size":6250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25600000,"num_objects":6250,"num_object_clones":0,"num_object_copies":12500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6250,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2277},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6477","reported_seq":6579,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:24.496495+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:24.496495+0000","last_peered":"2024-09-10T15:54:24.496495+0000","last_clean":"2024-09-10T15:54:24.496495+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:24.496495+0000","last_undegraded":"2024-09-10T15:54:24.496495+0000","last_fullsized":"2024-09-10T15:54:24.496495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6477,"log_dups_size":0,"ondisk_log_size":6477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26505216,"num_objects":6471,"num_object_clones":0,"num_object_copies":12942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4813,"num_write_kb":19252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6211","reported_seq":6329,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:24.501427+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:24.501427+0000","last_peered":"2024-09-10T15:54:24.501427+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:24.501427+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25423872,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4479,"num_write_kb":17916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2405}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6220","reported_seq":6294,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:24.489940+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:24.489940+0000","last_peered":"2024-09-10T15:54:24.489940+0000","last_clean":"2024-09-10T15:54:24.489940+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:24.489940+0000","last_undegraded":"2024-09-10T15:54:24.489940+0000","last_fullsized":"2024-09-10T15:54:24.489940+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6220,"log_dups_size":0,"ondisk_log_size":6220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25477120,"num_objects":6220,"num_object_clones":0,"num_object_copies":12440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6220,"num_write_kb":24880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6123","reported_seq":6245,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:22.477410+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:22.477410+0000","last_peered":"2024-09-10T15:54:22.477410+0000","last_clean":"2024-09-10T15:54:22.477410+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:22.477410+0000","last_undegraded":"2024-09-10T15:54:22.477410+0000","last_fullsized":"2024-09-10T15:54:22.477410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6123,"log_dups_size":0,"ondisk_log_size":6123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051136,"num_objects":6116,"num_object_clones":0,"num_object_copies":12232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3194,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6264","reported_seq":6391,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.529692+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:22.529692+0000","last_peered":"2024-09-10T15:54:22.529692+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:22.529692+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6264,"log_dups_size":0,"ondisk_log_size":6264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25636864,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2982,"num_write_kb":11928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2391},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6134","reported_seq":6268,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.567601+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:22.567601+0000","last_peered":"2024-09-10T15:54:22.567601+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:22.567601+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6134,"log_dups_size":0,"ondisk_log_size":6134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25100288,"num_objects":6128,"num_object_clones":0,"num_object_copies":12256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2784,"num_write_kb":11136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2294},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6276","reported_seq":6402,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:22.553704+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:22.553704+0000","last_peered":"2024-09-10T15:54:22.553704+0000","last_clean":"2024-09-10T15:54:22.553704+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:22.553704+0000","last_undegraded":"2024-09-10T15:54:22.553704+0000","last_fullsized":"2024-09-10T15:54:22.553704+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6276,"log_dups_size":0,"ondisk_log_size":6276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25698304,"num_objects":6274,"num_object_clones":0,"num_object_copies":12548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3035,"num_write_kb":12140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6367","reported_seq":10281,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:24.489679+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:24.489679+0000","last_peered":"2024-09-10T15:54:24.489679+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:24.489679+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6367,"log_dups_size":0,"ondisk_log_size":6367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26050560,"num_objects":6360,"num_object_clones":0,"num_object_copies":12720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2095,"num_objects_unfound":0,"num_objects_dirty":6360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4344,"num_write_kb":17376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1895,"num_bytes_recovered":7761920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2095},{"shards":"2","objects":4263}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6173","reported_seq":6295,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.514301+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:22.514301+0000","last_peered":"2024-09-10T15:54:22.514301+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:22.514301+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":6173,"log_dups_size":0,"ondisk_log_size":6173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25251840,"num_objects":6165,"num_object_clones":0,"num_object_copies":12330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3037,"num_write_kb":12148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2302},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6267","reported_seq":6384,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:24.497029+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:24.497029+0000","last_peered":"2024-09-10T15:54:24.497029+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:24.497029+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6267,"log_dups_size":0,"ondisk_log_size":6267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25649152,"num_objects":6262,"num_object_clones":0,"num_object_copies":12524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4241,"num_write_kb":16964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2430}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6174","reported_seq":6295,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:24.456328+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:24.456328+0000","last_peered":"2024-09-10T15:54:24.456328+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:24.456328+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6174,"log_dups_size":0,"ondisk_log_size":6174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25264128,"num_objects":6168,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4043,"num_write_kb":16172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2339}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6258","reported_seq":6378,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.530333+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:22.530333+0000","last_peered":"2024-09-10T15:54:22.530333+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:22.530333+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6258,"log_dups_size":0,"ondisk_log_size":6258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25624576,"num_objects":6256,"num_object_clones":0,"num_object_copies":12512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3450,"num_write_kb":13800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2423},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6248","reported_seq":6406,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.552641+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:22.552641+0000","last_peered":"2024-09-10T15:54:22.552641+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:22.552641+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6248,"log_dups_size":0,"ondisk_log_size":6248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25559040,"num_objects":6240,"num_object_clones":0,"num_object_copies":12480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2666,"num_write_kb":10664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2323},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6189","reported_seq":6323,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.575668+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:22.575668+0000","last_peered":"2024-09-10T15:54:22.575668+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:22.575668+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6189,"log_dups_size":0,"ondisk_log_size":6189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2795,"num_write_kb":11180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2308},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6358","reported_seq":6491,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:22.496830+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:22.496830+0000","last_peered":"2024-09-10T15:54:22.496830+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:22.496830+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6358,"log_dups_size":0,"ondisk_log_size":6358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25993699,"num_objects":6347,"num_object_clones":0,"num_object_copies":12694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3177,"num_write_kb":12708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2344},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":409219555,"num_objects":99908,"num_object_clones":0,"num_object_copies":199816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2095,"num_objects_unfound":0,"num_objects_dirty":99908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61510,"num_write_kb":246040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1897,"num_bytes_recovered":7770112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":825753600,"data_stored":825663275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99989,"ondisk_log_size":99989,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626447,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":249728,"kb_used_data":119536,"kb_used_omap":4,"kb_used_meta":130171,"kb_avail":94122112,"statfs":{"total":96636764160,"available":96381042688,"internally_reserved":0,"allocated":122404864,"data_stored":122069720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":133295790},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149731,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540772,"kb_used_data":360592,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93831068,"statfs":{"total":96636764160,"available":96083013632,"internally_reserved":0,"allocated":369246208,"data_stored":368990550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184479405},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,1,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513520,"kb_used_data":325024,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93858320,"statfs":{"total":96636764160,"available":96110919680,"internally_reserved":0,"allocated":332824576,"data_stored":332568918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":192999082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":369586176,"data_stored":369582563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122863616,"data_stored":122780517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333303808,"data_stored":333300195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:28.303 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:54:28.303 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'6250", 'reported_seq': 6321, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:22.500210+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:54:22.500210+0000', 'last_peered': '2024-09-10T15:54:22.500210+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:54:22.500210+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 6250, 'log_dups_size': 0, 'ondisk_log_size': 6250, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25600000, 'num_objects': 6250, 'num_object_clones': 0, 'num_object_copies': 12500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6250, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6250, 'num_write_kb': 25000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2277}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:28.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:54:28.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'6211", 'reported_seq': 6329, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:24.501427+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:54:24.501427+0000', 'last_peered': '2024-09-10T15:54:24.501427+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:54:24.501427+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 6211, 'log_dups_size': 0, 'ondisk_log_size': 6211, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25423872, 'num_objects': 6207, 'num_object_clones': 0, 'num_object_copies': 12414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4479, 'num_write_kb': 17916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 2405}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:28.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:54:28.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'6264", 'reported_seq': 6391, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:22.529692+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:54:22.529692+0000', 'last_peered': '2024-09-10T15:54:22.529692+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:54:22.529692+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 6264, 'log_dups_size': 0, 'ondisk_log_size': 6264, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25636864, 'num_objects': 6259, 'num_object_clones': 0, 'num_object_copies': 12518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2982, 'num_write_kb': 11928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2391}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:28.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:54:28.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'6134", 'reported_seq': 6268, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:22.567601+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:54:22.567601+0000', 'last_peered': '2024-09-10T15:54:22.567601+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:54:22.567601+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 6134, 'log_dups_size': 0, 'ondisk_log_size': 6134, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25100288, 'num_objects': 6128, 'num_object_clones': 0, 'num_object_copies': 12256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6128, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2784, 'num_write_kb': 11136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2294}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:28.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:54:28.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'6367", 'reported_seq': 10281, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:24.489679+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:54:24.489679+0000', 'last_peered': '2024-09-10T15:54:24.489679+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:54:24.489679+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 6367, 'log_dups_size': 0, 'ondisk_log_size': 6367, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26050560, 'num_objects': 6360, 'num_object_clones': 0, 'num_object_copies': 12720, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2095, 'num_objects_unfound': 0, 'num_objects_dirty': 6360, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4344, 'num_write_kb': 17376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1895, 'num_bytes_recovered': 7761920, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2095}, {'shards': '2', 'objects': 4263}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:28.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:54:28.304 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'6173", 'reported_seq': 6295, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:22.514301+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:54:22.514301+0000', 'last_peered': '2024-09-10T15:54:22.514301+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:54:22.514301+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 6173, 'log_dups_size': 0, 'ondisk_log_size': 6173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25251840, 'num_objects': 6165, 'num_object_clones': 0, 'num_object_copies': 12330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3037, 'num_write_kb': 12148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2302}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:28.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:54:28.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'6267", 'reported_seq': 6384, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:24.497029+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:54:24.497029+0000', 'last_peered': '2024-09-10T15:54:24.497029+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:54:24.497029+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 6267, 'log_dups_size': 0, 'ondisk_log_size': 6267, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25649152, 'num_objects': 6262, 'num_object_clones': 0, 'num_object_copies': 12524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6262, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4241, 'num_write_kb': 16964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 2430}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:28.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:54:28.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'6174", 'reported_seq': 6295, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:24.456328+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:54:24.456328+0000', 'last_peered': '2024-09-10T15:54:24.456328+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:54:24.456328+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 6174, 'log_dups_size': 0, 'ondisk_log_size': 6174, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25264128, 'num_objects': 6168, 'num_object_clones': 0, 'num_object_copies': 12336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4043, 'num_write_kb': 16172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 2339}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:28.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:54:28.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'6258", 'reported_seq': 6378, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:22.530333+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:54:22.530333+0000', 'last_peered': '2024-09-10T15:54:22.530333+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:54:22.530333+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 6258, 'log_dups_size': 0, 'ondisk_log_size': 6258, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25624576, 'num_objects': 6256, 'num_object_clones': 0, 'num_object_copies': 12512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6256, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3450, 'num_write_kb': 13800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2423}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:28.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:54:28.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'6248", 'reported_seq': 6406, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:22.552641+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:54:22.552641+0000', 'last_peered': '2024-09-10T15:54:22.552641+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:54:22.552641+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 6248, 'log_dups_size': 0, 'ondisk_log_size': 6248, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25559040, 'num_objects': 6240, 'num_object_clones': 0, 'num_object_copies': 12480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6240, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2666, 'num_write_kb': 10664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2323}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:28.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:54:28.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'6189", 'reported_seq': 6323, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:22.575668+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:54:22.575668+0000', 'last_peered': '2024-09-10T15:54:22.575668+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:54:22.575668+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 6189, 'log_dups_size': 0, 'ondisk_log_size': 6189, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25333760, 'num_objects': 6185, 'num_object_clones': 0, 'num_object_copies': 12370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2795, 'num_write_kb': 11180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2308}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:28.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:54:28.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'6358", 'reported_seq': 6491, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:22.496830+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:54:22.496830+0000', 'last_peered': '2024-09-10T15:54:22.496830+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:54:22.496830+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 6358, 'log_dups_size': 0, 'ondisk_log_size': 6358, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25993699, 'num_objects': 6347, 'num_object_clones': 0, 'num_object_copies': 12694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6347, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3177, 'num_write_kb': 12708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2344}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:28.306 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:54:28.813 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 111 20 102822 102802 3.6172 1.66016 0.0257339 0.0215916 2024-09-10T15:54:28.855 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:28.855 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/199820 objects degraded (21.346%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":194,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":99910,"data_bytes":409219574,"bytes_used":1335316480,"bytes_avail":288574976000,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":199820,"degraded_ratio":0.21345711140026025,"misplaced_objects":2095,"misplaced_total":199820,"misplaced_ratio":0.010484435992393152,"recovering_objects_per_sec":42,"recovering_bytes_per_sec":172674,"recovering_keys_per_sec":0,"num_objects_recovered":506,"num_bytes_recovered":2072576,"num_keys_recovered":0,"write_bytes_sec":2266607,"read_op_per_sec":0,"write_op_per_sec":553},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T15:54:18.459081+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:54:28.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:54:28.882 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:29.813 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 112 20 103358 103338 3.6036 2.09375 0.0192024 0.0216723 2024-09-10T15:54:30.105 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:30.105 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:30.126 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-10T15:54:28.461383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":416362998,"num_objects":101654,"num_object_clones":0,"num_object_copies":203308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2095,"num_objects_unfound":0,"num_objects_dirty":101654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63256,"num_write_kb":253018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1899,"num_bytes_recovered":7770131,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101736,"ondisk_log_size":101736,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1312948,"kb_used_data":822404,"kb_used_omap":12,"kb_used_meta":490483,"kb_avail":281802572,"statfs":{"total":289910292480,"available":288565833728,"internally_reserved":0,"allocated":842141696,"data_stored":841302462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":502254597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,10,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29749248,"num_objects":7263,"num_object_clones":0,"num_object_copies":14526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-362,"num_objects_unfound":0,"num_objects_dirty":7263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7263,"num_write_kb":29052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":1482752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002797"},"pg_stats":[{"pgid":"2.f","version":"44'6417","reported_seq":6489,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.555205+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:27.555205+0000","last_peered":"2024-09-10T15:54:27.555205+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:27.555205+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6417,"log_dups_size":0,"ondisk_log_size":6417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26284032,"num_objects":6417,"num_object_clones":0,"num_object_copies":12834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6417,"num_write_kb":25668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2444},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6477","reported_seq":6579,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:24.496495+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:24.496495+0000","last_peered":"2024-09-10T15:54:24.496495+0000","last_clean":"2024-09-10T15:54:24.496495+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:24.496495+0000","last_undegraded":"2024-09-10T15:54:24.496495+0000","last_fullsized":"2024-09-10T15:54:24.496495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6477,"log_dups_size":0,"ondisk_log_size":6477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26505216,"num_objects":6471,"num_object_clones":0,"num_object_copies":12942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4813,"num_write_kb":19252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6211","reported_seq":6329,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:24.501427+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:24.501427+0000","last_peered":"2024-09-10T15:54:24.501427+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:24.501427+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25423872,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4479,"num_write_kb":17916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2405}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6220","reported_seq":6294,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:24.489940+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:24.489940+0000","last_peered":"2024-09-10T15:54:24.489940+0000","last_clean":"2024-09-10T15:54:24.489940+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:24.489940+0000","last_undegraded":"2024-09-10T15:54:24.489940+0000","last_fullsized":"2024-09-10T15:54:24.489940+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6220,"log_dups_size":0,"ondisk_log_size":6220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25477120,"num_objects":6220,"num_object_clones":0,"num_object_copies":12440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6220,"num_write_kb":24880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6277","reported_seq":6399,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:27.531785+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:27.531785+0000","last_peered":"2024-09-10T15:54:27.531785+0000","last_clean":"2024-09-10T15:54:27.531785+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:27.531785+0000","last_undegraded":"2024-09-10T15:54:27.531785+0000","last_fullsized":"2024-09-10T15:54:27.531785+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6277,"log_dups_size":0,"ondisk_log_size":6277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3348,"num_write_kb":13392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6450","reported_seq":6579,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.597763+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:27.597763+0000","last_peered":"2024-09-10T15:54:27.597763+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:27.597763+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6450,"log_dups_size":0,"ondisk_log_size":6450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26398720,"num_objects":6445,"num_object_clones":0,"num_object_copies":12890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3168,"num_write_kb":12672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2577},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6299","reported_seq":6434,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.542670+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:27.542670+0000","last_peered":"2024-09-10T15:54:27.542670+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:27.542670+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6299,"log_dups_size":0,"ondisk_log_size":6299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25776128,"num_objects":6293,"num_object_clones":0,"num_object_copies":12586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2949,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2459},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6452","reported_seq":6575,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:27.555592+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:27.555592+0000","last_peered":"2024-09-10T15:54:27.555592+0000","last_clean":"2024-09-10T15:54:27.555592+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:27.555592+0000","last_undegraded":"2024-09-10T15:54:27.555592+0000","last_fullsized":"2024-09-10T15:54:27.555592+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6452,"log_dups_size":0,"ondisk_log_size":6452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26419200,"num_objects":6450,"num_object_clones":0,"num_object_copies":12900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3211,"num_write_kb":12844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6367","reported_seq":10281,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:24.489679+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:24.489679+0000","last_peered":"2024-09-10T15:54:24.489679+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:24.489679+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6367,"log_dups_size":0,"ondisk_log_size":6367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26050560,"num_objects":6360,"num_object_clones":0,"num_object_copies":12720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2095,"num_objects_unfound":0,"num_objects_dirty":6360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4344,"num_write_kb":17376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1895,"num_bytes_recovered":7761920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2095},{"shards":"2","objects":4263}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470670+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470670+0000","last_peered":"2024-09-10T15:53:19.470670+0000","last_clean":"2024-09-10T15:53:19.470670+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470670+0000","last_undegraded":"2024-09-10T15:53:19.470670+0000","last_fullsized":"2024-09-10T15:53:19.470670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6348","reported_seq":6470,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.554273+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:27.554273+0000","last_peered":"2024-09-10T15:54:27.554273+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:27.554273+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":6348,"log_dups_size":0,"ondisk_log_size":6348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25968640,"num_objects":6340,"num_object_clones":0,"num_object_copies":12680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3212,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2477},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6267","reported_seq":6384,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:24.497029+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:24.497029+0000","last_peered":"2024-09-10T15:54:24.497029+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:24.497029+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6267,"log_dups_size":0,"ondisk_log_size":6267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25649152,"num_objects":6262,"num_object_clones":0,"num_object_copies":12524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4241,"num_write_kb":16964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2430}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6174","reported_seq":6295,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:24.456328+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:24.456328+0000","last_peered":"2024-09-10T15:54:24.456328+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:24.456328+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6174,"log_dups_size":0,"ondisk_log_size":6174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25264128,"num_objects":6168,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4043,"num_write_kb":16172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2339}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6431","reported_seq":6553,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.486994+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:27.486994+0000","last_peered":"2024-09-10T15:54:27.486994+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:27.486994+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6431,"log_dups_size":0,"ondisk_log_size":6431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3623,"num_write_kb":14492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2596},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6426","reported_seq":6582,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.613075+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:27.613075+0000","last_peered":"2024-09-10T15:54:27.613075+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:27.613075+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6426,"log_dups_size":0,"ondisk_log_size":6426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26288128,"num_objects":6418,"num_object_clones":0,"num_object_copies":12836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2844,"num_write_kb":11376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2501},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469281+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469281+0000","last_peered":"2024-09-10T15:53:19.469281+0000","last_clean":"2024-09-10T15:53:19.469281+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469281+0000","last_undegraded":"2024-09-10T15:53:19.469281+0000","last_fullsized":"2024-09-10T15:53:19.469281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6374","reported_seq":6508,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.602999+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:27.602999+0000","last_peered":"2024-09-10T15:54:27.602999+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:27.602999+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6374,"log_dups_size":0,"ondisk_log_size":6374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26091520,"num_objects":6370,"num_object_clones":0,"num_object_copies":12740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2980,"num_write_kb":11920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2493},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6543","reported_seq":6677,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.606036+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:27.606036+0000","last_peered":"2024-09-10T15:54:27.606036+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:27.606036+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6543,"log_dups_size":0,"ondisk_log_size":6543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26751459,"num_objects":6532,"num_object_clones":0,"num_object_copies":13064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3362,"num_write_kb":13448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2529},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":416362979,"num_objects":101652,"num_object_clones":0,"num_object_copies":203304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":2095,"num_objects_unfound":0,"num_objects_dirty":101652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63254,"num_write_kb":253016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1897,"num_bytes_recovered":7770112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":843460608,"data_stored":843377509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101733,"ondisk_log_size":101733,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626448,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246140,"kb_used_data":128428,"kb_used_omap":4,"kb_used_meta":117691,"kb_avail":94125700,"statfs":{"total":96636764160,"available":96384716800,"internally_reserved":0,"allocated":131510272,"data_stored":131182354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":120516270},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149732,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553288,"kb_used_data":368952,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93818552,"statfs":{"total":96636764160,"available":96070197248,"internally_reserved":0,"allocated":377806848,"data_stored":377551190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188739245},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,6,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513520,"kb_used_data":325024,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93858320,"statfs":{"total":96636764160,"available":96110919680,"internally_reserved":0,"allocated":332824576,"data_stored":332568918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":192999082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":378167296,"data_stored":378163683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131989504,"data_stored":131913631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333303808,"data_stored":333300195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:30.813 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 113 20 103935 103915 3.59165 2.25391 0.0168977 0.021742 2024-09-10T15:54:31.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 114 20 104504 104484 3.57964 2.22266 0.0226461 0.0218157 2024-09-10T15:54:32.813 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 115 20 105079 105059 3.56804 2.24609 0.0482842 0.0218866 2024-09-10T15:54:33.128 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:33.718 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:33.721 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:33.745 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-10T15:54:32.462358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":420307446,"num_objects":102617,"num_object_clones":0,"num_object_copies":205234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1926,"num_objects_unfound":0,"num_objects_dirty":102617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64219,"num_write_kb":256870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2070,"num_bytes_recovered":8462355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102699,"ondisk_log_size":102699,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1305580,"kb_used_data":827516,"kb_used_omap":12,"kb_used_meta":478003,"kb_avail":281809940,"statfs":{"total":289910292480,"available":288573378560,"internally_reserved":0,"allocated":847376384,"data_stored":846537150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":489475077},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,7,11,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22016000,"num_objects":5375,"num_object_clones":0,"num_object_copies":10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-363,"num_objects_unfound":0,"num_objects_dirty":5375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5375,"num_write_kb":21500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":1486848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002941"},"pg_stats":[{"pgid":"2.f","version":"44'6417","reported_seq":6489,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.555205+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:27.555205+0000","last_peered":"2024-09-10T15:54:27.555205+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:27.555205+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6417,"log_dups_size":0,"ondisk_log_size":6417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26284032,"num_objects":6417,"num_object_clones":0,"num_object_copies":12834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6417,"num_write_kb":25668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2444},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6653","reported_seq":6755,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:29.491006+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:29.491006+0000","last_peered":"2024-09-10T15:54:29.491006+0000","last_clean":"2024-09-10T15:54:29.491006+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:29.491006+0000","last_undegraded":"2024-09-10T15:54:29.491006+0000","last_fullsized":"2024-09-10T15:54:29.491006+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6653,"log_dups_size":0,"ondisk_log_size":6653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27226112,"num_objects":6647,"num_object_clones":0,"num_object_copies":13294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":19956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6387","reported_seq":6505,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:29.457467+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:29.457467+0000","last_peered":"2024-09-10T15:54:29.457467+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:29.457467+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6387,"log_dups_size":0,"ondisk_log_size":6387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26144768,"num_objects":6383,"num_object_clones":0,"num_object_copies":12766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2581}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6380","reported_seq":6454,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:29.453360+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:29.453360+0000","last_peered":"2024-09-10T15:54:29.453360+0000","last_clean":"2024-09-10T15:54:29.453360+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:29.453360+0000","last_undegraded":"2024-09-10T15:54:29.453360+0000","last_fullsized":"2024-09-10T15:54:29.453360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6380,"log_dups_size":0,"ondisk_log_size":6380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26132480,"num_objects":6380,"num_object_clones":0,"num_object_copies":12760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6380,"num_write_kb":25520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6277","reported_seq":6399,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:27.531785+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:27.531785+0000","last_peered":"2024-09-10T15:54:27.531785+0000","last_clean":"2024-09-10T15:54:27.531785+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:27.531785+0000","last_undegraded":"2024-09-10T15:54:27.531785+0000","last_fullsized":"2024-09-10T15:54:27.531785+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6277,"log_dups_size":0,"ondisk_log_size":6277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3348,"num_write_kb":13392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6450","reported_seq":6579,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.597763+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:27.597763+0000","last_peered":"2024-09-10T15:54:27.597763+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:27.597763+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6450,"log_dups_size":0,"ondisk_log_size":6450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26398720,"num_objects":6445,"num_object_clones":0,"num_object_copies":12890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3168,"num_write_kb":12672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2577},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6299","reported_seq":6434,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.542670+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:27.542670+0000","last_peered":"2024-09-10T15:54:27.542670+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:27.542670+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6299,"log_dups_size":0,"ondisk_log_size":6299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25776128,"num_objects":6293,"num_object_clones":0,"num_object_copies":12586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2949,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2459},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6452","reported_seq":6575,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:27.555592+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:27.555592+0000","last_peered":"2024-09-10T15:54:27.555592+0000","last_clean":"2024-09-10T15:54:27.555592+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:27.555592+0000","last_undegraded":"2024-09-10T15:54:27.555592+0000","last_fullsized":"2024-09-10T15:54:27.555592+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6452,"log_dups_size":0,"ondisk_log_size":6452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26419200,"num_objects":6450,"num_object_clones":0,"num_object_copies":12900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3211,"num_write_kb":12844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6528","reported_seq":10784,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:29.526359+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:29.526359+0000","last_peered":"2024-09-10T15:54:29.526359+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:29.526359+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6528,"log_dups_size":0,"ondisk_log_size":6528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710016,"num_objects":6521,"num_object_clones":0,"num_object_copies":13042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1926,"num_objects_unfound":0,"num_objects_dirty":6521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4505,"num_write_kb":18020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2066,"num_bytes_recovered":8454144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1926},{"shards":"2","objects":4593}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6348","reported_seq":6470,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.554273+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:27.554273+0000","last_peered":"2024-09-10T15:54:27.554273+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:27.554273+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":6348,"log_dups_size":0,"ondisk_log_size":6348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25968640,"num_objects":6340,"num_object_clones":0,"num_object_copies":12680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3212,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2477},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6413","reported_seq":6532,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:29.522081+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:29.522081+0000","last_peered":"2024-09-10T15:54:29.522081+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:29.522081+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6413,"log_dups_size":0,"ondisk_log_size":6413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247168,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4387,"num_write_kb":17548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2576}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6318","reported_seq":6439,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:29.486330+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:29.486330+0000","last_peered":"2024-09-10T15:54:29.486330+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:29.486330+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6318,"log_dups_size":0,"ondisk_log_size":6318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25853952,"num_objects":6312,"num_object_clones":0,"num_object_copies":12624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4187,"num_write_kb":16748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2483}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6431","reported_seq":6553,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.486994+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:27.486994+0000","last_peered":"2024-09-10T15:54:27.486994+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:27.486994+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6431,"log_dups_size":0,"ondisk_log_size":6431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3623,"num_write_kb":14492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2596},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6426","reported_seq":6582,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.613075+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:27.613075+0000","last_peered":"2024-09-10T15:54:27.613075+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:27.613075+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6426,"log_dups_size":0,"ondisk_log_size":6426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26288128,"num_objects":6418,"num_object_clones":0,"num_object_copies":12836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2844,"num_write_kb":11376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2501},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6374","reported_seq":6508,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.602999+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:27.602999+0000","last_peered":"2024-09-10T15:54:27.602999+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:27.602999+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6374,"log_dups_size":0,"ondisk_log_size":6374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26091520,"num_objects":6370,"num_object_clones":0,"num_object_copies":12740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2980,"num_write_kb":11920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2493},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6543","reported_seq":6677,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.606036+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:27.606036+0000","last_peered":"2024-09-10T15:54:27.606036+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:27.606036+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6543,"log_dups_size":0,"ondisk_log_size":6543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26751459,"num_objects":6532,"num_object_clones":0,"num_object_copies":13064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3362,"num_write_kb":13448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2529},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":420307427,"num_objects":102615,"num_object_clones":0,"num_object_copies":205230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1926,"num_objects_unfound":0,"num_objects_dirty":102615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64217,"num_write_kb":256868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2068,"num_bytes_recovered":8462336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":848707584,"data_stored":848624485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102696,"ondisk_log_size":102696,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626448,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246140,"kb_used_data":128428,"kb_used_omap":4,"kb_used_meta":117691,"kb_avail":94125700,"statfs":{"total":96636764160,"available":96384716800,"internally_reserved":0,"allocated":131510272,"data_stored":131182354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":120516270},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149732,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553288,"kb_used_data":368952,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93818552,"statfs":{"total":96636764160,"available":96070197248,"internally_reserved":0,"allocated":377806848,"data_stored":377551190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188739245},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,6,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506152,"kb_used_data":330136,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93865688,"statfs":{"total":96636764160,"available":96118464512,"internally_reserved":0,"allocated":338059264,"data_stored":337803606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":378167296,"data_stored":378163683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131989504,"data_stored":131913631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338550784,"data_stored":338547171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:33.746 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:33.813 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 116 20 105618 105598 3.55544 2.10547 0.0328482 0.0219646 2024-09-10T15:54:34.316 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:34.317 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:34.339 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-10T15:54:32.462358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":420307446,"num_objects":102617,"num_object_clones":0,"num_object_copies":205234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1926,"num_objects_unfound":0,"num_objects_dirty":102617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64219,"num_write_kb":256870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2070,"num_bytes_recovered":8462355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102699,"ondisk_log_size":102699,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1305580,"kb_used_data":827516,"kb_used_omap":12,"kb_used_meta":478003,"kb_avail":281809940,"statfs":{"total":289910292480,"available":288573378560,"internally_reserved":0,"allocated":847376384,"data_stored":846537150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":489475077},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,7,11,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22016000,"num_objects":5375,"num_object_clones":0,"num_object_copies":10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-363,"num_objects_unfound":0,"num_objects_dirty":5375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5375,"num_write_kb":21500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":1486848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002941"},"pg_stats":[{"pgid":"2.f","version":"44'6417","reported_seq":6489,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.555205+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:27.555205+0000","last_peered":"2024-09-10T15:54:27.555205+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:27.555205+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6417,"log_dups_size":0,"ondisk_log_size":6417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26284032,"num_objects":6417,"num_object_clones":0,"num_object_copies":12834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6417,"num_write_kb":25668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2444},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6653","reported_seq":6755,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:29.491006+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:29.491006+0000","last_peered":"2024-09-10T15:54:29.491006+0000","last_clean":"2024-09-10T15:54:29.491006+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:29.491006+0000","last_undegraded":"2024-09-10T15:54:29.491006+0000","last_fullsized":"2024-09-10T15:54:29.491006+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6653,"log_dups_size":0,"ondisk_log_size":6653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27226112,"num_objects":6647,"num_object_clones":0,"num_object_copies":13294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":19956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6387","reported_seq":6505,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:29.457467+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:29.457467+0000","last_peered":"2024-09-10T15:54:29.457467+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:29.457467+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6387,"log_dups_size":0,"ondisk_log_size":6387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26144768,"num_objects":6383,"num_object_clones":0,"num_object_copies":12766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2581}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6380","reported_seq":6454,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:29.453360+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:29.453360+0000","last_peered":"2024-09-10T15:54:29.453360+0000","last_clean":"2024-09-10T15:54:29.453360+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:29.453360+0000","last_undegraded":"2024-09-10T15:54:29.453360+0000","last_fullsized":"2024-09-10T15:54:29.453360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6380,"log_dups_size":0,"ondisk_log_size":6380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26132480,"num_objects":6380,"num_object_clones":0,"num_object_copies":12760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6380,"num_write_kb":25520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6277","reported_seq":6399,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:27.531785+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:27.531785+0000","last_peered":"2024-09-10T15:54:27.531785+0000","last_clean":"2024-09-10T15:54:27.531785+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:27.531785+0000","last_undegraded":"2024-09-10T15:54:27.531785+0000","last_fullsized":"2024-09-10T15:54:27.531785+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6277,"log_dups_size":0,"ondisk_log_size":6277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3348,"num_write_kb":13392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6450","reported_seq":6579,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.597763+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:27.597763+0000","last_peered":"2024-09-10T15:54:27.597763+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:27.597763+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6450,"log_dups_size":0,"ondisk_log_size":6450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26398720,"num_objects":6445,"num_object_clones":0,"num_object_copies":12890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3168,"num_write_kb":12672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2577},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6299","reported_seq":6434,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.542670+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:27.542670+0000","last_peered":"2024-09-10T15:54:27.542670+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:27.542670+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6299,"log_dups_size":0,"ondisk_log_size":6299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25776128,"num_objects":6293,"num_object_clones":0,"num_object_copies":12586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2949,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2459},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6452","reported_seq":6575,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:27.555592+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:27.555592+0000","last_peered":"2024-09-10T15:54:27.555592+0000","last_clean":"2024-09-10T15:54:27.555592+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:27.555592+0000","last_undegraded":"2024-09-10T15:54:27.555592+0000","last_fullsized":"2024-09-10T15:54:27.555592+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6452,"log_dups_size":0,"ondisk_log_size":6452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26419200,"num_objects":6450,"num_object_clones":0,"num_object_copies":12900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3211,"num_write_kb":12844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6528","reported_seq":10784,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:29.526359+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:29.526359+0000","last_peered":"2024-09-10T15:54:29.526359+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:29.526359+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6528,"log_dups_size":0,"ondisk_log_size":6528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710016,"num_objects":6521,"num_object_clones":0,"num_object_copies":13042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1926,"num_objects_unfound":0,"num_objects_dirty":6521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4505,"num_write_kb":18020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2066,"num_bytes_recovered":8454144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1926},{"shards":"2","objects":4593}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6348","reported_seq":6470,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.554273+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:27.554273+0000","last_peered":"2024-09-10T15:54:27.554273+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:27.554273+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":6348,"log_dups_size":0,"ondisk_log_size":6348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25968640,"num_objects":6340,"num_object_clones":0,"num_object_copies":12680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3212,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2477},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6413","reported_seq":6532,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:29.522081+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:29.522081+0000","last_peered":"2024-09-10T15:54:29.522081+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:29.522081+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6413,"log_dups_size":0,"ondisk_log_size":6413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247168,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4387,"num_write_kb":17548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2576}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6318","reported_seq":6439,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:29.486330+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:29.486330+0000","last_peered":"2024-09-10T15:54:29.486330+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:29.486330+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6318,"log_dups_size":0,"ondisk_log_size":6318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25853952,"num_objects":6312,"num_object_clones":0,"num_object_copies":12624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4187,"num_write_kb":16748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2483}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6431","reported_seq":6553,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.486994+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:27.486994+0000","last_peered":"2024-09-10T15:54:27.486994+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:27.486994+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6431,"log_dups_size":0,"ondisk_log_size":6431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3623,"num_write_kb":14492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2596},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6426","reported_seq":6582,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.613075+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:27.613075+0000","last_peered":"2024-09-10T15:54:27.613075+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:27.613075+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6426,"log_dups_size":0,"ondisk_log_size":6426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26288128,"num_objects":6418,"num_object_clones":0,"num_object_copies":12836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2844,"num_write_kb":11376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2501},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6374","reported_seq":6508,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.602999+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:27.602999+0000","last_peered":"2024-09-10T15:54:27.602999+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:27.602999+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6374,"log_dups_size":0,"ondisk_log_size":6374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26091520,"num_objects":6370,"num_object_clones":0,"num_object_copies":12740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2980,"num_write_kb":11920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2493},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6543","reported_seq":6677,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:27.606036+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:27.606036+0000","last_peered":"2024-09-10T15:54:27.606036+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:27.606036+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6543,"log_dups_size":0,"ondisk_log_size":6543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26751459,"num_objects":6532,"num_object_clones":0,"num_object_copies":13064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3362,"num_write_kb":13448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2529},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":420307427,"num_objects":102615,"num_object_clones":0,"num_object_copies":205230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1926,"num_objects_unfound":0,"num_objects_dirty":102615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64217,"num_write_kb":256868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2068,"num_bytes_recovered":8462336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":848707584,"data_stored":848624485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102696,"ondisk_log_size":102696,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626448,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246140,"kb_used_data":128428,"kb_used_omap":4,"kb_used_meta":117691,"kb_avail":94125700,"statfs":{"total":96636764160,"available":96384716800,"internally_reserved":0,"allocated":131510272,"data_stored":131182354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":120516270},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149732,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553288,"kb_used_data":368952,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93818552,"statfs":{"total":96636764160,"available":96070197248,"internally_reserved":0,"allocated":377806848,"data_stored":377551190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188739245},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,6,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506152,"kb_used_data":330136,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93865688,"statfs":{"total":96636764160,"available":96118464512,"internally_reserved":0,"allocated":338059264,"data_stored":337803606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":378167296,"data_stored":378163683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131989504,"data_stored":131913631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338550784,"data_stored":338547171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:34.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:54:34.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'6417", 'reported_seq': 6489, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:27.555205+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:54:27.555205+0000', 'last_peered': '2024-09-10T15:54:27.555205+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:54:27.555205+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 6417, 'log_dups_size': 0, 'ondisk_log_size': 6417, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26284032, 'num_objects': 6417, 'num_object_clones': 0, 'num_object_copies': 12834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6417, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6417, 'num_write_kb': 25668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2444}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:34.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:54:34.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'6387", 'reported_seq': 6505, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:29.457467+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:54:29.457467+0000', 'last_peered': '2024-09-10T15:54:29.457467+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:54:29.457467+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 6387, 'log_dups_size': 0, 'ondisk_log_size': 6387, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26144768, 'num_objects': 6383, 'num_object_clones': 0, 'num_object_copies': 12766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6383, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4655, 'num_write_kb': 18620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 2581}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:34.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:54:34.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'6450", 'reported_seq': 6579, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:27.597763+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:54:27.597763+0000', 'last_peered': '2024-09-10T15:54:27.597763+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:54:27.597763+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 6450, 'log_dups_size': 0, 'ondisk_log_size': 6450, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26398720, 'num_objects': 6445, 'num_object_clones': 0, 'num_object_copies': 12890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6445, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3168, 'num_write_kb': 12672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2577}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:34.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:54:34.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'6299", 'reported_seq': 6434, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:27.542670+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:54:27.542670+0000', 'last_peered': '2024-09-10T15:54:27.542670+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:54:27.542670+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 6299, 'log_dups_size': 0, 'ondisk_log_size': 6299, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25776128, 'num_objects': 6293, 'num_object_clones': 0, 'num_object_copies': 12586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6293, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2949, 'num_write_kb': 11796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2459}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:34.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:54:34.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'6528", 'reported_seq': 10784, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:29.526359+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:54:29.526359+0000', 'last_peered': '2024-09-10T15:54:29.526359+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:54:29.526359+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 6528, 'log_dups_size': 0, 'ondisk_log_size': 6528, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26710016, 'num_objects': 6521, 'num_object_clones': 0, 'num_object_copies': 13042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1926, 'num_objects_unfound': 0, 'num_objects_dirty': 6521, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4505, 'num_write_kb': 18020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2066, 'num_bytes_recovered': 8454144, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1926}, {'shards': '2', 'objects': 4593}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:34.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:54:34.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'6348", 'reported_seq': 6470, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:27.554273+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:54:27.554273+0000', 'last_peered': '2024-09-10T15:54:27.554273+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:54:27.554273+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 6348, 'log_dups_size': 0, 'ondisk_log_size': 6348, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25968640, 'num_objects': 6340, 'num_object_clones': 0, 'num_object_copies': 12680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6340, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3212, 'num_write_kb': 12848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2477}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:34.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:54:34.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'6413", 'reported_seq': 6532, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:29.522081+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:54:29.522081+0000', 'last_peered': '2024-09-10T15:54:29.522081+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:54:29.522081+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 6413, 'log_dups_size': 0, 'ondisk_log_size': 6413, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26247168, 'num_objects': 6408, 'num_object_clones': 0, 'num_object_copies': 12816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6408, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4387, 'num_write_kb': 17548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 2576}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:34.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:54:34.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'6318", 'reported_seq': 6439, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:29.486330+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:54:29.486330+0000', 'last_peered': '2024-09-10T15:54:29.486330+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:54:29.486330+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 6318, 'log_dups_size': 0, 'ondisk_log_size': 6318, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25853952, 'num_objects': 6312, 'num_object_clones': 0, 'num_object_copies': 12624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6312, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4187, 'num_write_kb': 16748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 2483}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:34.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:54:34.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'6431", 'reported_seq': 6553, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:27.486994+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:54:27.486994+0000', 'last_peered': '2024-09-10T15:54:27.486994+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:54:27.486994+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 6431, 'log_dups_size': 0, 'ondisk_log_size': 6431, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26333184, 'num_objects': 6429, 'num_object_clones': 0, 'num_object_copies': 12858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3623, 'num_write_kb': 14492, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2596}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:34.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:54:34.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'6426", 'reported_seq': 6582, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:27.613075+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:54:27.613075+0000', 'last_peered': '2024-09-10T15:54:27.613075+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:54:27.613075+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 6426, 'log_dups_size': 0, 'ondisk_log_size': 6426, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26288128, 'num_objects': 6418, 'num_object_clones': 0, 'num_object_copies': 12836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6418, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2844, 'num_write_kb': 11376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2501}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:34.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:54:34.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'6374", 'reported_seq': 6508, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:27.602999+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:54:27.602999+0000', 'last_peered': '2024-09-10T15:54:27.602999+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:54:27.602999+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 6374, 'log_dups_size': 0, 'ondisk_log_size': 6374, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26091520, 'num_objects': 6370, 'num_object_clones': 0, 'num_object_copies': 12740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6370, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2980, 'num_write_kb': 11920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2493}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:34.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:54:34.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'6543", 'reported_seq': 6677, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:27.606036+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:54:27.606036+0000', 'last_peered': '2024-09-10T15:54:27.606036+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:54:27.606036+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 6543, 'log_dups_size': 0, 'ondisk_log_size': 6543, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26751459, 'num_objects': 6532, 'num_object_clones': 0, 'num_object_copies': 13064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6532, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3362, 'num_write_kb': 13448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2529}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:34.344 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:54:34.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 117 20 106118 106098 3.54174 1.95312 0.0260895 0.0220479 2024-09-10T15:54:35.014 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:35.015 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/205234 objects degraded (20.783%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":200,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":102617,"data_bytes":420307446,"bytes_used":1336913920,"bytes_avail":288573378560,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":205234,"degraded_ratio":0.20782618864320723,"misplaced_objects":1926,"misplaced_total":205234,"misplaced_ratio":0.0093844099905473748,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":123873,"recovering_keys_per_sec":0,"num_objects_recovered":365,"num_bytes_recovered":1486848,"num_keys_recovered":0,"write_bytes_sec":1834217,"read_op_per_sec":0,"write_op_per_sec":447},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-10T15:54:30.461931+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:54:35.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:54:35.036 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:35.609 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:35.611 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:35.635 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-10T15:54:34.462821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":427332086,"num_objects":104332,"num_object_clones":0,"num_object_copies":208664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1926,"num_objects_unfound":0,"num_objects_dirty":104332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65934,"num_write_kb":263730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2070,"num_bytes_recovered":8462355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104414,"ondisk_log_size":104414,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1336316,"kb_used_data":844364,"kb_used_omap":12,"kb_used_meta":491891,"kb_avail":281779204,"statfs":{"total":289910292480,"available":288541904896,"internally_reserved":0,"allocated":864628736,"data_stored":863789502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":503696389},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,5,8,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29040640,"num_objects":7090,"num_object_clones":0,"num_object_copies":14180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-363,"num_objects_unfound":0,"num_objects_dirty":7090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7090,"num_write_kb":28360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":1486848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002920"},"pg_stats":[{"pgid":"2.f","version":"44'6598","reported_seq":6668,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:32.625561+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:32.625561+0000","last_peered":"2024-09-10T15:54:32.625561+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:32.625561+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6598,"log_dups_size":0,"ondisk_log_size":6598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6598,"num_write_kb":26392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2625},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6653","reported_seq":6755,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:29.491006+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:29.491006+0000","last_peered":"2024-09-10T15:54:29.491006+0000","last_clean":"2024-09-10T15:54:29.491006+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:29.491006+0000","last_undegraded":"2024-09-10T15:54:29.491006+0000","last_fullsized":"2024-09-10T15:54:29.491006+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6653,"log_dups_size":0,"ondisk_log_size":6653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27226112,"num_objects":6647,"num_object_clones":0,"num_object_copies":13294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":19956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6387","reported_seq":6505,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:29.457467+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:29.457467+0000","last_peered":"2024-09-10T15:54:29.457467+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:29.457467+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6387,"log_dups_size":0,"ondisk_log_size":6387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26144768,"num_objects":6383,"num_object_clones":0,"num_object_copies":12766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2581}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6380","reported_seq":6454,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:29.453360+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:29.453360+0000","last_peered":"2024-09-10T15:54:29.453360+0000","last_clean":"2024-09-10T15:54:29.453360+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:29.453360+0000","last_undegraded":"2024-09-10T15:54:29.453360+0000","last_fullsized":"2024-09-10T15:54:29.453360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6380,"log_dups_size":0,"ondisk_log_size":6380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26132480,"num_objects":6380,"num_object_clones":0,"num_object_copies":12760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6380,"num_write_kb":25520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6449","reported_seq":6571,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:32.624883+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:32.624883+0000","last_peered":"2024-09-10T15:54:32.624883+0000","last_clean":"2024-09-10T15:54:32.624883+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:32.624883+0000","last_undegraded":"2024-09-10T15:54:32.624883+0000","last_fullsized":"2024-09-10T15:54:32.624883+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6449,"log_dups_size":0,"ondisk_log_size":6449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26386432,"num_objects":6442,"num_object_clones":0,"num_object_copies":12884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3520,"num_write_kb":14080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6623","reported_seq":6750,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:32.621801+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:32.621801+0000","last_peered":"2024-09-10T15:54:32.621801+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:32.621801+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6623,"log_dups_size":0,"ondisk_log_size":6623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27107328,"num_objects":6618,"num_object_clones":0,"num_object_copies":13236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3341,"num_write_kb":13364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2750},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6475","reported_seq":6611,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:32.600136+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:32.600136+0000","last_peered":"2024-09-10T15:54:32.600136+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:32.600136+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6475,"log_dups_size":0,"ondisk_log_size":6475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26497024,"num_objects":6469,"num_object_clones":0,"num_object_copies":12938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3125,"num_write_kb":12500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2635},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6616","reported_seq":6741,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:32.621791+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:32.621791+0000","last_peered":"2024-09-10T15:54:32.621791+0000","last_clean":"2024-09-10T15:54:32.621791+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:32.621791+0000","last_undegraded":"2024-09-10T15:54:32.621791+0000","last_fullsized":"2024-09-10T15:54:32.621791+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6616,"log_dups_size":0,"ondisk_log_size":6616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27090944,"num_objects":6614,"num_object_clones":0,"num_object_copies":13228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3375,"num_write_kb":13500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6528","reported_seq":10784,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:29.526359+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:29.526359+0000","last_peered":"2024-09-10T15:54:29.526359+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:29.526359+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6528,"log_dups_size":0,"ondisk_log_size":6528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710016,"num_objects":6521,"num_object_clones":0,"num_object_copies":13042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1926,"num_objects_unfound":0,"num_objects_dirty":6521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4505,"num_write_kb":18020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2066,"num_bytes_recovered":8454144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1926},{"shards":"2","objects":4593}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6522","reported_seq":6644,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:32.555944+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:32.555944+0000","last_peered":"2024-09-10T15:54:32.555944+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:32.555944+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":6522,"log_dups_size":0,"ondisk_log_size":6522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26681344,"num_objects":6514,"num_object_clones":0,"num_object_copies":13028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3386,"num_write_kb":13544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2651},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6413","reported_seq":6532,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:29.522081+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:29.522081+0000","last_peered":"2024-09-10T15:54:29.522081+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:29.522081+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6413,"log_dups_size":0,"ondisk_log_size":6413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247168,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4387,"num_write_kb":17548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2576}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6318","reported_seq":6439,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:29.486330+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:29.486330+0000","last_peered":"2024-09-10T15:54:29.486330+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:29.486330+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6318,"log_dups_size":0,"ondisk_log_size":6318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25853952,"num_objects":6312,"num_object_clones":0,"num_object_copies":12624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4187,"num_write_kb":16748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2483}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6584","reported_seq":6706,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:32.688482+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:32.688482+0000","last_peered":"2024-09-10T15:54:32.688482+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:32.688482+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6584,"log_dups_size":0,"ondisk_log_size":6584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26959872,"num_objects":6582,"num_object_clones":0,"num_object_copies":13164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3776,"num_write_kb":15104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2749},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6612","reported_seq":6769,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:32.682082+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:32.682082+0000","last_peered":"2024-09-10T15:54:32.682082+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:32.682082+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6612,"log_dups_size":0,"ondisk_log_size":6612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27049984,"num_objects":6604,"num_object_clones":0,"num_object_copies":13208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3030,"num_write_kb":12120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2687},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6550","reported_seq":6685,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:32.556981+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:32.556981+0000","last_peered":"2024-09-10T15:54:32.556981+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:32.556981+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6550,"log_dups_size":0,"ondisk_log_size":6550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26812416,"num_objects":6546,"num_object_clones":0,"num_object_copies":13092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3156,"num_write_kb":12624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2669},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6703","reported_seq":6837,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:32.602551+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:32.602551+0000","last_peered":"2024-09-10T15:54:32.602551+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:32.602551+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6703,"log_dups_size":0,"ondisk_log_size":6703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27406819,"num_objects":6692,"num_object_clones":0,"num_object_copies":13384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3522,"num_write_kb":14088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2689},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":427332067,"num_objects":104330,"num_object_clones":0,"num_object_copies":208660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1926,"num_objects_unfound":0,"num_objects_dirty":104330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65932,"num_write_kb":263728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2068,"num_bytes_recovered":8462336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":866357248,"data_stored":866274149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104411,"ondisk_log_size":104411,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626449,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263204,"kb_used_data":137172,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94108636,"statfs":{"total":96636764160,"available":96367243264,"internally_reserved":0,"allocated":140464128,"data_stored":140136210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":129035950},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149733,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566960,"kb_used_data":377056,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93804880,"statfs":{"total":96636764160,"available":96056197120,"internally_reserved":0,"allocated":386105344,"data_stored":385849686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":194440877},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506152,"kb_used_data":330136,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93865688,"statfs":{"total":96636764160,"available":96118464512,"internally_reserved":0,"allocated":338059264,"data_stored":337803606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":386584576,"data_stored":386580963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":141221888,"data_stored":141146015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338550784,"data_stored":338547171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:35.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 118 20 106653 106633 3.52944 2.08984 0.0348006 0.0221268 2024-09-10T15:54:36.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 119 20 107234 107214 3.51885 2.26953 0.0605486 0.0221946 2024-09-10T15:54:37.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:2024-09-10T15:54:37.813768+0000 min lat: 0.00198913 max lat: 2.66872 avg lat: 0.0222601 2024-09-10T15:54:37.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:54:37.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 120 20 107816 107796 3.50847 2.27344 0.0505058 0.0222601 2024-09-10T15:54:38.637 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:38.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 121 20 108364 108344 3.49716 2.14062 0.0281051 0.0223323 2024-09-10T15:54:39.266 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:39.267 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:39.294 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-10T15:54:38.463820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438563318,"num_objects":107074,"num_object_clones":0,"num_object_copies":214148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1743,"num_objects_unfound":0,"num_objects_dirty":107074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68676,"num_write_kb":274698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2253,"num_bytes_recovered":9211923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107156,"ondisk_log_size":107156,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1411512,"kb_used_data":866824,"kb_used_omap":12,"kb_used_meta":544627,"kb_avail":281704008,"statfs":{"total":289910292480,"available":288464904192,"internally_reserved":0,"allocated":887627776,"data_stored":886788542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":557698053},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,7,12,8,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29343744,"num_objects":7164,"num_object_clones":0,"num_object_copies":14328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-352,"num_objects_unfound":0,"num_objects_dirty":7164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7164,"num_write_kb":28656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":1441792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002916"},"pg_stats":[{"pgid":"2.f","version":"44'6748","reported_seq":6820,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.577415+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:37.577415+0000","last_peered":"2024-09-10T15:54:37.577415+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:37.577415+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6748,"log_dups_size":0,"ondisk_log_size":6748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27639808,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6748,"num_write_kb":26992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2775},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6800","reported_seq":6902,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:34.507673+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:34.507673+0000","last_peered":"2024-09-10T15:54:34.507673+0000","last_clean":"2024-09-10T15:54:34.507673+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:34.507673+0000","last_undegraded":"2024-09-10T15:54:34.507673+0000","last_fullsized":"2024-09-10T15:54:34.507673+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6800,"log_dups_size":0,"ondisk_log_size":6800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27828224,"num_objects":6794,"num_object_clones":0,"num_object_copies":13588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5136,"num_write_kb":20544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6562","reported_seq":6680,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:34.422839+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:34.422839+0000","last_peered":"2024-09-10T15:54:34.422839+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:34.422839+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6562,"log_dups_size":0,"ondisk_log_size":6562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26861568,"num_objects":6558,"num_object_clones":0,"num_object_copies":13116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4830,"num_write_kb":19320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2756}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6554","reported_seq":6628,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:34.513846+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:34.513846+0000","last_peered":"2024-09-10T15:54:34.513846+0000","last_clean":"2024-09-10T15:54:34.513846+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:34.513846+0000","last_undegraded":"2024-09-10T15:54:34.513846+0000","last_fullsized":"2024-09-10T15:54:34.513846+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6554,"log_dups_size":0,"ondisk_log_size":6554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26845184,"num_objects":6554,"num_object_clones":0,"num_object_copies":13108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6554,"num_write_kb":26216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6633","reported_seq":6755,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:37.668670+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:37.668670+0000","last_peered":"2024-09-10T15:54:37.668670+0000","last_clean":"2024-09-10T15:54:37.668670+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:37.668670+0000","last_undegraded":"2024-09-10T15:54:37.668670+0000","last_fullsized":"2024-09-10T15:54:37.668670+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6633,"log_dups_size":0,"ondisk_log_size":6633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27140096,"num_objects":6626,"num_object_clones":0,"num_object_copies":13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3704,"num_write_kb":14816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6809","reported_seq":6938,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.668100+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:37.668100+0000","last_peered":"2024-09-10T15:54:37.668100+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:37.668100+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6809,"log_dups_size":0,"ondisk_log_size":6809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3527,"num_write_kb":14108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2936},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6621","reported_seq":6757,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.634025+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:37.634025+0000","last_peered":"2024-09-10T15:54:37.634025+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:37.634025+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6621,"log_dups_size":0,"ondisk_log_size":6621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27095040,"num_objects":6615,"num_object_clones":0,"num_object_copies":13230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3271,"num_write_kb":13084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2781},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6801","reported_seq":6927,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:37.591338+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:37.591338+0000","last_peered":"2024-09-10T15:54:37.591338+0000","last_clean":"2024-09-10T15:54:37.591338+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:37.591338+0000","last_undegraded":"2024-09-10T15:54:37.591338+0000","last_fullsized":"2024-09-10T15:54:37.591338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6801,"log_dups_size":0,"ondisk_log_size":6801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27848704,"num_objects":6799,"num_object_clones":0,"num_object_copies":13598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3560,"num_write_kb":14240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6683","reported_seq":11305,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:34.521604+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:34.521604+0000","last_peered":"2024-09-10T15:54:34.521604+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:34.521604+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6683,"log_dups_size":0,"ondisk_log_size":6683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27344896,"num_objects":6676,"num_object_clones":0,"num_object_copies":13352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1743,"num_objects_unfound":0,"num_objects_dirty":6676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4660,"num_write_kb":18640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2249,"num_bytes_recovered":9203712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1743},{"shards":"2","objects":4931}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6703","reported_seq":6825,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.605370+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:37.605370+0000","last_peered":"2024-09-10T15:54:37.605370+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:37.605370+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":6703,"log_dups_size":0,"ondisk_log_size":6703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27422720,"num_objects":6695,"num_object_clones":0,"num_object_copies":13390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3567,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2832},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6592","reported_seq":6712,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:34.459996+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:34.459996+0000","last_peered":"2024-09-10T15:54:34.459996+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:34.459996+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6592,"log_dups_size":0,"ondisk_log_size":6592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980352,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4566,"num_write_kb":18264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2755}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6490","reported_seq":6611,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:34.464504+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:34.464504+0000","last_peered":"2024-09-10T15:54:34.464504+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:34.464504+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6490,"log_dups_size":0,"ondisk_log_size":6490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26558464,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4359,"num_write_kb":17436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2655}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6759","reported_seq":6882,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.667226+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:37.667226+0000","last_peered":"2024-09-10T15:54:37.667226+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:37.667226+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6759,"log_dups_size":0,"ondisk_log_size":6759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27676672,"num_objects":6757,"num_object_clones":0,"num_object_copies":13514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3951,"num_write_kb":15804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2924},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6792","reported_seq":6951,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.676224+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:37.676224+0000","last_peered":"2024-09-10T15:54:37.676224+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:37.676224+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6792,"log_dups_size":0,"ondisk_log_size":6792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27787264,"num_objects":6784,"num_object_clones":0,"num_object_copies":13568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3210,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2867},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6727","reported_seq":6862,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.704840+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:37.704840+0000","last_peered":"2024-09-10T15:54:37.704840+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:37.704840+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6727,"log_dups_size":0,"ondisk_log_size":6727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27537408,"num_objects":6723,"num_object_clones":0,"num_object_copies":13446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3333,"num_write_kb":13332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2846},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6879","reported_seq":7013,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.670982+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:37.670982+0000","last_peered":"2024-09-10T15:54:37.670982+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:37.670982+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6879,"log_dups_size":0,"ondisk_log_size":6879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28127715,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3698,"num_write_kb":14792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2865},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":438563299,"num_objects":107072,"num_object_clones":0,"num_object_copies":214144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1743,"num_objects_unfound":0,"num_objects_dirty":107072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68674,"num_write_kb":274696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2251,"num_bytes_recovered":9211904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":889511936,"data_stored":889428837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107153,"ondisk_log_size":107153,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626450,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298408,"kb_used_data":146008,"kb_used_omap":4,"kb_used_meta":152379,"kb_avail":94073432,"statfs":{"total":96636764160,"available":96331194368,"internally_reserved":0,"allocated":149512192,"data_stored":149184274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":156036782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149734,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597332,"kb_used_data":385220,"kb_used_omap":4,"kb_used_meta":212091,"kb_avail":93774508,"statfs":{"total":96636764160,"available":96025096192,"internally_reserved":0,"allocated":394465280,"data_stored":394209622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":217181869},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,6,8,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515772,"kb_used_data":335596,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93856068,"statfs":{"total":96636764160,"available":96108613632,"internally_reserved":0,"allocated":343650304,"data_stored":343394646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395141120,"data_stored":395137507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":150245376,"data_stored":150169503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344125440,"data_stored":344121827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:39.295 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:39.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 122 20 108843 108823 3.48383 1.87109 0.0421226 0.0224176 2024-09-10T15:54:39.843 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:39.843 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:39.869 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-10T15:54:38.463820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438563318,"num_objects":107074,"num_object_clones":0,"num_object_copies":214148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1743,"num_objects_unfound":0,"num_objects_dirty":107074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68676,"num_write_kb":274698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2253,"num_bytes_recovered":9211923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107156,"ondisk_log_size":107156,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1411512,"kb_used_data":866824,"kb_used_omap":12,"kb_used_meta":544627,"kb_avail":281704008,"statfs":{"total":289910292480,"available":288464904192,"internally_reserved":0,"allocated":887627776,"data_stored":886788542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":557698053},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,7,12,8,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29343744,"num_objects":7164,"num_object_clones":0,"num_object_copies":14328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-352,"num_objects_unfound":0,"num_objects_dirty":7164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7164,"num_write_kb":28656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":1441792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002916"},"pg_stats":[{"pgid":"2.f","version":"44'6748","reported_seq":6820,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.577415+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:37.577415+0000","last_peered":"2024-09-10T15:54:37.577415+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:37.577415+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6748,"log_dups_size":0,"ondisk_log_size":6748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27639808,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6748,"num_write_kb":26992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2775},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6800","reported_seq":6902,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:34.507673+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:34.507673+0000","last_peered":"2024-09-10T15:54:34.507673+0000","last_clean":"2024-09-10T15:54:34.507673+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:34.507673+0000","last_undegraded":"2024-09-10T15:54:34.507673+0000","last_fullsized":"2024-09-10T15:54:34.507673+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6800,"log_dups_size":0,"ondisk_log_size":6800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27828224,"num_objects":6794,"num_object_clones":0,"num_object_copies":13588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5136,"num_write_kb":20544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6562","reported_seq":6680,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:34.422839+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:34.422839+0000","last_peered":"2024-09-10T15:54:34.422839+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:34.422839+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6562,"log_dups_size":0,"ondisk_log_size":6562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26861568,"num_objects":6558,"num_object_clones":0,"num_object_copies":13116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4830,"num_write_kb":19320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2756}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6554","reported_seq":6628,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:34.513846+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:34.513846+0000","last_peered":"2024-09-10T15:54:34.513846+0000","last_clean":"2024-09-10T15:54:34.513846+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:34.513846+0000","last_undegraded":"2024-09-10T15:54:34.513846+0000","last_fullsized":"2024-09-10T15:54:34.513846+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6554,"log_dups_size":0,"ondisk_log_size":6554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26845184,"num_objects":6554,"num_object_clones":0,"num_object_copies":13108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6554,"num_write_kb":26216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6633","reported_seq":6755,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:37.668670+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:37.668670+0000","last_peered":"2024-09-10T15:54:37.668670+0000","last_clean":"2024-09-10T15:54:37.668670+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:37.668670+0000","last_undegraded":"2024-09-10T15:54:37.668670+0000","last_fullsized":"2024-09-10T15:54:37.668670+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6633,"log_dups_size":0,"ondisk_log_size":6633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27140096,"num_objects":6626,"num_object_clones":0,"num_object_copies":13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3704,"num_write_kb":14816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6809","reported_seq":6938,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.668100+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:37.668100+0000","last_peered":"2024-09-10T15:54:37.668100+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:37.668100+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6809,"log_dups_size":0,"ondisk_log_size":6809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3527,"num_write_kb":14108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2936},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6621","reported_seq":6757,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.634025+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:37.634025+0000","last_peered":"2024-09-10T15:54:37.634025+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:37.634025+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6621,"log_dups_size":0,"ondisk_log_size":6621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27095040,"num_objects":6615,"num_object_clones":0,"num_object_copies":13230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3271,"num_write_kb":13084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2781},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6801","reported_seq":6927,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:37.591338+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:37.591338+0000","last_peered":"2024-09-10T15:54:37.591338+0000","last_clean":"2024-09-10T15:54:37.591338+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:37.591338+0000","last_undegraded":"2024-09-10T15:54:37.591338+0000","last_fullsized":"2024-09-10T15:54:37.591338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6801,"log_dups_size":0,"ondisk_log_size":6801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27848704,"num_objects":6799,"num_object_clones":0,"num_object_copies":13598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3560,"num_write_kb":14240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6683","reported_seq":11305,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:34.521604+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:34.521604+0000","last_peered":"2024-09-10T15:54:34.521604+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:34.521604+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6683,"log_dups_size":0,"ondisk_log_size":6683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27344896,"num_objects":6676,"num_object_clones":0,"num_object_copies":13352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1743,"num_objects_unfound":0,"num_objects_dirty":6676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4660,"num_write_kb":18640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2249,"num_bytes_recovered":9203712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1743},{"shards":"2","objects":4931}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6703","reported_seq":6825,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.605370+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:37.605370+0000","last_peered":"2024-09-10T15:54:37.605370+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:37.605370+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":6703,"log_dups_size":0,"ondisk_log_size":6703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27422720,"num_objects":6695,"num_object_clones":0,"num_object_copies":13390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3567,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2832},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6592","reported_seq":6712,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:34.459996+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:34.459996+0000","last_peered":"2024-09-10T15:54:34.459996+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:34.459996+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6592,"log_dups_size":0,"ondisk_log_size":6592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980352,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4566,"num_write_kb":18264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2755}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6490","reported_seq":6611,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:34.464504+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:34.464504+0000","last_peered":"2024-09-10T15:54:34.464504+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:34.464504+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6490,"log_dups_size":0,"ondisk_log_size":6490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26558464,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4359,"num_write_kb":17436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2655}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6759","reported_seq":6882,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.667226+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:37.667226+0000","last_peered":"2024-09-10T15:54:37.667226+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:37.667226+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6759,"log_dups_size":0,"ondisk_log_size":6759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27676672,"num_objects":6757,"num_object_clones":0,"num_object_copies":13514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3951,"num_write_kb":15804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2924},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6792","reported_seq":6951,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.676224+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:37.676224+0000","last_peered":"2024-09-10T15:54:37.676224+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:37.676224+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6792,"log_dups_size":0,"ondisk_log_size":6792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27787264,"num_objects":6784,"num_object_clones":0,"num_object_copies":13568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3210,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2867},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6727","reported_seq":6862,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.704840+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:37.704840+0000","last_peered":"2024-09-10T15:54:37.704840+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:37.704840+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6727,"log_dups_size":0,"ondisk_log_size":6727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27537408,"num_objects":6723,"num_object_clones":0,"num_object_copies":13446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3333,"num_write_kb":13332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2846},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6879","reported_seq":7013,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.670982+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:37.670982+0000","last_peered":"2024-09-10T15:54:37.670982+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:37.670982+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6879,"log_dups_size":0,"ondisk_log_size":6879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28127715,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3698,"num_write_kb":14792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2865},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":438563299,"num_objects":107072,"num_object_clones":0,"num_object_copies":214144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1743,"num_objects_unfound":0,"num_objects_dirty":107072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68674,"num_write_kb":274696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2251,"num_bytes_recovered":9211904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":889511936,"data_stored":889428837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107153,"ondisk_log_size":107153,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626450,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298408,"kb_used_data":146008,"kb_used_omap":4,"kb_used_meta":152379,"kb_avail":94073432,"statfs":{"total":96636764160,"available":96331194368,"internally_reserved":0,"allocated":149512192,"data_stored":149184274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":156036782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149734,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597332,"kb_used_data":385220,"kb_used_omap":4,"kb_used_meta":212091,"kb_avail":93774508,"statfs":{"total":96636764160,"available":96025096192,"internally_reserved":0,"allocated":394465280,"data_stored":394209622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":217181869},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,6,8,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515772,"kb_used_data":335596,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93856068,"statfs":{"total":96636764160,"available":96108613632,"internally_reserved":0,"allocated":343650304,"data_stored":343394646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395141120,"data_stored":395137507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":150245376,"data_stored":150169503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344125440,"data_stored":344121827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:39.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:54:39.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'6748", 'reported_seq': 6820, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:37.577415+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:54:37.577415+0000', 'last_peered': '2024-09-10T15:54:37.577415+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:54:37.577415+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 6748, 'log_dups_size': 0, 'ondisk_log_size': 6748, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27639808, 'num_objects': 6748, 'num_object_clones': 0, 'num_object_copies': 13496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6748, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6748, 'num_write_kb': 26992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2775}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:39.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:54:39.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'6562", 'reported_seq': 6680, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:34.422839+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:54:34.422839+0000', 'last_peered': '2024-09-10T15:54:34.422839+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:54:34.422839+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 6562, 'log_dups_size': 0, 'ondisk_log_size': 6562, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26861568, 'num_objects': 6558, 'num_object_clones': 0, 'num_object_copies': 13116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6558, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4830, 'num_write_kb': 19320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 2756}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:39.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:54:39.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'6809", 'reported_seq': 6938, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:37.668100+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:54:37.668100+0000', 'last_peered': '2024-09-10T15:54:37.668100+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:54:37.668100+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 6809, 'log_dups_size': 0, 'ondisk_log_size': 6809, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27869184, 'num_objects': 6804, 'num_object_clones': 0, 'num_object_copies': 13608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3527, 'num_write_kb': 14108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2936}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:39.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:54:39.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'6621", 'reported_seq': 6757, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:37.634025+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:54:37.634025+0000', 'last_peered': '2024-09-10T15:54:37.634025+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:54:37.634025+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 6621, 'log_dups_size': 0, 'ondisk_log_size': 6621, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27095040, 'num_objects': 6615, 'num_object_clones': 0, 'num_object_copies': 13230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6615, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3271, 'num_write_kb': 13084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2781}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:39.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:54:39.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'6683", 'reported_seq': 11305, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:34.521604+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:54:34.521604+0000', 'last_peered': '2024-09-10T15:54:34.521604+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:54:34.521604+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 6683, 'log_dups_size': 0, 'ondisk_log_size': 6683, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27344896, 'num_objects': 6676, 'num_object_clones': 0, 'num_object_copies': 13352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1743, 'num_objects_unfound': 0, 'num_objects_dirty': 6676, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4660, 'num_write_kb': 18640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2249, 'num_bytes_recovered': 9203712, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1743}, {'shards': '2', 'objects': 4931}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:39.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:54:39.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'6703", 'reported_seq': 6825, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:37.605370+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:54:37.605370+0000', 'last_peered': '2024-09-10T15:54:37.605370+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:54:37.605370+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 6703, 'log_dups_size': 0, 'ondisk_log_size': 6703, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27422720, 'num_objects': 6695, 'num_object_clones': 0, 'num_object_copies': 13390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3567, 'num_write_kb': 14268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2832}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:39.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:54:39.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'6592", 'reported_seq': 6712, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:34.459996+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:54:34.459996+0000', 'last_peered': '2024-09-10T15:54:34.459996+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:54:34.459996+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 6592, 'log_dups_size': 0, 'ondisk_log_size': 6592, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26980352, 'num_objects': 6587, 'num_object_clones': 0, 'num_object_copies': 13174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6587, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4566, 'num_write_kb': 18264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 2755}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:39.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:54:39.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'6490", 'reported_seq': 6611, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:34.464504+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:54:34.464504+0000', 'last_peered': '2024-09-10T15:54:34.464504+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:54:34.464504+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 6490, 'log_dups_size': 0, 'ondisk_log_size': 6490, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26558464, 'num_objects': 6484, 'num_object_clones': 0, 'num_object_copies': 12968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6484, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4359, 'num_write_kb': 17436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 2655}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:39.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:54:39.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'6759", 'reported_seq': 6882, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:37.667226+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:54:37.667226+0000', 'last_peered': '2024-09-10T15:54:37.667226+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:54:37.667226+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 6759, 'log_dups_size': 0, 'ondisk_log_size': 6759, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27676672, 'num_objects': 6757, 'num_object_clones': 0, 'num_object_copies': 13514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6757, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3951, 'num_write_kb': 15804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2924}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:39.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:54:39.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'6792", 'reported_seq': 6951, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:37.676224+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:54:37.676224+0000', 'last_peered': '2024-09-10T15:54:37.676224+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:54:37.676224+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 6792, 'log_dups_size': 0, 'ondisk_log_size': 6792, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27787264, 'num_objects': 6784, 'num_object_clones': 0, 'num_object_copies': 13568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6784, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3210, 'num_write_kb': 12840, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2867}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:39.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:54:39.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'6727", 'reported_seq': 6862, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:37.704840+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:54:37.704840+0000', 'last_peered': '2024-09-10T15:54:37.704840+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:54:37.704840+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 6727, 'log_dups_size': 0, 'ondisk_log_size': 6727, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27537408, 'num_objects': 6723, 'num_object_clones': 0, 'num_object_copies': 13446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6723, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3333, 'num_write_kb': 13332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2846}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:39.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:54:39.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'6879", 'reported_seq': 7013, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:37.670982+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:54:37.670982+0000', 'last_peered': '2024-09-10T15:54:37.670982+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:54:37.670982+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 6879, 'log_dups_size': 0, 'ondisk_log_size': 6879, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28127715, 'num_objects': 6868, 'num_object_clones': 0, 'num_object_copies': 13736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3698, 'num_write_kb': 14792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2865}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:39.873 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:54:40.571 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:40.572 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/214148 objects degraded (19.918%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":206,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":107074,"data_bytes":438563318,"bytes_used":1445388288,"bytes_avail":288464904192,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":214148,"degraded_ratio":0.19917533668304163,"misplaced_objects":1743,"misplaced_total":214148,"misplaced_ratio":0.0081392308123353942,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":120120,"recovering_keys_per_sec":0,"num_objects_recovered":354,"num_bytes_recovered":1441792,"num_keys_recovered":0,"write_bytes_sec":2444717,"read_op_per_sec":0,"write_op_per_sec":596},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-10T15:54:30.461931+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:54:40.592 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:54:40.593 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:40.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 123 20 109359 109339 3.4719 2.01562 0.0266227 0.0224953 2024-09-10T15:54:41.148 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:41.149 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:41.175 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-10T15:54:40.464211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":442786294,"num_objects":108105,"num_object_clones":0,"num_object_copies":216210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1582,"num_objects_unfound":0,"num_objects_dirty":108105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69707,"num_write_kb":278822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2414,"num_bytes_recovered":9871379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108187,"ondisk_log_size":108187,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1421232,"kb_used_data":872384,"kb_used_omap":12,"kb_used_meta":548787,"kb_avail":281694288,"statfs":{"total":289910292480,"available":288454950912,"internally_reserved":0,"allocated":893321216,"data_stored":892481982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":561957893},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,15,11,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26423296,"num_objects":6451,"num_object_clones":0,"num_object_copies":12902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-513,"num_objects_unfound":0,"num_objects_dirty":6451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6451,"num_write_kb":25804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":515,"num_bytes_recovered":2101248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002827"},"pg_stats":[{"pgid":"2.f","version":"44'6748","reported_seq":6820,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.577415+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:37.577415+0000","last_peered":"2024-09-10T15:54:37.577415+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:37.577415+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6748,"log_dups_size":0,"ondisk_log_size":6748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27639808,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6748,"num_write_kb":26992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2775},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6977","reported_seq":7079,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:39.473833+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:39.473833+0000","last_peered":"2024-09-10T15:54:39.473833+0000","last_clean":"2024-09-10T15:54:39.473833+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:39.473833+0000","last_undegraded":"2024-09-10T15:54:39.473833+0000","last_fullsized":"2024-09-10T15:54:39.473833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6977,"log_dups_size":0,"ondisk_log_size":6977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28553216,"num_objects":6971,"num_object_clones":0,"num_object_copies":13942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5313,"num_write_kb":21252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6733","reported_seq":6851,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:39.424284+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:39.424284+0000","last_peered":"2024-09-10T15:54:39.424284+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:39.424284+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6733,"log_dups_size":0,"ondisk_log_size":6733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27561984,"num_objects":6729,"num_object_clones":0,"num_object_copies":13458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5001,"num_write_kb":20004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6740","reported_seq":6815,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:39.531950+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:39.531950+0000","last_peered":"2024-09-10T15:54:39.531950+0000","last_clean":"2024-09-10T15:54:39.531950+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:39.531950+0000","last_undegraded":"2024-09-10T15:54:39.531950+0000","last_fullsized":"2024-09-10T15:54:39.531950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6740,"log_dups_size":0,"ondisk_log_size":6740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27607040,"num_objects":6740,"num_object_clones":0,"num_object_copies":13480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6740,"num_write_kb":26960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6633","reported_seq":6755,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:37.668670+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:37.668670+0000","last_peered":"2024-09-10T15:54:37.668670+0000","last_clean":"2024-09-10T15:54:37.668670+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:37.668670+0000","last_undegraded":"2024-09-10T15:54:37.668670+0000","last_fullsized":"2024-09-10T15:54:37.668670+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6633,"log_dups_size":0,"ondisk_log_size":6633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27140096,"num_objects":6626,"num_object_clones":0,"num_object_copies":13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3704,"num_write_kb":14816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6809","reported_seq":6938,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.668100+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:37.668100+0000","last_peered":"2024-09-10T15:54:37.668100+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:37.668100+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6809,"log_dups_size":0,"ondisk_log_size":6809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3527,"num_write_kb":14108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2936},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6621","reported_seq":6757,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.634025+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:37.634025+0000","last_peered":"2024-09-10T15:54:37.634025+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:37.634025+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6621,"log_dups_size":0,"ondisk_log_size":6621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27095040,"num_objects":6615,"num_object_clones":0,"num_object_copies":13230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3271,"num_write_kb":13084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2781},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6801","reported_seq":6927,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:37.591338+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:37.591338+0000","last_peered":"2024-09-10T15:54:37.591338+0000","last_clean":"2024-09-10T15:54:37.591338+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:37.591338+0000","last_undegraded":"2024-09-10T15:54:37.591338+0000","last_fullsized":"2024-09-10T15:54:37.591338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6801,"log_dups_size":0,"ondisk_log_size":6801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27848704,"num_objects":6799,"num_object_clones":0,"num_object_copies":13598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3560,"num_write_kb":14240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6835","reported_seq":11780,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:39.523489+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:39.523489+0000","last_peered":"2024-09-10T15:54:39.523489+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:39.523489+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6835,"log_dups_size":0,"ondisk_log_size":6835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967488,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1582,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4812,"num_write_kb":19248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2410,"num_bytes_recovered":9863168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1582},{"shards":"2","objects":5244}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6703","reported_seq":6825,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.605370+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:37.605370+0000","last_peered":"2024-09-10T15:54:37.605370+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:37.605370+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":6703,"log_dups_size":0,"ondisk_log_size":6703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27422720,"num_objects":6695,"num_object_clones":0,"num_object_copies":13390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3567,"num_write_kb":14268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2832},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6781","reported_seq":6901,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:39.479594+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:39.479594+0000","last_peered":"2024-09-10T15:54:39.479594+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:39.479594+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6781,"log_dups_size":0,"ondisk_log_size":6781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27754496,"num_objects":6776,"num_object_clones":0,"num_object_copies":13552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4755,"num_write_kb":19020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2944}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6646","reported_seq":6767,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:39.518172+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:39.518172+0000","last_peered":"2024-09-10T15:54:39.518172+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:39.518172+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6646,"log_dups_size":0,"ondisk_log_size":6646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27197440,"num_objects":6640,"num_object_clones":0,"num_object_copies":13280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4515,"num_write_kb":18060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2811}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6759","reported_seq":6882,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.667226+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:37.667226+0000","last_peered":"2024-09-10T15:54:37.667226+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:37.667226+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6759,"log_dups_size":0,"ondisk_log_size":6759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27676672,"num_objects":6757,"num_object_clones":0,"num_object_copies":13514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3951,"num_write_kb":15804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2924},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6792","reported_seq":6951,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.676224+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:37.676224+0000","last_peered":"2024-09-10T15:54:37.676224+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:37.676224+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6792,"log_dups_size":0,"ondisk_log_size":6792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27787264,"num_objects":6784,"num_object_clones":0,"num_object_copies":13568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3210,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2867},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6727","reported_seq":6862,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.704840+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:37.704840+0000","last_peered":"2024-09-10T15:54:37.704840+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:37.704840+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6727,"log_dups_size":0,"ondisk_log_size":6727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27537408,"num_objects":6723,"num_object_clones":0,"num_object_copies":13446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3333,"num_write_kb":13332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2846},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6879","reported_seq":7013,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:37.670982+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:37.670982+0000","last_peered":"2024-09-10T15:54:37.670982+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:37.670982+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":6879,"log_dups_size":0,"ondisk_log_size":6879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28127715,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3698,"num_write_kb":14792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2865},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":442786275,"num_objects":108103,"num_object_clones":0,"num_object_copies":216206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1582,"num_objects_unfound":0,"num_objects_dirty":108103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69705,"num_write_kb":278820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2412,"num_bytes_recovered":9871360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":895119360,"data_stored":895036261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108184,"ondisk_log_size":108184,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626450,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298408,"kb_used_data":146008,"kb_used_omap":4,"kb_used_meta":152379,"kb_avail":94073432,"statfs":{"total":96636764160,"available":96331194368,"internally_reserved":0,"allocated":149512192,"data_stored":149184274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":156036782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149734,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597332,"kb_used_data":385220,"kb_used_omap":4,"kb_used_meta":212091,"kb_avail":93774508,"statfs":{"total":96636764160,"available":96025096192,"internally_reserved":0,"allocated":394465280,"data_stored":394209622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":217181869},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,6,8,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525492,"kb_used_data":341156,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93846348,"statfs":{"total":96636764160,"available":96098660352,"internally_reserved":0,"allocated":349343744,"data_stored":349088086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395141120,"data_stored":395137507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":150245376,"data_stored":150169503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":349732864,"data_stored":349729251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:41.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 124 20 109923 109903 3.46166 2.20312 0.0158116 0.0225593 2024-09-10T15:54:42.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 125 20 110490 110470 3.45169 2.21484 0.0185975 0.0226228 2024-09-10T15:54:43.814 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 126 20 111069 111049 3.44224 2.26172 0.0314114 0.0226903 2024-09-10T15:54:44.178 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:44.692 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:44.694 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:44.723 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-10T15:54:44.465169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":449569270,"num_objects":109761,"num_object_clones":0,"num_object_copies":219522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1582,"num_objects_unfound":0,"num_objects_dirty":109761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71363,"num_write_kb":285446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2414,"num_bytes_recovered":9871379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109843,"ondisk_log_size":109843,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488096,"kb_used_data":889264,"kb_used_omap":12,"kb_used_meta":598771,"kb_avail":281627424,"statfs":{"total":289910292480,"available":288386482176,"internally_reserved":0,"allocated":910606336,"data_stored":909767102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":613141509},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,12,10,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29261824,"num_objects":7144,"num_object_clones":0,"num_object_copies":14288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-344,"num_objects_unfound":0,"num_objects_dirty":7144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7144,"num_write_kb":28576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002810"},"pg_stats":[{"pgid":"2.f","version":"44'6925","reported_seq":6997,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.725094+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:42.725094+0000","last_peered":"2024-09-10T15:54:42.725094+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:42.725094+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6925,"log_dups_size":0,"ondisk_log_size":6925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28364800,"num_objects":6925,"num_object_clones":0,"num_object_copies":13850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6925,"num_write_kb":27700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2952},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6977","reported_seq":7079,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:39.473833+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:39.473833+0000","last_peered":"2024-09-10T15:54:39.473833+0000","last_clean":"2024-09-10T15:54:39.473833+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:39.473833+0000","last_undegraded":"2024-09-10T15:54:39.473833+0000","last_fullsized":"2024-09-10T15:54:39.473833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6977,"log_dups_size":0,"ondisk_log_size":6977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28553216,"num_objects":6971,"num_object_clones":0,"num_object_copies":13942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5313,"num_write_kb":21252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6733","reported_seq":6851,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:39.424284+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:39.424284+0000","last_peered":"2024-09-10T15:54:39.424284+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:39.424284+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6733,"log_dups_size":0,"ondisk_log_size":6733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27561984,"num_objects":6729,"num_object_clones":0,"num_object_copies":13458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5001,"num_write_kb":20004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6740","reported_seq":6815,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:39.531950+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:39.531950+0000","last_peered":"2024-09-10T15:54:39.531950+0000","last_clean":"2024-09-10T15:54:39.531950+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:39.531950+0000","last_undegraded":"2024-09-10T15:54:39.531950+0000","last_fullsized":"2024-09-10T15:54:39.531950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6740,"log_dups_size":0,"ondisk_log_size":6740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27607040,"num_objects":6740,"num_object_clones":0,"num_object_copies":13480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6740,"num_write_kb":26960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6793","reported_seq":6915,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:42.583227+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:42.583227+0000","last_peered":"2024-09-10T15:54:42.583227+0000","last_clean":"2024-09-10T15:54:42.583227+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:42.583227+0000","last_undegraded":"2024-09-10T15:54:42.583227+0000","last_fullsized":"2024-09-10T15:54:42.583227+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6793,"log_dups_size":0,"ondisk_log_size":6793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27795456,"num_objects":6786,"num_object_clones":0,"num_object_copies":13572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3864,"num_write_kb":15456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6994","reported_seq":7124,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.705625+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:42.705625+0000","last_peered":"2024-09-10T15:54:42.705625+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:42.705625+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6994,"log_dups_size":0,"ondisk_log_size":6994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28626944,"num_objects":6989,"num_object_clones":0,"num_object_copies":13978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3712,"num_write_kb":14848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3121},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6769","reported_seq":6905,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.686190+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:42.686190+0000","last_peered":"2024-09-10T15:54:42.686190+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:42.686190+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6769,"log_dups_size":0,"ondisk_log_size":6769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27701248,"num_objects":6763,"num_object_clones":0,"num_object_copies":13526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3419,"num_write_kb":13676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2929},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6971","reported_seq":7097,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:42.572813+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:42.572813+0000","last_peered":"2024-09-10T15:54:42.572813+0000","last_clean":"2024-09-10T15:54:42.572813+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:42.572813+0000","last_undegraded":"2024-09-10T15:54:42.572813+0000","last_fullsized":"2024-09-10T15:54:42.572813+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6971,"log_dups_size":0,"ondisk_log_size":6971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28545024,"num_objects":6969,"num_object_clones":0,"num_object_copies":13938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3730,"num_write_kb":14920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6835","reported_seq":11780,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:39.523489+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:39.523489+0000","last_peered":"2024-09-10T15:54:39.523489+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:39.523489+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6835,"log_dups_size":0,"ondisk_log_size":6835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967488,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1582,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4812,"num_write_kb":19248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2410,"num_bytes_recovered":9863168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1582},{"shards":"2","objects":5244}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6874","reported_seq":6996,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.642942+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:42.642942+0000","last_peered":"2024-09-10T15:54:42.642942+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:42.642942+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":6874,"log_dups_size":0,"ondisk_log_size":6874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28123136,"num_objects":6866,"num_object_clones":0,"num_object_copies":13732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3738,"num_write_kb":14952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3003},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6781","reported_seq":6901,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:39.479594+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:39.479594+0000","last_peered":"2024-09-10T15:54:39.479594+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:39.479594+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6781,"log_dups_size":0,"ondisk_log_size":6781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27754496,"num_objects":6776,"num_object_clones":0,"num_object_copies":13552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4755,"num_write_kb":19020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2944}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6646","reported_seq":6767,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:39.518172+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:39.518172+0000","last_peered":"2024-09-10T15:54:39.518172+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:39.518172+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6646,"log_dups_size":0,"ondisk_log_size":6646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27197440,"num_objects":6640,"num_object_clones":0,"num_object_copies":13280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4515,"num_write_kb":18060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2811}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6916","reported_seq":7039,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.723730+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:42.723730+0000","last_peered":"2024-09-10T15:54:42.723730+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:42.723730+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6916,"log_dups_size":0,"ondisk_log_size":6916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28319744,"num_objects":6914,"num_object_clones":0,"num_object_copies":13828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4108,"num_write_kb":16432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3081},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6938","reported_seq":7097,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.681640+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:42.681640+0000","last_peered":"2024-09-10T15:54:42.681640+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:42.681640+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3356,"num_write_kb":13424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3013},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6902","reported_seq":7037,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.658335+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:42.658335+0000","last_peered":"2024-09-10T15:54:42.658335+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:42.658335+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6902,"log_dups_size":0,"ondisk_log_size":6902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28254208,"num_objects":6898,"num_object_clones":0,"num_object_copies":13796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3508,"num_write_kb":14032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3021},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7046","reported_seq":7180,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.692980+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:42.692980+0000","last_peered":"2024-09-10T15:54:42.692980+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:42.692980+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28811747,"num_objects":7035,"num_object_clones":0,"num_object_copies":14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3865,"num_write_kb":15460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3032},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":449569251,"num_objects":109759,"num_object_clones":0,"num_object_copies":219518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1582,"num_objects_unfound":0,"num_objects_dirty":109759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71361,"num_write_kb":285444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2412,"num_bytes_recovered":9871360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":912232448,"data_stored":912149349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109840,"ondisk_log_size":109840,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626451,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":294632,"kb_used_data":154712,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":94077208,"statfs":{"total":96636764160,"available":96335060992,"internally_reserved":0,"allocated":158425088,"data_stored":158097170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143257262},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149735,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667972,"kb_used_data":393396,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93703868,"statfs":{"total":96636764160,"available":95952760832,"internally_reserved":0,"allocated":402837504,"data_stored":402581846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281145005},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525492,"kb_used_data":341156,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93846348,"statfs":{"total":96636764160,"available":96098660352,"internally_reserved":0,"allocated":349343744,"data_stored":349088086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403312640,"data_stored":403309027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":159186944,"data_stored":159111071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":349732864,"data_stored":349729251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:44.725 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:44.815 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 127 20 111549 111529 3.42988 1.875 0.0225033 0.0227686 2024-09-10T15:54:45.310 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:45.311 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:45.336 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-10T15:54:44.465169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":449569270,"num_objects":109761,"num_object_clones":0,"num_object_copies":219522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1582,"num_objects_unfound":0,"num_objects_dirty":109761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71363,"num_write_kb":285446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2414,"num_bytes_recovered":9871379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109843,"ondisk_log_size":109843,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488096,"kb_used_data":889264,"kb_used_omap":12,"kb_used_meta":598771,"kb_avail":281627424,"statfs":{"total":289910292480,"available":288386482176,"internally_reserved":0,"allocated":910606336,"data_stored":909767102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":613141509},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,12,10,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29261824,"num_objects":7144,"num_object_clones":0,"num_object_copies":14288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-344,"num_objects_unfound":0,"num_objects_dirty":7144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7144,"num_write_kb":28576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002810"},"pg_stats":[{"pgid":"2.f","version":"44'6925","reported_seq":6997,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.725094+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:42.725094+0000","last_peered":"2024-09-10T15:54:42.725094+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:42.725094+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6925,"log_dups_size":0,"ondisk_log_size":6925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28364800,"num_objects":6925,"num_object_clones":0,"num_object_copies":13850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6925,"num_write_kb":27700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2952},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6977","reported_seq":7079,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:39.473833+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:39.473833+0000","last_peered":"2024-09-10T15:54:39.473833+0000","last_clean":"2024-09-10T15:54:39.473833+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:39.473833+0000","last_undegraded":"2024-09-10T15:54:39.473833+0000","last_fullsized":"2024-09-10T15:54:39.473833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":6977,"log_dups_size":0,"ondisk_log_size":6977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28553216,"num_objects":6971,"num_object_clones":0,"num_object_copies":13942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5313,"num_write_kb":21252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6733","reported_seq":6851,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:39.424284+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:39.424284+0000","last_peered":"2024-09-10T15:54:39.424284+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:39.424284+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6733,"log_dups_size":0,"ondisk_log_size":6733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27561984,"num_objects":6729,"num_object_clones":0,"num_object_copies":13458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5001,"num_write_kb":20004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":2927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6740","reported_seq":6815,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:39.531950+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:39.531950+0000","last_peered":"2024-09-10T15:54:39.531950+0000","last_clean":"2024-09-10T15:54:39.531950+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:39.531950+0000","last_undegraded":"2024-09-10T15:54:39.531950+0000","last_fullsized":"2024-09-10T15:54:39.531950+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6740,"log_dups_size":0,"ondisk_log_size":6740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27607040,"num_objects":6740,"num_object_clones":0,"num_object_copies":13480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6740,"num_write_kb":26960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6793","reported_seq":6915,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:42.583227+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:42.583227+0000","last_peered":"2024-09-10T15:54:42.583227+0000","last_clean":"2024-09-10T15:54:42.583227+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:42.583227+0000","last_undegraded":"2024-09-10T15:54:42.583227+0000","last_fullsized":"2024-09-10T15:54:42.583227+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6793,"log_dups_size":0,"ondisk_log_size":6793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27795456,"num_objects":6786,"num_object_clones":0,"num_object_copies":13572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3864,"num_write_kb":15456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6994","reported_seq":7124,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.705625+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:42.705625+0000","last_peered":"2024-09-10T15:54:42.705625+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:42.705625+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6994,"log_dups_size":0,"ondisk_log_size":6994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28626944,"num_objects":6989,"num_object_clones":0,"num_object_copies":13978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3712,"num_write_kb":14848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3121},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6769","reported_seq":6905,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.686190+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:42.686190+0000","last_peered":"2024-09-10T15:54:42.686190+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:42.686190+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6769,"log_dups_size":0,"ondisk_log_size":6769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27701248,"num_objects":6763,"num_object_clones":0,"num_object_copies":13526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3419,"num_write_kb":13676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2929},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6971","reported_seq":7097,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:42.572813+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:42.572813+0000","last_peered":"2024-09-10T15:54:42.572813+0000","last_clean":"2024-09-10T15:54:42.572813+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:42.572813+0000","last_undegraded":"2024-09-10T15:54:42.572813+0000","last_fullsized":"2024-09-10T15:54:42.572813+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6971,"log_dups_size":0,"ondisk_log_size":6971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28545024,"num_objects":6969,"num_object_clones":0,"num_object_copies":13938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3730,"num_write_kb":14920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6835","reported_seq":11780,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:39.523489+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:39.523489+0000","last_peered":"2024-09-10T15:54:39.523489+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:39.523489+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":6835,"log_dups_size":0,"ondisk_log_size":6835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967488,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1582,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4812,"num_write_kb":19248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2410,"num_bytes_recovered":9863168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1582},{"shards":"2","objects":5244}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6874","reported_seq":6996,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.642942+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:42.642942+0000","last_peered":"2024-09-10T15:54:42.642942+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:42.642942+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":6874,"log_dups_size":0,"ondisk_log_size":6874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28123136,"num_objects":6866,"num_object_clones":0,"num_object_copies":13732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3738,"num_write_kb":14952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3003},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6781","reported_seq":6901,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:39.479594+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:39.479594+0000","last_peered":"2024-09-10T15:54:39.479594+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:39.479594+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6781,"log_dups_size":0,"ondisk_log_size":6781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27754496,"num_objects":6776,"num_object_clones":0,"num_object_copies":13552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4755,"num_write_kb":19020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":2944}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6646","reported_seq":6767,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:39.518172+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:39.518172+0000","last_peered":"2024-09-10T15:54:39.518172+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:39.518172+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6646,"log_dups_size":0,"ondisk_log_size":6646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27197440,"num_objects":6640,"num_object_clones":0,"num_object_copies":13280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4515,"num_write_kb":18060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2811}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6916","reported_seq":7039,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.723730+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:42.723730+0000","last_peered":"2024-09-10T15:54:42.723730+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:42.723730+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6916,"log_dups_size":0,"ondisk_log_size":6916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28319744,"num_objects":6914,"num_object_clones":0,"num_object_copies":13828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4108,"num_write_kb":16432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3081},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6938","reported_seq":7097,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.681640+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:42.681640+0000","last_peered":"2024-09-10T15:54:42.681640+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:42.681640+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3356,"num_write_kb":13424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3013},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6902","reported_seq":7037,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.658335+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:42.658335+0000","last_peered":"2024-09-10T15:54:42.658335+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:42.658335+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6902,"log_dups_size":0,"ondisk_log_size":6902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28254208,"num_objects":6898,"num_object_clones":0,"num_object_copies":13796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3508,"num_write_kb":14032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3021},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7046","reported_seq":7180,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.692980+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:42.692980+0000","last_peered":"2024-09-10T15:54:42.692980+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:42.692980+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28811747,"num_objects":7035,"num_object_clones":0,"num_object_copies":14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3865,"num_write_kb":15460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3032},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":449569251,"num_objects":109759,"num_object_clones":0,"num_object_copies":219518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1582,"num_objects_unfound":0,"num_objects_dirty":109759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71361,"num_write_kb":285444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2412,"num_bytes_recovered":9871360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":912232448,"data_stored":912149349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109840,"ondisk_log_size":109840,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626451,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":294632,"kb_used_data":154712,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":94077208,"statfs":{"total":96636764160,"available":96335060992,"internally_reserved":0,"allocated":158425088,"data_stored":158097170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143257262},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149735,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667972,"kb_used_data":393396,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93703868,"statfs":{"total":96636764160,"available":95952760832,"internally_reserved":0,"allocated":402837504,"data_stored":402581846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281145005},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525492,"kb_used_data":341156,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93846348,"statfs":{"total":96636764160,"available":96098660352,"internally_reserved":0,"allocated":349343744,"data_stored":349088086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403312640,"data_stored":403309027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":159186944,"data_stored":159111071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":349732864,"data_stored":349729251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:45.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:54:45.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'6925", 'reported_seq': 6997, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:42.725094+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:54:42.725094+0000', 'last_peered': '2024-09-10T15:54:42.725094+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:54:42.725094+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 6925, 'log_dups_size': 0, 'ondisk_log_size': 6925, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28364800, 'num_objects': 6925, 'num_object_clones': 0, 'num_object_copies': 13850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6925, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6925, 'num_write_kb': 27700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2952}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:45.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:54:45.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'6733", 'reported_seq': 6851, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:39.424284+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:54:39.424284+0000', 'last_peered': '2024-09-10T15:54:39.424284+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:54:39.424284+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 6733, 'log_dups_size': 0, 'ondisk_log_size': 6733, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27561984, 'num_objects': 6729, 'num_object_clones': 0, 'num_object_copies': 13458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6729, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5001, 'num_write_kb': 20004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 2927}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:45.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:54:45.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'6994", 'reported_seq': 7124, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:42.705625+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:54:42.705625+0000', 'last_peered': '2024-09-10T15:54:42.705625+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:54:42.705625+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 6994, 'log_dups_size': 0, 'ondisk_log_size': 6994, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28626944, 'num_objects': 6989, 'num_object_clones': 0, 'num_object_copies': 13978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6989, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3712, 'num_write_kb': 14848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3121}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:45.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:54:45.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'6769", 'reported_seq': 6905, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:42.686190+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:54:42.686190+0000', 'last_peered': '2024-09-10T15:54:42.686190+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:54:42.686190+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 6769, 'log_dups_size': 0, 'ondisk_log_size': 6769, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27701248, 'num_objects': 6763, 'num_object_clones': 0, 'num_object_copies': 13526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6763, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3419, 'num_write_kb': 13676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2929}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:45.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:54:45.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'6835", 'reported_seq': 11780, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:39.523489+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:54:39.523489+0000', 'last_peered': '2024-09-10T15:54:39.523489+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:54:39.523489+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 6835, 'log_dups_size': 0, 'ondisk_log_size': 6835, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27967488, 'num_objects': 6828, 'num_object_clones': 0, 'num_object_copies': 13656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1582, 'num_objects_unfound': 0, 'num_objects_dirty': 6828, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4812, 'num_write_kb': 19248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2410, 'num_bytes_recovered': 9863168, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1582}, {'shards': '2', 'objects': 5244}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:45.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:54:45.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'6874", 'reported_seq': 6996, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:42.642942+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:54:42.642942+0000', 'last_peered': '2024-09-10T15:54:42.642942+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:54:42.642942+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 6874, 'log_dups_size': 0, 'ondisk_log_size': 6874, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28123136, 'num_objects': 6866, 'num_object_clones': 0, 'num_object_copies': 13732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6866, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3738, 'num_write_kb': 14952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3003}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:45.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:54:45.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'6781", 'reported_seq': 6901, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:39.479594+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:54:39.479594+0000', 'last_peered': '2024-09-10T15:54:39.479594+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:54:39.479594+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 6781, 'log_dups_size': 0, 'ondisk_log_size': 6781, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27754496, 'num_objects': 6776, 'num_object_clones': 0, 'num_object_copies': 13552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4755, 'num_write_kb': 19020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 2944}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:45.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:54:45.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'6646", 'reported_seq': 6767, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:39.518172+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:54:39.518172+0000', 'last_peered': '2024-09-10T15:54:39.518172+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:54:39.518172+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 6646, 'log_dups_size': 0, 'ondisk_log_size': 6646, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27197440, 'num_objects': 6640, 'num_object_clones': 0, 'num_object_copies': 13280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6640, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4515, 'num_write_kb': 18060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 2811}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:45.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:54:45.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'6916", 'reported_seq': 7039, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:42.723730+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:54:42.723730+0000', 'last_peered': '2024-09-10T15:54:42.723730+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:54:42.723730+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 6916, 'log_dups_size': 0, 'ondisk_log_size': 6916, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28319744, 'num_objects': 6914, 'num_object_clones': 0, 'num_object_copies': 13828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6914, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4108, 'num_write_kb': 16432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3081}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:45.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:54:45.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'6938", 'reported_seq': 7097, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:42.681640+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:54:42.681640+0000', 'last_peered': '2024-09-10T15:54:42.681640+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:54:42.681640+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 6938, 'log_dups_size': 0, 'ondisk_log_size': 6938, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28385280, 'num_objects': 6930, 'num_object_clones': 0, 'num_object_copies': 13860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6930, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3356, 'num_write_kb': 13424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3013}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:45.340 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:54:45.340 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'6902", 'reported_seq': 7037, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:42.658335+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:54:42.658335+0000', 'last_peered': '2024-09-10T15:54:42.658335+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:54:42.658335+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 6902, 'log_dups_size': 0, 'ondisk_log_size': 6902, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28254208, 'num_objects': 6898, 'num_object_clones': 0, 'num_object_copies': 13796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6898, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3508, 'num_write_kb': 14032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3021}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:45.340 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:54:45.340 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'7046", 'reported_seq': 7180, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:42.692980+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:54:42.692980+0000', 'last_peered': '2024-09-10T15:54:42.692980+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:54:42.692980+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 7046, 'log_dups_size': 0, 'ondisk_log_size': 7046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28811747, 'num_objects': 7035, 'num_object_clones': 0, 'num_object_copies': 14070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7035, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3865, 'num_write_kb': 15460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3032}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:45.340 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:54:45.815 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 128 20 112057 112037 3.41859 1.98438 0.0264314 0.022846 2024-09-10T15:54:46.037 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:46.037 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/219522 objects degraded (19.430%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":211,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":109761,"data_bytes":449569270,"bytes_used":1523810304,"bytes_avail":288386482176,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":219522,"degraded_ratio":0.19429943240313044,"misplaced_objects":1582,"misplaced_total":219522,"misplaced_ratio":0.007206566995563087,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":117391,"recovering_keys_per_sec":0,"num_objects_recovered":344,"num_bytes_recovered":1409024,"num_keys_recovered":0,"write_bytes_sec":2437914,"read_op_per_sec":0,"write_op_per_sec":595},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-10T15:54:30.461931+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:54:46.061 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:54:46.061 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:46.598 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:46.631 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-10T15:54:46.465756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":453849590,"num_objects":110806,"num_object_clones":0,"num_object_copies":221612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1395,"num_objects_unfound":0,"num_objects_dirty":110806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72408,"num_write_kb":289626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2601,"num_bytes_recovered":10637331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110888,"ondisk_log_size":110888,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481128,"kb_used_data":894776,"kb_used_omap":12,"kb_used_meta":586291,"kb_avail":281634392,"statfs":{"total":289910292480,"available":288393617408,"internally_reserved":0,"allocated":916250624,"data_stored":915411390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":600361989},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,3,6,11,9,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26517504,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-531,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6474,"num_write_kb":25896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":2174976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002935"},"pg_stats":[{"pgid":"2.f","version":"44'6925","reported_seq":6997,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.725094+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:42.725094+0000","last_peered":"2024-09-10T15:54:42.725094+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:42.725094+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6925,"log_dups_size":0,"ondisk_log_size":6925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28364800,"num_objects":6925,"num_object_clones":0,"num_object_copies":13850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6925,"num_write_kb":27700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2952},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7162","reported_seq":7264,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:44.556998+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:44.556998+0000","last_peered":"2024-09-10T15:54:44.556998+0000","last_clean":"2024-09-10T15:54:44.556998+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:44.556998+0000","last_undegraded":"2024-09-10T15:54:44.556998+0000","last_fullsized":"2024-09-10T15:54:44.556998+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7162,"log_dups_size":0,"ondisk_log_size":7162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29310976,"num_objects":7156,"num_object_clones":0,"num_object_copies":14312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5498,"num_write_kb":21992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6924","reported_seq":7040,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:44.532787+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:44.532787+0000","last_peered":"2024-09-10T15:54:44.532787+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:44.532787+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6924,"log_dups_size":0,"ondisk_log_size":6924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5192,"num_write_kb":20768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6903","reported_seq":6978,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:44.514946+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:44.514946+0000","last_peered":"2024-09-10T15:54:44.514946+0000","last_clean":"2024-09-10T15:54:44.514946+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:44.514946+0000","last_undegraded":"2024-09-10T15:54:44.514946+0000","last_fullsized":"2024-09-10T15:54:44.514946+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6903,"log_dups_size":0,"ondisk_log_size":6903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28274688,"num_objects":6903,"num_object_clones":0,"num_object_copies":13806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6903,"num_write_kb":27612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6793","reported_seq":6915,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:42.583227+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:42.583227+0000","last_peered":"2024-09-10T15:54:42.583227+0000","last_clean":"2024-09-10T15:54:42.583227+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:42.583227+0000","last_undegraded":"2024-09-10T15:54:42.583227+0000","last_fullsized":"2024-09-10T15:54:42.583227+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6793,"log_dups_size":0,"ondisk_log_size":6793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27795456,"num_objects":6786,"num_object_clones":0,"num_object_copies":13572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3864,"num_write_kb":15456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6994","reported_seq":7124,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.705625+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:42.705625+0000","last_peered":"2024-09-10T15:54:42.705625+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:42.705625+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":6994,"log_dups_size":0,"ondisk_log_size":6994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28626944,"num_objects":6989,"num_object_clones":0,"num_object_copies":13978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3712,"num_write_kb":14848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3121},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6769","reported_seq":6905,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.686190+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:42.686190+0000","last_peered":"2024-09-10T15:54:42.686190+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:42.686190+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6769,"log_dups_size":0,"ondisk_log_size":6769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27701248,"num_objects":6763,"num_object_clones":0,"num_object_copies":13526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3419,"num_write_kb":13676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2929},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6971","reported_seq":7097,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:42.572813+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:42.572813+0000","last_peered":"2024-09-10T15:54:42.572813+0000","last_clean":"2024-09-10T15:54:42.572813+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:42.572813+0000","last_undegraded":"2024-09-10T15:54:42.572813+0000","last_fullsized":"2024-09-10T15:54:42.572813+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":6971,"log_dups_size":0,"ondisk_log_size":6971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28545024,"num_objects":6969,"num_object_clones":0,"num_object_copies":13938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3730,"num_write_kb":14920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7027","reported_seq":12346,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:44.552164+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:44.552164+0000","last_peered":"2024-09-10T15:54:44.552164+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:44.552164+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7027,"log_dups_size":0,"ondisk_log_size":7027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28753920,"num_objects":7020,"num_object_clones":0,"num_object_copies":14040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1395,"num_objects_unfound":0,"num_objects_dirty":7020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5004,"num_write_kb":20016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2597,"num_bytes_recovered":10629120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1395},{"shards":"2","objects":5623}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6874","reported_seq":6996,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.642942+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:42.642942+0000","last_peered":"2024-09-10T15:54:42.642942+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:42.642942+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":6874,"log_dups_size":0,"ondisk_log_size":6874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28123136,"num_objects":6866,"num_object_clones":0,"num_object_copies":13732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3738,"num_write_kb":14952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3003},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6945","reported_seq":7065,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:44.539930+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:44.539930+0000","last_peered":"2024-09-10T15:54:44.539930+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:44.539930+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6945,"log_dups_size":0,"ondisk_log_size":6945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28426240,"num_objects":6940,"num_object_clones":0,"num_object_copies":13880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4919,"num_write_kb":19676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3108}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6796","reported_seq":6917,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:44.451815+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:44.451815+0000","last_peered":"2024-09-10T15:54:44.451815+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:44.451815+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6796,"log_dups_size":0,"ondisk_log_size":6796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27811840,"num_objects":6790,"num_object_clones":0,"num_object_copies":13580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4665,"num_write_kb":18660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2961}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6916","reported_seq":7039,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.723730+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:42.723730+0000","last_peered":"2024-09-10T15:54:42.723730+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:42.723730+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":6916,"log_dups_size":0,"ondisk_log_size":6916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28319744,"num_objects":6914,"num_object_clones":0,"num_object_copies":13828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4108,"num_write_kb":16432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3081},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6938","reported_seq":7097,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.681640+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:42.681640+0000","last_peered":"2024-09-10T15:54:42.681640+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:42.681640+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3356,"num_write_kb":13424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3013},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6902","reported_seq":7037,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.658335+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:42.658335+0000","last_peered":"2024-09-10T15:54:42.658335+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:42.658335+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":6902,"log_dups_size":0,"ondisk_log_size":6902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28254208,"num_objects":6898,"num_object_clones":0,"num_object_copies":13796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3508,"num_write_kb":14032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3021},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7046","reported_seq":7180,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:42.692980+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:42.692980+0000","last_peered":"2024-09-10T15:54:42.692980+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:42.692980+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28811747,"num_objects":7035,"num_object_clones":0,"num_object_copies":14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3865,"num_write_kb":15460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3032},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":453849571,"num_objects":110804,"num_object_clones":0,"num_object_copies":221608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1395,"num_objects_unfound":0,"num_objects_dirty":110804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72406,"num_write_kb":289624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2599,"num_bytes_recovered":10637312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":917958656,"data_stored":917875557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110885,"ondisk_log_size":110885,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626451,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":294632,"kb_used_data":154712,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":94077208,"statfs":{"total":96636764160,"available":96335060992,"internally_reserved":0,"allocated":158425088,"data_stored":158097170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143257262},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149735,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667972,"kb_used_data":393396,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93703868,"statfs":{"total":96636764160,"available":95952760832,"internally_reserved":0,"allocated":402837504,"data_stored":402581846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281145005},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518524,"kb_used_data":346668,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93853316,"statfs":{"total":96636764160,"available":96105795584,"internally_reserved":0,"allocated":354988032,"data_stored":354732374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,3,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403312640,"data_stored":403309027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":159186944,"data_stored":159111071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355459072,"data_stored":355455459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:46.815 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 129 19 112561 112542 3.40738 1.97266 0.075483 0.0229206 2024-09-10T15:54:47.815 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 130 20 113120 113100 3.39794 2.17969 0.0426736 0.0229844 2024-09-10T15:54:48.815 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 131 20 113691 113671 3.38902 2.23047 0.0374524 0.0230449 2024-09-10T15:54:49.633 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:49.816 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 132 20 114246 114226 3.37977 2.16797 0.0370394 0.0231078 2024-09-10T15:54:50.207 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:50.210 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:50.229 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-10T15:54:48.466220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":460771830,"num_objects":112496,"num_object_clones":0,"num_object_copies":224992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1395,"num_objects_unfound":0,"num_objects_dirty":112496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74098,"num_write_kb":296386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2601,"num_bytes_recovered":10637331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112578,"ondisk_log_size":112578,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511616,"kb_used_data":911376,"kb_used_omap":12,"kb_used_meta":600179,"kb_avail":281603904,"statfs":{"total":289910292480,"available":288362397696,"internally_reserved":0,"allocated":933249024,"data_stored":932409790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":614583301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,3,5,13,7,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29335552,"num_objects":7162,"num_object_clones":0,"num_object_copies":14324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-348,"num_objects_unfound":0,"num_objects_dirty":7162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7162,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":1425408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002895"},"pg_stats":[{"pgid":"2.f","version":"44'7097","reported_seq":7169,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.750958+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:47.750958+0000","last_peered":"2024-09-10T15:54:47.750958+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:47.750958+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7097,"log_dups_size":0,"ondisk_log_size":7097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29069312,"num_objects":7097,"num_object_clones":0,"num_object_copies":14194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7097,"num_write_kb":28388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3124},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7162","reported_seq":7264,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:44.556998+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:44.556998+0000","last_peered":"2024-09-10T15:54:44.556998+0000","last_clean":"2024-09-10T15:54:44.556998+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:44.556998+0000","last_undegraded":"2024-09-10T15:54:44.556998+0000","last_fullsized":"2024-09-10T15:54:44.556998+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7162,"log_dups_size":0,"ondisk_log_size":7162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29310976,"num_objects":7156,"num_object_clones":0,"num_object_copies":14312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5498,"num_write_kb":21992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6924","reported_seq":7040,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:44.532787+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:44.532787+0000","last_peered":"2024-09-10T15:54:44.532787+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:44.532787+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":6924,"log_dups_size":0,"ondisk_log_size":6924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5192,"num_write_kb":20768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6903","reported_seq":6978,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:44.514946+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:44.514946+0000","last_peered":"2024-09-10T15:54:44.514946+0000","last_clean":"2024-09-10T15:54:44.514946+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:44.514946+0000","last_undegraded":"2024-09-10T15:54:44.514946+0000","last_fullsized":"2024-09-10T15:54:44.514946+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":6903,"log_dups_size":0,"ondisk_log_size":6903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28274688,"num_objects":6903,"num_object_clones":0,"num_object_copies":13806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6903,"num_write_kb":27612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6956","reported_seq":7078,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:47.793473+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:47.793473+0000","last_peered":"2024-09-10T15:54:47.793473+0000","last_clean":"2024-09-10T15:54:47.793473+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:47.793473+0000","last_undegraded":"2024-09-10T15:54:47.793473+0000","last_fullsized":"2024-09-10T15:54:47.793473+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6956,"log_dups_size":0,"ondisk_log_size":6956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28463104,"num_objects":6949,"num_object_clones":0,"num_object_copies":13898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4027,"num_write_kb":16108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7164","reported_seq":7294,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.706678+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:47.706678+0000","last_peered":"2024-09-10T15:54:47.706678+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:47.706678+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7164,"log_dups_size":0,"ondisk_log_size":7164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29323264,"num_objects":7159,"num_object_clones":0,"num_object_copies":14318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3882,"num_write_kb":15528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3291},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6935","reported_seq":7071,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.665395+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:47.665395+0000","last_peered":"2024-09-10T15:54:47.665395+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:47.665395+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6935,"log_dups_size":0,"ondisk_log_size":6935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28381184,"num_objects":6929,"num_object_clones":0,"num_object_copies":13858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3585,"num_write_kb":14340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3095},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7138","reported_seq":7264,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:47.742775+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:47.742775+0000","last_peered":"2024-09-10T15:54:47.742775+0000","last_clean":"2024-09-10T15:54:47.742775+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:47.742775+0000","last_undegraded":"2024-09-10T15:54:47.742775+0000","last_fullsized":"2024-09-10T15:54:47.742775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7138,"log_dups_size":0,"ondisk_log_size":7138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29229056,"num_objects":7136,"num_object_clones":0,"num_object_copies":14272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3897,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7027","reported_seq":12346,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:44.552164+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:44.552164+0000","last_peered":"2024-09-10T15:54:44.552164+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:44.552164+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7027,"log_dups_size":0,"ondisk_log_size":7027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28753920,"num_objects":7020,"num_object_clones":0,"num_object_copies":14040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1395,"num_objects_unfound":0,"num_objects_dirty":7020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5004,"num_write_kb":20016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2597,"num_bytes_recovered":10629120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1395},{"shards":"2","objects":5623}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7032","reported_seq":7154,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.744155+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:47.744155+0000","last_peered":"2024-09-10T15:54:47.744155+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:47.744155+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7032,"log_dups_size":0,"ondisk_log_size":7032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28770304,"num_objects":7024,"num_object_clones":0,"num_object_copies":14048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3896,"num_write_kb":15584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3161},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6945","reported_seq":7065,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:44.539930+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:44.539930+0000","last_peered":"2024-09-10T15:54:44.539930+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:44.539930+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":6945,"log_dups_size":0,"ondisk_log_size":6945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28426240,"num_objects":6940,"num_object_clones":0,"num_object_copies":13880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4919,"num_write_kb":19676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3108}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6796","reported_seq":6917,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:44.451815+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:44.451815+0000","last_peered":"2024-09-10T15:54:44.451815+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:44.451815+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6796,"log_dups_size":0,"ondisk_log_size":6796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27811840,"num_objects":6790,"num_object_clones":0,"num_object_copies":13580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4665,"num_write_kb":18660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":2961}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7087","reported_seq":7209,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.793098+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:47.793098+0000","last_peered":"2024-09-10T15:54:47.793098+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:47.793098+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7087,"log_dups_size":0,"ondisk_log_size":7087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29020160,"num_objects":7085,"num_object_clones":0,"num_object_copies":14170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4279,"num_write_kb":17116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3252},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7108","reported_seq":7267,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.744389+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:47.744389+0000","last_peered":"2024-09-10T15:54:47.744389+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:47.744389+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7108,"log_dups_size":0,"ondisk_log_size":7108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29081600,"num_objects":7100,"num_object_clones":0,"num_object_copies":14200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3526,"num_write_kb":14104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3183},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7063","reported_seq":7198,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.639210+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:47.639210+0000","last_peered":"2024-09-10T15:54:47.639210+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:47.639210+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7063,"log_dups_size":0,"ondisk_log_size":7063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28913664,"num_objects":7059,"num_object_clones":0,"num_object_copies":14118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3669,"num_write_kb":14676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3182},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7238","reported_seq":7372,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.769316+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:47.769316+0000","last_peered":"2024-09-10T15:54:47.769316+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:47.769316+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7238,"log_dups_size":0,"ondisk_log_size":7238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29598179,"num_objects":7227,"num_object_clones":0,"num_object_copies":14454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4057,"num_write_kb":16228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3224},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":460771811,"num_objects":112494,"num_object_clones":0,"num_object_copies":224988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1395,"num_objects_unfound":0,"num_objects_dirty":112494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74096,"num_write_kb":296384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2599,"num_bytes_recovered":10637312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":935190528,"data_stored":935107429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112575,"ondisk_log_size":112575,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626452,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329720,"kb_used_data":163432,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":94042120,"statfs":{"total":96636764160,"available":96299130880,"internally_reserved":0,"allocated":167354368,"data_stored":167026450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170258094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149736,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663372,"kb_used_data":401276,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93708468,"statfs":{"total":96636764160,"available":95957471232,"internally_reserved":0,"allocated":410906624,"data_stored":410650966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268365485},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518524,"kb_used_data":346668,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93853316,"statfs":{"total":96636764160,"available":96105795584,"internally_reserved":0,"allocated":354988032,"data_stored":354732374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,3,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411602944,"data_stored":411599331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":168128512,"data_stored":168052639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355459072,"data_stored":355455459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:50.231 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:50.782 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:50.782 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:50.807 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-10T15:54:50.466722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464855542,"num_objects":113493,"num_object_clones":0,"num_object_copies":226986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1208,"num_objects_unfound":0,"num_objects_dirty":113493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75095,"num_write_kb":300374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2788,"num_bytes_recovered":11403283,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113575,"ondisk_log_size":113575,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520848,"kb_used_data":916448,"kb_used_omap":12,"kb_used_meta":604339,"kb_avail":281594672,"statfs":{"total":289910292480,"available":288352944128,"internally_reserved":0,"allocated":938442752,"data_stored":937603518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":618843141},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,13,6,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26292224,"num_objects":6419,"num_object_clones":0,"num_object_copies":12838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-535,"num_objects_unfound":0,"num_objects_dirty":6419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6419,"num_write_kb":25676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":535,"num_bytes_recovered":2191360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002901"},"pg_stats":[{"pgid":"2.f","version":"44'7097","reported_seq":7169,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.750958+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:47.750958+0000","last_peered":"2024-09-10T15:54:47.750958+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:47.750958+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7097,"log_dups_size":0,"ondisk_log_size":7097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29069312,"num_objects":7097,"num_object_clones":0,"num_object_copies":14194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7097,"num_write_kb":28388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3124},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7338","reported_seq":7440,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:49.530384+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:49.530384+0000","last_peered":"2024-09-10T15:54:49.530384+0000","last_clean":"2024-09-10T15:54:49.530384+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:49.530384+0000","last_undegraded":"2024-09-10T15:54:49.530384+0000","last_fullsized":"2024-09-10T15:54:49.530384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7338,"log_dups_size":0,"ondisk_log_size":7338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5674,"num_write_kb":22696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7082","reported_seq":7201,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.548059+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:49.548059+0000","last_peered":"2024-09-10T15:54:49.548059+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:49.548059+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28991488,"num_objects":7078,"num_object_clones":0,"num_object_copies":14156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5350,"num_write_kb":21400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7075","reported_seq":7150,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:49.543415+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:49.543415+0000","last_peered":"2024-09-10T15:54:49.543415+0000","last_clean":"2024-09-10T15:54:49.543415+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:49.543415+0000","last_undegraded":"2024-09-10T15:54:49.543415+0000","last_fullsized":"2024-09-10T15:54:49.543415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7075,"log_dups_size":0,"ondisk_log_size":7075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28979200,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7075,"num_write_kb":28300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6956","reported_seq":7078,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:47.793473+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:47.793473+0000","last_peered":"2024-09-10T15:54:47.793473+0000","last_clean":"2024-09-10T15:54:47.793473+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:47.793473+0000","last_undegraded":"2024-09-10T15:54:47.793473+0000","last_fullsized":"2024-09-10T15:54:47.793473+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6956,"log_dups_size":0,"ondisk_log_size":6956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28463104,"num_objects":6949,"num_object_clones":0,"num_object_copies":13898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4027,"num_write_kb":16108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7164","reported_seq":7294,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.706678+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:47.706678+0000","last_peered":"2024-09-10T15:54:47.706678+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:47.706678+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7164,"log_dups_size":0,"ondisk_log_size":7164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29323264,"num_objects":7159,"num_object_clones":0,"num_object_copies":14318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3882,"num_write_kb":15528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3291},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6935","reported_seq":7071,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.665395+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:47.665395+0000","last_peered":"2024-09-10T15:54:47.665395+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:47.665395+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6935,"log_dups_size":0,"ondisk_log_size":6935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28381184,"num_objects":6929,"num_object_clones":0,"num_object_copies":13858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3585,"num_write_kb":14340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3095},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7138","reported_seq":7264,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:47.742775+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:47.742775+0000","last_peered":"2024-09-10T15:54:47.742775+0000","last_clean":"2024-09-10T15:54:47.742775+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:47.742775+0000","last_undegraded":"2024-09-10T15:54:47.742775+0000","last_fullsized":"2024-09-10T15:54:47.742775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7138,"log_dups_size":0,"ondisk_log_size":7138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29229056,"num_objects":7136,"num_object_clones":0,"num_object_copies":14272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3897,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7172","reported_seq":12864,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.569560+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:49.569560+0000","last_peered":"2024-09-10T15:54:49.569560+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:49.569560+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7172,"log_dups_size":0,"ondisk_log_size":7172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1208,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5149,"num_write_kb":20596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2784,"num_bytes_recovered":11395072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1208},{"shards":"2","objects":5955}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7032","reported_seq":7154,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.744155+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:47.744155+0000","last_peered":"2024-09-10T15:54:47.744155+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:47.744155+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7032,"log_dups_size":0,"ondisk_log_size":7032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28770304,"num_objects":7024,"num_object_clones":0,"num_object_copies":14048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3896,"num_write_kb":15584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3161},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7135","reported_seq":7255,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.542652+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:49.542652+0000","last_peered":"2024-09-10T15:54:49.542652+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:49.542652+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7135,"log_dups_size":0,"ondisk_log_size":7135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5109,"num_write_kb":20436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6952","reported_seq":7073,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.526194+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:49.526194+0000","last_peered":"2024-09-10T15:54:49.526194+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:49.526194+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6952,"log_dups_size":0,"ondisk_log_size":6952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28450816,"num_objects":6946,"num_object_clones":0,"num_object_copies":13892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4821,"num_write_kb":19284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3117}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7087","reported_seq":7209,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.793098+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:47.793098+0000","last_peered":"2024-09-10T15:54:47.793098+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:47.793098+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7087,"log_dups_size":0,"ondisk_log_size":7087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29020160,"num_objects":7085,"num_object_clones":0,"num_object_copies":14170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4279,"num_write_kb":17116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3252},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7108","reported_seq":7267,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.744389+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:47.744389+0000","last_peered":"2024-09-10T15:54:47.744389+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:47.744389+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7108,"log_dups_size":0,"ondisk_log_size":7108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29081600,"num_objects":7100,"num_object_clones":0,"num_object_copies":14200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3526,"num_write_kb":14104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3183},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7063","reported_seq":7198,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.639210+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:47.639210+0000","last_peered":"2024-09-10T15:54:47.639210+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:47.639210+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7063,"log_dups_size":0,"ondisk_log_size":7063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28913664,"num_objects":7059,"num_object_clones":0,"num_object_copies":14118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3669,"num_write_kb":14676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3182},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7238","reported_seq":7372,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.769316+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:47.769316+0000","last_peered":"2024-09-10T15:54:47.769316+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:47.769316+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7238,"log_dups_size":0,"ondisk_log_size":7238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29598179,"num_objects":7227,"num_object_clones":0,"num_object_copies":14454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4057,"num_write_kb":16228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3224},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":464855523,"num_objects":113491,"num_object_clones":0,"num_object_copies":226982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1208,"num_objects_unfound":0,"num_objects_dirty":113491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75093,"num_write_kb":300372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2786,"num_bytes_recovered":11403264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":940621824,"data_stored":940538725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113572,"ondisk_log_size":113572,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626452,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329720,"kb_used_data":163432,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":94042120,"statfs":{"total":96636764160,"available":96299130880,"internally_reserved":0,"allocated":167354368,"data_stored":167026450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170258094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149736,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663372,"kb_used_data":401276,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93708468,"statfs":{"total":96636764160,"available":95957471232,"internally_reserved":0,"allocated":410906624,"data_stored":410650966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268365485},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527756,"kb_used_data":351740,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93844084,"statfs":{"total":96636764160,"available":96096342016,"internally_reserved":0,"allocated":360181760,"data_stored":359926102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411602944,"data_stored":411599331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":168128512,"data_stored":168052639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360890368,"data_stored":360886755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:50.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:54:50.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'7097", 'reported_seq': 7169, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:47.750958+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:54:47.750958+0000', 'last_peered': '2024-09-10T15:54:47.750958+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:54:47.750958+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 7097, 'log_dups_size': 0, 'ondisk_log_size': 7097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29069312, 'num_objects': 7097, 'num_object_clones': 0, 'num_object_copies': 14194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7097, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7097, 'num_write_kb': 28388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3124}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:50.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:54:50.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'7082", 'reported_seq': 7201, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:49.548059+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:54:49.548059+0000', 'last_peered': '2024-09-10T15:54:49.548059+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:54:49.548059+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 7082, 'log_dups_size': 0, 'ondisk_log_size': 7082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28991488, 'num_objects': 7078, 'num_object_clones': 0, 'num_object_copies': 14156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7078, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5350, 'num_write_kb': 21400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 3276}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:50.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:54:50.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'7164", 'reported_seq': 7294, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:47.706678+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:54:47.706678+0000', 'last_peered': '2024-09-10T15:54:47.706678+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:54:47.706678+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 7164, 'log_dups_size': 0, 'ondisk_log_size': 7164, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29323264, 'num_objects': 7159, 'num_object_clones': 0, 'num_object_copies': 14318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3882, 'num_write_kb': 15528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3291}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:50.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:54:50.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'6935", 'reported_seq': 7071, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:47.665395+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:54:47.665395+0000', 'last_peered': '2024-09-10T15:54:47.665395+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:54:47.665395+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 6935, 'log_dups_size': 0, 'ondisk_log_size': 6935, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28381184, 'num_objects': 6929, 'num_object_clones': 0, 'num_object_copies': 13858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6929, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3585, 'num_write_kb': 14340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3095}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:50.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:54:50.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'7172", 'reported_seq': 12864, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:49.569560+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:54:49.569560+0000', 'last_peered': '2024-09-10T15:54:49.569560+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:54:49.569560+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 7172, 'log_dups_size': 0, 'ondisk_log_size': 7172, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29347840, 'num_objects': 7165, 'num_object_clones': 0, 'num_object_copies': 14330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1208, 'num_objects_unfound': 0, 'num_objects_dirty': 7165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5149, 'num_write_kb': 20596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2784, 'num_bytes_recovered': 11395072, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1208}, {'shards': '2', 'objects': 5955}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:50.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:54:50.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'7032", 'reported_seq': 7154, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:47.744155+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:54:47.744155+0000', 'last_peered': '2024-09-10T15:54:47.744155+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:54:47.744155+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 7032, 'log_dups_size': 0, 'ondisk_log_size': 7032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28770304, 'num_objects': 7024, 'num_object_clones': 0, 'num_object_copies': 14048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3896, 'num_write_kb': 15584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3161}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:50.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:54:50.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'7135", 'reported_seq': 7255, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:49.542652+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:54:49.542652+0000', 'last_peered': '2024-09-10T15:54:49.542652+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:54:49.542652+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 7135, 'log_dups_size': 0, 'ondisk_log_size': 7135, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29204480, 'num_objects': 7130, 'num_object_clones': 0, 'num_object_copies': 14260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5109, 'num_write_kb': 20436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 3298}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:50.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:54:50.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'6952", 'reported_seq': 7073, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:49.526194+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:54:49.526194+0000', 'last_peered': '2024-09-10T15:54:49.526194+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:54:49.526194+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 6952, 'log_dups_size': 0, 'ondisk_log_size': 6952, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28450816, 'num_objects': 6946, 'num_object_clones': 0, 'num_object_copies': 13892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6946, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4821, 'num_write_kb': 19284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 3117}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:50.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:54:50.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'7087", 'reported_seq': 7209, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:47.793098+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:54:47.793098+0000', 'last_peered': '2024-09-10T15:54:47.793098+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:54:47.793098+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 7087, 'log_dups_size': 0, 'ondisk_log_size': 7087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29020160, 'num_objects': 7085, 'num_object_clones': 0, 'num_object_copies': 14170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7085, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4279, 'num_write_kb': 17116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3252}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:50.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:54:50.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'7108", 'reported_seq': 7267, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:47.744389+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:54:47.744389+0000', 'last_peered': '2024-09-10T15:54:47.744389+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:54:47.744389+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 7108, 'log_dups_size': 0, 'ondisk_log_size': 7108, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29081600, 'num_objects': 7100, 'num_object_clones': 0, 'num_object_copies': 14200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7100, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3526, 'num_write_kb': 14104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3183}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:50.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:54:50.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'7063", 'reported_seq': 7198, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:47.639210+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:54:47.639210+0000', 'last_peered': '2024-09-10T15:54:47.639210+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:54:47.639210+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 7063, 'log_dups_size': 0, 'ondisk_log_size': 7063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28913664, 'num_objects': 7059, 'num_object_clones': 0, 'num_object_copies': 14118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7059, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3669, 'num_write_kb': 14676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3182}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:50.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:54:50.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'7238", 'reported_seq': 7372, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:47.769316+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:54:47.769316+0000', 'last_peered': '2024-09-10T15:54:47.769316+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:54:47.769316+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 7238, 'log_dups_size': 0, 'ondisk_log_size': 7238, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29598179, 'num_objects': 7227, 'num_object_clones': 0, 'num_object_copies': 14454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7227, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4057, 'num_write_kb': 16228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3224}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:50.811 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:54:50.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 133 20 114735 114715 3.36872 1.91016 0.0269756 0.0231831 2024-09-10T15:54:51.526 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:51.527 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/226986 objects degraded (18.791%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":217,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":113493,"data_bytes":464855542,"bytes_used":1557348352,"bytes_avail":288352944128,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":226986,"degraded_ratio":0.18791026759359608,"misplaced_objects":1208,"misplaced_total":226986,"misplaced_ratio":0.0053219141268624507,"recovering_objects_per_sec":44,"recovering_bytes_per_sec":182569,"recovering_keys_per_sec":0,"num_objects_recovered":535,"num_bytes_recovered":2191360,"num_keys_recovered":0,"write_bytes_sec":2190488,"read_op_per_sec":0,"write_op_per_sec":534},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-10T15:54:30.461931+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:54:51.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:54:51.554 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:51.816 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 134 20 115217 115197 3.35763 1.88281 0.01474 0.0232587 2024-09-10T15:54:52.126 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:52.127 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:52.150 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-10T15:54:50.466722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464855542,"num_objects":113493,"num_object_clones":0,"num_object_copies":226986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1208,"num_objects_unfound":0,"num_objects_dirty":113493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75095,"num_write_kb":300374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2788,"num_bytes_recovered":11403283,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113575,"ondisk_log_size":113575,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520848,"kb_used_data":916448,"kb_used_omap":12,"kb_used_meta":604339,"kb_avail":281594672,"statfs":{"total":289910292480,"available":288352944128,"internally_reserved":0,"allocated":938442752,"data_stored":937603518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":618843141},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,13,6,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26292224,"num_objects":6419,"num_object_clones":0,"num_object_copies":12838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-535,"num_objects_unfound":0,"num_objects_dirty":6419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6419,"num_write_kb":25676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":535,"num_bytes_recovered":2191360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002901"},"pg_stats":[{"pgid":"2.f","version":"44'7097","reported_seq":7169,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.750958+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:47.750958+0000","last_peered":"2024-09-10T15:54:47.750958+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:47.750958+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7097,"log_dups_size":0,"ondisk_log_size":7097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29069312,"num_objects":7097,"num_object_clones":0,"num_object_copies":14194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7097,"num_write_kb":28388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3124},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7338","reported_seq":7440,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:49.530384+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:49.530384+0000","last_peered":"2024-09-10T15:54:49.530384+0000","last_clean":"2024-09-10T15:54:49.530384+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:49.530384+0000","last_undegraded":"2024-09-10T15:54:49.530384+0000","last_fullsized":"2024-09-10T15:54:49.530384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7338,"log_dups_size":0,"ondisk_log_size":7338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5674,"num_write_kb":22696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7082","reported_seq":7201,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.548059+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:49.548059+0000","last_peered":"2024-09-10T15:54:49.548059+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:49.548059+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28991488,"num_objects":7078,"num_object_clones":0,"num_object_copies":14156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5350,"num_write_kb":21400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7075","reported_seq":7150,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:49.543415+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:49.543415+0000","last_peered":"2024-09-10T15:54:49.543415+0000","last_clean":"2024-09-10T15:54:49.543415+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:49.543415+0000","last_undegraded":"2024-09-10T15:54:49.543415+0000","last_fullsized":"2024-09-10T15:54:49.543415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7075,"log_dups_size":0,"ondisk_log_size":7075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28979200,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7075,"num_write_kb":28300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6956","reported_seq":7078,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:47.793473+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:47.793473+0000","last_peered":"2024-09-10T15:54:47.793473+0000","last_clean":"2024-09-10T15:54:47.793473+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:47.793473+0000","last_undegraded":"2024-09-10T15:54:47.793473+0000","last_fullsized":"2024-09-10T15:54:47.793473+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":6956,"log_dups_size":0,"ondisk_log_size":6956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28463104,"num_objects":6949,"num_object_clones":0,"num_object_copies":13898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4027,"num_write_kb":16108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7164","reported_seq":7294,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.706678+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:47.706678+0000","last_peered":"2024-09-10T15:54:47.706678+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:47.706678+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7164,"log_dups_size":0,"ondisk_log_size":7164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29323264,"num_objects":7159,"num_object_clones":0,"num_object_copies":14318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3882,"num_write_kb":15528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3291},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6935","reported_seq":7071,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.665395+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:47.665395+0000","last_peered":"2024-09-10T15:54:47.665395+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:47.665395+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":6935,"log_dups_size":0,"ondisk_log_size":6935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28381184,"num_objects":6929,"num_object_clones":0,"num_object_copies":13858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3585,"num_write_kb":14340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3095},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7138","reported_seq":7264,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:47.742775+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:47.742775+0000","last_peered":"2024-09-10T15:54:47.742775+0000","last_clean":"2024-09-10T15:54:47.742775+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:47.742775+0000","last_undegraded":"2024-09-10T15:54:47.742775+0000","last_fullsized":"2024-09-10T15:54:47.742775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7138,"log_dups_size":0,"ondisk_log_size":7138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29229056,"num_objects":7136,"num_object_clones":0,"num_object_copies":14272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3897,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7172","reported_seq":12864,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.569560+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:49.569560+0000","last_peered":"2024-09-10T15:54:49.569560+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:49.569560+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7172,"log_dups_size":0,"ondisk_log_size":7172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1208,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5149,"num_write_kb":20596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2784,"num_bytes_recovered":11395072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1208},{"shards":"2","objects":5955}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7032","reported_seq":7154,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.744155+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:47.744155+0000","last_peered":"2024-09-10T15:54:47.744155+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:47.744155+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7032,"log_dups_size":0,"ondisk_log_size":7032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28770304,"num_objects":7024,"num_object_clones":0,"num_object_copies":14048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3896,"num_write_kb":15584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3161},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7135","reported_seq":7255,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.542652+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:49.542652+0000","last_peered":"2024-09-10T15:54:49.542652+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:49.542652+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7135,"log_dups_size":0,"ondisk_log_size":7135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5109,"num_write_kb":20436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6952","reported_seq":7073,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.526194+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:49.526194+0000","last_peered":"2024-09-10T15:54:49.526194+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:49.526194+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6952,"log_dups_size":0,"ondisk_log_size":6952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28450816,"num_objects":6946,"num_object_clones":0,"num_object_copies":13892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4821,"num_write_kb":19284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3117}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7087","reported_seq":7209,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.793098+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:47.793098+0000","last_peered":"2024-09-10T15:54:47.793098+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:47.793098+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7087,"log_dups_size":0,"ondisk_log_size":7087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29020160,"num_objects":7085,"num_object_clones":0,"num_object_copies":14170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4279,"num_write_kb":17116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3252},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7108","reported_seq":7267,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.744389+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:47.744389+0000","last_peered":"2024-09-10T15:54:47.744389+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:47.744389+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7108,"log_dups_size":0,"ondisk_log_size":7108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29081600,"num_objects":7100,"num_object_clones":0,"num_object_copies":14200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3526,"num_write_kb":14104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3183},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7063","reported_seq":7198,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.639210+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:47.639210+0000","last_peered":"2024-09-10T15:54:47.639210+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:47.639210+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7063,"log_dups_size":0,"ondisk_log_size":7063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28913664,"num_objects":7059,"num_object_clones":0,"num_object_copies":14118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3669,"num_write_kb":14676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3182},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7238","reported_seq":7372,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:47.769316+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:47.769316+0000","last_peered":"2024-09-10T15:54:47.769316+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:47.769316+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7238,"log_dups_size":0,"ondisk_log_size":7238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29598179,"num_objects":7227,"num_object_clones":0,"num_object_copies":14454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4057,"num_write_kb":16228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3224},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":464855523,"num_objects":113491,"num_object_clones":0,"num_object_copies":226982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1208,"num_objects_unfound":0,"num_objects_dirty":113491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75093,"num_write_kb":300372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2786,"num_bytes_recovered":11403264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":940621824,"data_stored":940538725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113572,"ondisk_log_size":113572,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626452,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329720,"kb_used_data":163432,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":94042120,"statfs":{"total":96636764160,"available":96299130880,"internally_reserved":0,"allocated":167354368,"data_stored":167026450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170258094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149736,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663372,"kb_used_data":401276,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93708468,"statfs":{"total":96636764160,"available":95957471232,"internally_reserved":0,"allocated":410906624,"data_stored":410650966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268365485},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527756,"kb_used_data":351740,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93844084,"statfs":{"total":96636764160,"available":96096342016,"internally_reserved":0,"allocated":360181760,"data_stored":359926102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411602944,"data_stored":411599331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":168128512,"data_stored":168052639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360890368,"data_stored":360886755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:52.816 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 135 20 115729 115709 3.34758 2 0.0107573 0.0233282 2024-09-10T15:54:53.816 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 136 20 116282 116262 3.33884 2.16016 0.0287898 0.0233908 2024-09-10T15:54:54.816 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 137 20 116827 116807 3.33001 2.12891 0.0343615 0.0234539 2024-09-10T15:54:55.152 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:55.712 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:55.712 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:55.736 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-10T15:54:54.467815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471593462,"num_objects":115138,"num_object_clones":0,"num_object_copies":230276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1208,"num_objects_unfound":0,"num_objects_dirty":115138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76740,"num_write_kb":306954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2788,"num_bytes_recovered":11403283,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115220,"ondisk_log_size":115220,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529212,"kb_used_data":933132,"kb_used_omap":12,"kb_used_meta":596019,"kb_avail":281586308,"statfs":{"total":289910292480,"available":288344379392,"internally_reserved":0,"allocated":955527168,"data_stored":954687934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":610323461},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,9,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28807168,"num_objects":7033,"num_object_clones":0,"num_object_copies":14066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-374,"num_objects_unfound":0,"num_objects_dirty":7033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7033,"num_write_kb":28132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":1531904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003108"},"pg_stats":[{"pgid":"2.f","version":"44'7279","reported_seq":7349,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.845516+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:52.845516+0000","last_peered":"2024-09-10T15:54:52.845516+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:52.845516+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7279,"log_dups_size":0,"ondisk_log_size":7279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29814784,"num_objects":7279,"num_object_clones":0,"num_object_copies":14558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7279,"num_write_kb":29116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3306},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7338","reported_seq":7440,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:49.530384+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:49.530384+0000","last_peered":"2024-09-10T15:54:49.530384+0000","last_clean":"2024-09-10T15:54:49.530384+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:49.530384+0000","last_undegraded":"2024-09-10T15:54:49.530384+0000","last_fullsized":"2024-09-10T15:54:49.530384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7338,"log_dups_size":0,"ondisk_log_size":7338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5674,"num_write_kb":22696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7082","reported_seq":7201,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.548059+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:49.548059+0000","last_peered":"2024-09-10T15:54:49.548059+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:49.548059+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28991488,"num_objects":7078,"num_object_clones":0,"num_object_copies":14156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5350,"num_write_kb":21400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7075","reported_seq":7150,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:49.543415+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:49.543415+0000","last_peered":"2024-09-10T15:54:49.543415+0000","last_clean":"2024-09-10T15:54:49.543415+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:49.543415+0000","last_undegraded":"2024-09-10T15:54:49.543415+0000","last_fullsized":"2024-09-10T15:54:49.543415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7075,"log_dups_size":0,"ondisk_log_size":7075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28979200,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7075,"num_write_kb":28300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7116","reported_seq":7238,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:52.810805+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:52.810805+0000","last_peered":"2024-09-10T15:54:52.810805+0000","last_clean":"2024-09-10T15:54:52.810805+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:52.810805+0000","last_undegraded":"2024-09-10T15:54:52.810805+0000","last_fullsized":"2024-09-10T15:54:52.810805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7116,"log_dups_size":0,"ondisk_log_size":7116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29118464,"num_objects":7109,"num_object_clones":0,"num_object_copies":14218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4187,"num_write_kb":16748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7334","reported_seq":7464,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.742327+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:52.742327+0000","last_peered":"2024-09-10T15:54:52.742327+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:52.742327+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7334,"log_dups_size":0,"ondisk_log_size":7334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30019584,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4052,"num_write_kb":16208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3461},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7099","reported_seq":7235,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.773121+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:52.773121+0000","last_peered":"2024-09-10T15:54:52.773121+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:52.773121+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7099,"log_dups_size":0,"ondisk_log_size":7099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29052928,"num_objects":7093,"num_object_clones":0,"num_object_copies":14186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3749,"num_write_kb":14996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3259},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7298","reported_seq":7424,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:52.789032+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:52.789032+0000","last_peered":"2024-09-10T15:54:52.789032+0000","last_clean":"2024-09-10T15:54:52.789032+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:52.789032+0000","last_undegraded":"2024-09-10T15:54:52.789032+0000","last_fullsized":"2024-09-10T15:54:52.789032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7298,"log_dups_size":0,"ondisk_log_size":7298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4057,"num_write_kb":16228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7172","reported_seq":12864,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.569560+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:49.569560+0000","last_peered":"2024-09-10T15:54:49.569560+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:49.569560+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7172,"log_dups_size":0,"ondisk_log_size":7172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1208,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5149,"num_write_kb":20596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2784,"num_bytes_recovered":11395072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1208},{"shards":"2","objects":5955}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7183","reported_seq":7304,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.737601+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:52.737601+0000","last_peered":"2024-09-10T15:54:52.737601+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:52.737601+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7183,"log_dups_size":0,"ondisk_log_size":7183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29388800,"num_objects":7175,"num_object_clones":0,"num_object_copies":14350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4047,"num_write_kb":16188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3312},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7135","reported_seq":7255,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.542652+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:49.542652+0000","last_peered":"2024-09-10T15:54:49.542652+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:49.542652+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7135,"log_dups_size":0,"ondisk_log_size":7135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5109,"num_write_kb":20436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6952","reported_seq":7073,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.526194+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:49.526194+0000","last_peered":"2024-09-10T15:54:49.526194+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:49.526194+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6952,"log_dups_size":0,"ondisk_log_size":6952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28450816,"num_objects":6946,"num_object_clones":0,"num_object_copies":13892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4821,"num_write_kb":19284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3117}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7263","reported_seq":7385,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.840078+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:52.840078+0000","last_peered":"2024-09-10T15:54:52.840078+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:52.840078+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7263,"log_dups_size":0,"ondisk_log_size":7263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29741056,"num_objects":7261,"num_object_clones":0,"num_object_copies":14522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4455,"num_write_kb":17820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3428},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7257","reported_seq":7416,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.812414+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:52.812414+0000","last_peered":"2024-09-10T15:54:52.812414+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:52.812414+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7257,"log_dups_size":0,"ondisk_log_size":7257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691904,"num_objects":7249,"num_object_clones":0,"num_object_copies":14498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3675,"num_write_kb":14700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3332},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7210","reported_seq":7345,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.828222+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:52.828222+0000","last_peered":"2024-09-10T15:54:52.828222+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:52.828222+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7210,"log_dups_size":0,"ondisk_log_size":7210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29515776,"num_objects":7206,"num_object_clones":0,"num_object_copies":14412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3816,"num_write_kb":15264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3329},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7424","reported_seq":7558,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.777327+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:52.777327+0000","last_peered":"2024-09-10T15:54:52.777327+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:52.777327+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7424,"log_dups_size":0,"ondisk_log_size":7424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30360035,"num_objects":7413,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4243,"num_write_kb":16972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3410},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471593443,"num_objects":115136,"num_object_clones":0,"num_object_copies":230272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1208,"num_objects_unfound":0,"num_objects_dirty":115136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76738,"num_write_kb":306952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2786,"num_bytes_recovered":11403264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":957607936,"data_stored":957524837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115217,"ondisk_log_size":115217,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626453,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325932,"kb_used_data":172124,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":94045908,"statfs":{"total":96636764160,"available":96303009792,"internally_reserved":0,"allocated":176254976,"data_stored":175927058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157478574},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149737,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675524,"kb_used_data":409268,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93696316,"statfs":{"total":96636764160,"available":95945027584,"internally_reserved":0,"allocated":419090432,"data_stored":418834774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272625325},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527756,"kb_used_data":351740,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93844084,"statfs":{"total":96636764160,"available":96096342016,"internally_reserved":0,"allocated":360181760,"data_stored":359926102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419729408,"data_stored":419725795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":176988160,"data_stored":176912287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360890368,"data_stored":360886755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:55.738 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:55.816 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 138 20 117339 117319 3.32037 2 0.0778758 0.0235203 2024-09-10T15:54:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:56.296 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:56.329 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-10T15:54:54.467815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471593462,"num_objects":115138,"num_object_clones":0,"num_object_copies":230276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1208,"num_objects_unfound":0,"num_objects_dirty":115138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76740,"num_write_kb":306954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2788,"num_bytes_recovered":11403283,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115220,"ondisk_log_size":115220,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529212,"kb_used_data":933132,"kb_used_omap":12,"kb_used_meta":596019,"kb_avail":281586308,"statfs":{"total":289910292480,"available":288344379392,"internally_reserved":0,"allocated":955527168,"data_stored":954687934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":610323461},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,9,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28807168,"num_objects":7033,"num_object_clones":0,"num_object_copies":14066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-374,"num_objects_unfound":0,"num_objects_dirty":7033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7033,"num_write_kb":28132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":1531904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003108"},"pg_stats":[{"pgid":"2.f","version":"44'7279","reported_seq":7349,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.845516+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:52.845516+0000","last_peered":"2024-09-10T15:54:52.845516+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:52.845516+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7279,"log_dups_size":0,"ondisk_log_size":7279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29814784,"num_objects":7279,"num_object_clones":0,"num_object_copies":14558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7279,"num_write_kb":29116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3306},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7338","reported_seq":7440,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:49.530384+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:49.530384+0000","last_peered":"2024-09-10T15:54:49.530384+0000","last_clean":"2024-09-10T15:54:49.530384+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:49.530384+0000","last_undegraded":"2024-09-10T15:54:49.530384+0000","last_fullsized":"2024-09-10T15:54:49.530384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7338,"log_dups_size":0,"ondisk_log_size":7338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5674,"num_write_kb":22696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7082","reported_seq":7201,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.548059+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:49.548059+0000","last_peered":"2024-09-10T15:54:49.548059+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:49.548059+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28991488,"num_objects":7078,"num_object_clones":0,"num_object_copies":14156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5350,"num_write_kb":21400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7075","reported_seq":7150,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:49.543415+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:49.543415+0000","last_peered":"2024-09-10T15:54:49.543415+0000","last_clean":"2024-09-10T15:54:49.543415+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:49.543415+0000","last_undegraded":"2024-09-10T15:54:49.543415+0000","last_fullsized":"2024-09-10T15:54:49.543415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7075,"log_dups_size":0,"ondisk_log_size":7075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28979200,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7075,"num_write_kb":28300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7116","reported_seq":7238,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:52.810805+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:52.810805+0000","last_peered":"2024-09-10T15:54:52.810805+0000","last_clean":"2024-09-10T15:54:52.810805+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:52.810805+0000","last_undegraded":"2024-09-10T15:54:52.810805+0000","last_fullsized":"2024-09-10T15:54:52.810805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7116,"log_dups_size":0,"ondisk_log_size":7116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29118464,"num_objects":7109,"num_object_clones":0,"num_object_copies":14218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4187,"num_write_kb":16748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7334","reported_seq":7464,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.742327+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:52.742327+0000","last_peered":"2024-09-10T15:54:52.742327+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:52.742327+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7334,"log_dups_size":0,"ondisk_log_size":7334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30019584,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4052,"num_write_kb":16208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3461},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7099","reported_seq":7235,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.773121+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:52.773121+0000","last_peered":"2024-09-10T15:54:52.773121+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:52.773121+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7099,"log_dups_size":0,"ondisk_log_size":7099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29052928,"num_objects":7093,"num_object_clones":0,"num_object_copies":14186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3749,"num_write_kb":14996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3259},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7298","reported_seq":7424,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:52.789032+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:52.789032+0000","last_peered":"2024-09-10T15:54:52.789032+0000","last_clean":"2024-09-10T15:54:52.789032+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:52.789032+0000","last_undegraded":"2024-09-10T15:54:52.789032+0000","last_fullsized":"2024-09-10T15:54:52.789032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7298,"log_dups_size":0,"ondisk_log_size":7298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4057,"num_write_kb":16228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7172","reported_seq":12864,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.569560+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:49.569560+0000","last_peered":"2024-09-10T15:54:49.569560+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:49.569560+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7172,"log_dups_size":0,"ondisk_log_size":7172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1208,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5149,"num_write_kb":20596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2784,"num_bytes_recovered":11395072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1208},{"shards":"2","objects":5955}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7183","reported_seq":7304,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.737601+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:52.737601+0000","last_peered":"2024-09-10T15:54:52.737601+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:52.737601+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7183,"log_dups_size":0,"ondisk_log_size":7183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29388800,"num_objects":7175,"num_object_clones":0,"num_object_copies":14350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4047,"num_write_kb":16188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3312},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7135","reported_seq":7255,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.542652+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:49.542652+0000","last_peered":"2024-09-10T15:54:49.542652+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:49.542652+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7135,"log_dups_size":0,"ondisk_log_size":7135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5109,"num_write_kb":20436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6952","reported_seq":7073,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:49.526194+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:49.526194+0000","last_peered":"2024-09-10T15:54:49.526194+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:49.526194+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":6952,"log_dups_size":0,"ondisk_log_size":6952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28450816,"num_objects":6946,"num_object_clones":0,"num_object_copies":13892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4821,"num_write_kb":19284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3117}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7263","reported_seq":7385,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.840078+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:52.840078+0000","last_peered":"2024-09-10T15:54:52.840078+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:52.840078+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7263,"log_dups_size":0,"ondisk_log_size":7263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29741056,"num_objects":7261,"num_object_clones":0,"num_object_copies":14522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4455,"num_write_kb":17820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3428},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7257","reported_seq":7416,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.812414+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:52.812414+0000","last_peered":"2024-09-10T15:54:52.812414+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:52.812414+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7257,"log_dups_size":0,"ondisk_log_size":7257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691904,"num_objects":7249,"num_object_clones":0,"num_object_copies":14498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3675,"num_write_kb":14700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3332},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7210","reported_seq":7345,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.828222+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:52.828222+0000","last_peered":"2024-09-10T15:54:52.828222+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:52.828222+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7210,"log_dups_size":0,"ondisk_log_size":7210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29515776,"num_objects":7206,"num_object_clones":0,"num_object_copies":14412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3816,"num_write_kb":15264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3329},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7424","reported_seq":7558,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.777327+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:52.777327+0000","last_peered":"2024-09-10T15:54:52.777327+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:52.777327+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7424,"log_dups_size":0,"ondisk_log_size":7424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30360035,"num_objects":7413,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4243,"num_write_kb":16972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3410},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471593443,"num_objects":115136,"num_object_clones":0,"num_object_copies":230272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1208,"num_objects_unfound":0,"num_objects_dirty":115136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76738,"num_write_kb":306952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2786,"num_bytes_recovered":11403264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":957607936,"data_stored":957524837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115217,"ondisk_log_size":115217,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626453,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325932,"kb_used_data":172124,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":94045908,"statfs":{"total":96636764160,"available":96303009792,"internally_reserved":0,"allocated":176254976,"data_stored":175927058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157478574},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149737,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675524,"kb_used_data":409268,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93696316,"statfs":{"total":96636764160,"available":95945027584,"internally_reserved":0,"allocated":419090432,"data_stored":418834774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272625325},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527756,"kb_used_data":351740,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93844084,"statfs":{"total":96636764160,"available":96096342016,"internally_reserved":0,"allocated":360181760,"data_stored":359926102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419729408,"data_stored":419725795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":176988160,"data_stored":176912287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360890368,"data_stored":360886755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:56.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:54:56.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'7279", 'reported_seq': 7349, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:52.845516+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:54:52.845516+0000', 'last_peered': '2024-09-10T15:54:52.845516+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:54:52.845516+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 7279, 'log_dups_size': 0, 'ondisk_log_size': 7279, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29814784, 'num_objects': 7279, 'num_object_clones': 0, 'num_object_copies': 14558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7279, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7279, 'num_write_kb': 29116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3306}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:56.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:54:56.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'7082", 'reported_seq': 7201, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:49.548059+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:54:49.548059+0000', 'last_peered': '2024-09-10T15:54:49.548059+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:54:49.548059+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 7082, 'log_dups_size': 0, 'ondisk_log_size': 7082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28991488, 'num_objects': 7078, 'num_object_clones': 0, 'num_object_copies': 14156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7078, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5350, 'num_write_kb': 21400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 3276}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:56.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:54:56.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'7334", 'reported_seq': 7464, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:52.742327+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:54:52.742327+0000', 'last_peered': '2024-09-10T15:54:52.742327+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:54:52.742327+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 7334, 'log_dups_size': 0, 'ondisk_log_size': 7334, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30019584, 'num_objects': 7329, 'num_object_clones': 0, 'num_object_copies': 14658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7329, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4052, 'num_write_kb': 16208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3461}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:56.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:54:56.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'7099", 'reported_seq': 7235, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:52.773121+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:54:52.773121+0000', 'last_peered': '2024-09-10T15:54:52.773121+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:54:52.773121+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 7099, 'log_dups_size': 0, 'ondisk_log_size': 7099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29052928, 'num_objects': 7093, 'num_object_clones': 0, 'num_object_copies': 14186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7093, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3749, 'num_write_kb': 14996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3259}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:56.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:54:56.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'7172", 'reported_seq': 12864, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:49.569560+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:54:49.569560+0000', 'last_peered': '2024-09-10T15:54:49.569560+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:54:49.569560+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 7172, 'log_dups_size': 0, 'ondisk_log_size': 7172, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29347840, 'num_objects': 7165, 'num_object_clones': 0, 'num_object_copies': 14330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1208, 'num_objects_unfound': 0, 'num_objects_dirty': 7165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5149, 'num_write_kb': 20596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2784, 'num_bytes_recovered': 11395072, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1208}, {'shards': '2', 'objects': 5955}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:56.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:54:56.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'7183", 'reported_seq': 7304, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:52.737601+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:54:52.737601+0000', 'last_peered': '2024-09-10T15:54:52.737601+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:54:52.737601+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 7183, 'log_dups_size': 0, 'ondisk_log_size': 7183, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29388800, 'num_objects': 7175, 'num_object_clones': 0, 'num_object_copies': 14350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7175, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4047, 'num_write_kb': 16188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3312}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:56.334 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:54:56.334 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'7135", 'reported_seq': 7255, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:49.542652+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:54:49.542652+0000', 'last_peered': '2024-09-10T15:54:49.542652+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:54:49.542652+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 7135, 'log_dups_size': 0, 'ondisk_log_size': 7135, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29204480, 'num_objects': 7130, 'num_object_clones': 0, 'num_object_copies': 14260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5109, 'num_write_kb': 20436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 3298}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:56.334 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:54:56.334 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'6952", 'reported_seq': 7073, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:49.526194+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:54:49.526194+0000', 'last_peered': '2024-09-10T15:54:49.526194+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:54:49.526194+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 6952, 'log_dups_size': 0, 'ondisk_log_size': 6952, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28450816, 'num_objects': 6946, 'num_object_clones': 0, 'num_object_copies': 13892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6946, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4821, 'num_write_kb': 19284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 3117}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:54:56.334 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:54:56.334 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'7263", 'reported_seq': 7385, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:52.840078+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:54:52.840078+0000', 'last_peered': '2024-09-10T15:54:52.840078+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:54:52.840078+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 7263, 'log_dups_size': 0, 'ondisk_log_size': 7263, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29741056, 'num_objects': 7261, 'num_object_clones': 0, 'num_object_copies': 14522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7261, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4455, 'num_write_kb': 17820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3428}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:56.334 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:54:56.334 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'7257", 'reported_seq': 7416, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:52.812414+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:54:52.812414+0000', 'last_peered': '2024-09-10T15:54:52.812414+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:54:52.812414+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 7257, 'log_dups_size': 0, 'ondisk_log_size': 7257, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29691904, 'num_objects': 7249, 'num_object_clones': 0, 'num_object_copies': 14498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7249, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3675, 'num_write_kb': 14700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3332}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:56.334 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:54:56.334 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'7210", 'reported_seq': 7345, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:52.828222+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:54:52.828222+0000', 'last_peered': '2024-09-10T15:54:52.828222+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:54:52.828222+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 7210, 'log_dups_size': 0, 'ondisk_log_size': 7210, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29515776, 'num_objects': 7206, 'num_object_clones': 0, 'num_object_copies': 14412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7206, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3816, 'num_write_kb': 15264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3329}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:56.335 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:54:56.335 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'7424", 'reported_seq': 7558, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:52.777327+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:54:52.777327+0000', 'last_peered': '2024-09-10T15:54:52.777327+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:54:52.777327+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 7424, 'log_dups_size': 0, 'ondisk_log_size': 7424, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30360035, 'num_objects': 7413, 'num_object_clones': 0, 'num_object_copies': 14826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7413, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4243, 'num_write_kb': 16972, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3410}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:54:56.335 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:54:56.816 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 139 20 117835 117815 3.31042 1.9375 0.0830911 0.0235906 2024-09-10T15:54:57.037 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:57.037 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/230276 objects degraded (18.523%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":222,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":115138,"data_bytes":471593462,"bytes_used":1565913088,"bytes_avail":288344379392,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":230276,"degraded_ratio":0.18522555542045199,"misplaced_objects":1208,"misplaced_total":230276,"misplaced_ratio":0.0052458788584133819,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":127625,"recovering_keys_per_sec":0,"num_objects_recovered":374,"num_bytes_recovered":1531904,"num_keys_recovered":0,"write_bytes_sec":2399975,"read_op_per_sec":0,"write_op_per_sec":585},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-10T15:54:30.461931+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:54:57.061 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:54:57.062 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:54:57.610 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:54:57.610 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:54:57.634 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-10T15:54:56.468330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":475529718,"num_objects":116099,"num_object_clones":0,"num_object_copies":232198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1007,"num_objects_unfound":0,"num_objects_dirty":116099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77701,"num_write_kb":310798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2989,"num_bytes_recovered":12226579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116181,"ondisk_log_size":116181,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538592,"kb_used_data":938352,"kb_used_omap":12,"kb_used_meta":600179,"kb_avail":281576928,"statfs":{"total":289910292480,"available":288334774272,"internally_reserved":0,"allocated":960872448,"data_stored":960033214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":614583301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,8,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25960448,"num_objects":6338,"num_object_clones":0,"num_object_copies":12676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-575,"num_objects_unfound":0,"num_objects_dirty":6338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6338,"num_write_kb":25352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":575,"num_bytes_recovered":2355200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003160"},"pg_stats":[{"pgid":"2.f","version":"44'7279","reported_seq":7349,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.845516+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:52.845516+0000","last_peered":"2024-09-10T15:54:52.845516+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:52.845516+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7279,"log_dups_size":0,"ondisk_log_size":7279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29814784,"num_objects":7279,"num_object_clones":0,"num_object_copies":14558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7279,"num_write_kb":29116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3306},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7508","reported_seq":7610,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:54.501363+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:54.501363+0000","last_peered":"2024-09-10T15:54:54.501363+0000","last_clean":"2024-09-10T15:54:54.501363+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:54.501363+0000","last_undegraded":"2024-09-10T15:54:54.501363+0000","last_fullsized":"2024-09-10T15:54:54.501363+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7508,"log_dups_size":0,"ondisk_log_size":7508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30728192,"num_objects":7502,"num_object_clones":0,"num_object_copies":15004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5844,"num_write_kb":23376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7268","reported_seq":7387,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:54.567348+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:54.567348+0000","last_peered":"2024-09-10T15:54:54.567348+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:54.567348+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7268,"log_dups_size":0,"ondisk_log_size":7268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29753344,"num_objects":7264,"num_object_clones":0,"num_object_copies":14528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5536,"num_write_kb":22144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3462}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7234","reported_seq":7309,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:54.518162+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:54.518162+0000","last_peered":"2024-09-10T15:54:54.518162+0000","last_clean":"2024-09-10T15:54:54.518162+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:54.518162+0000","last_undegraded":"2024-09-10T15:54:54.518162+0000","last_fullsized":"2024-09-10T15:54:54.518162+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7234,"log_dups_size":0,"ondisk_log_size":7234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29630464,"num_objects":7234,"num_object_clones":0,"num_object_copies":14468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7234,"num_write_kb":28936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7116","reported_seq":7238,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:52.810805+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:52.810805+0000","last_peered":"2024-09-10T15:54:52.810805+0000","last_clean":"2024-09-10T15:54:52.810805+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:52.810805+0000","last_undegraded":"2024-09-10T15:54:52.810805+0000","last_fullsized":"2024-09-10T15:54:52.810805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7116,"log_dups_size":0,"ondisk_log_size":7116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29118464,"num_objects":7109,"num_object_clones":0,"num_object_copies":14218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4187,"num_write_kb":16748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7334","reported_seq":7464,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.742327+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:52.742327+0000","last_peered":"2024-09-10T15:54:52.742327+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:52.742327+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7334,"log_dups_size":0,"ondisk_log_size":7334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30019584,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4052,"num_write_kb":16208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3461},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7099","reported_seq":7235,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.773121+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:52.773121+0000","last_peered":"2024-09-10T15:54:52.773121+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:52.773121+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7099,"log_dups_size":0,"ondisk_log_size":7099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29052928,"num_objects":7093,"num_object_clones":0,"num_object_copies":14186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3749,"num_write_kb":14996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3259},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7298","reported_seq":7424,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:52.789032+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:52.789032+0000","last_peered":"2024-09-10T15:54:52.789032+0000","last_clean":"2024-09-10T15:54:52.789032+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:52.789032+0000","last_undegraded":"2024-09-10T15:54:52.789032+0000","last_fullsized":"2024-09-10T15:54:52.789032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7298,"log_dups_size":0,"ondisk_log_size":7298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4057,"num_write_kb":16228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7301","reported_seq":13396,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:54.580360+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:54.580360+0000","last_peered":"2024-09-10T15:54:54.580360+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:54.580360+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7301,"log_dups_size":0,"ondisk_log_size":7301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29876224,"num_objects":7294,"num_object_clones":0,"num_object_copies":14588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1007,"num_objects_unfound":0,"num_objects_dirty":7294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5278,"num_write_kb":21112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2985,"num_bytes_recovered":12218368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1007},{"shards":"2","objects":6285}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7183","reported_seq":7304,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.737601+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:52.737601+0000","last_peered":"2024-09-10T15:54:52.737601+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:52.737601+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7183,"log_dups_size":0,"ondisk_log_size":7183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29388800,"num_objects":7175,"num_object_clones":0,"num_object_copies":14350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4047,"num_write_kb":16188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3312},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7297","reported_seq":7417,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:54.554741+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:54.554741+0000","last_peered":"2024-09-10T15:54:54.554741+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:54.554741+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7297,"log_dups_size":0,"ondisk_log_size":7297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29868032,"num_objects":7292,"num_object_clones":0,"num_object_copies":14584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5271,"num_write_kb":21084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3460}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7107","reported_seq":7228,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:54.560481+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:54.560481+0000","last_peered":"2024-09-10T15:54:54.560481+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:54.560481+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7107,"log_dups_size":0,"ondisk_log_size":7107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4976,"num_write_kb":19904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3272}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7263","reported_seq":7385,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.840078+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:52.840078+0000","last_peered":"2024-09-10T15:54:52.840078+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:52.840078+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7263,"log_dups_size":0,"ondisk_log_size":7263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29741056,"num_objects":7261,"num_object_clones":0,"num_object_copies":14522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4455,"num_write_kb":17820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3428},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7257","reported_seq":7416,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.812414+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:52.812414+0000","last_peered":"2024-09-10T15:54:52.812414+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:52.812414+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7257,"log_dups_size":0,"ondisk_log_size":7257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691904,"num_objects":7249,"num_object_clones":0,"num_object_copies":14498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3675,"num_write_kb":14700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3332},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7210","reported_seq":7345,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.828222+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:52.828222+0000","last_peered":"2024-09-10T15:54:52.828222+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:52.828222+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7210,"log_dups_size":0,"ondisk_log_size":7210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29515776,"num_objects":7206,"num_object_clones":0,"num_object_copies":14412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3816,"num_write_kb":15264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3329},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7424","reported_seq":7558,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:52.777327+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:52.777327+0000","last_peered":"2024-09-10T15:54:52.777327+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:52.777327+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7424,"log_dups_size":0,"ondisk_log_size":7424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30360035,"num_objects":7413,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4243,"num_write_kb":16972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3410},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":475529699,"num_objects":116097,"num_object_clones":0,"num_object_copies":232194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":1007,"num_objects_unfound":0,"num_objects_dirty":116097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77699,"num_write_kb":310796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2987,"num_bytes_recovered":12226560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":962777088,"data_stored":962693989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116178,"ondisk_log_size":116178,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626453,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325932,"kb_used_data":172124,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":94045908,"statfs":{"total":96636764160,"available":96303009792,"internally_reserved":0,"allocated":176254976,"data_stored":175927058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157478574},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149737,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675524,"kb_used_data":409268,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93696316,"statfs":{"total":96636764160,"available":95945027584,"internally_reserved":0,"allocated":419090432,"data_stored":418834774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272625325},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953507,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537136,"kb_used_data":356960,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93834704,"statfs":{"total":96636764160,"available":96086736896,"internally_reserved":0,"allocated":365527040,"data_stored":365271382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419729408,"data_stored":419725795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":176988160,"data_stored":176912287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":366059520,"data_stored":366055907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:54:57.816 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:2024-09-10T15:54:57.816160+0000 min lat: 0.00198913 max lat: 2.66872 avg lat: 0.0236623 2024-09-10T15:54:57.816 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:54:57.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 140 20 118329 118309 3.30056 1.92969 0.0200479 0.0236623 2024-09-10T15:54:58.816 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 141 20 118883 118863 3.2925 2.16406 0.00905819 0.0237194 2024-09-10T15:54:59.816 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 142 20 119432 119412 3.28441 2.14453 0.0175763 0.0237785 2024-09-10T15:55:00.636 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:00.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 143 20 119977 119957 3.27633 2.12891 0.0476879 0.0238375 2024-09-10T15:55:01.188 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:01.189 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:01.216 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-10T15:55:00.469375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":486281718,"num_objects":118724,"num_object_clones":0,"num_object_copies":237448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":821,"num_objects_unfound":0,"num_objects_dirty":118724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80326,"num_write_kb":321298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3175,"num_bytes_recovered":12988435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118806,"ondisk_log_size":118806,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1572824,"kb_used_data":960104,"kb_used_omap":12,"kb_used_meta":612659,"kb_avail":281542696,"statfs":{"total":289910292480,"available":288299720704,"internally_reserved":0,"allocated":983146496,"data_stored":982307262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":627362821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,5,6,14,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25509888,"num_objects":6228,"num_object_clones":0,"num_object_copies":12456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-574,"num_objects_unfound":0,"num_objects_dirty":6228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6228,"num_write_kb":24912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":574,"num_bytes_recovered":2351104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003155"},"pg_stats":[{"pgid":"2.f","version":"44'7449","reported_seq":7520,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.959511+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:57.959511+0000","last_peered":"2024-09-10T15:54:57.959511+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:57.959511+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7449,"log_dups_size":0,"ondisk_log_size":7449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30511104,"num_objects":7449,"num_object_clones":0,"num_object_copies":14898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7449,"num_write_kb":29796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3476},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7661","reported_seq":7763,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:59.320186+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:59.320186+0000","last_peered":"2024-09-10T15:54:59.320186+0000","last_clean":"2024-09-10T15:54:59.320186+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:59.320186+0000","last_undegraded":"2024-09-10T15:54:59.320186+0000","last_fullsized":"2024-09-10T15:54:59.320186+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7661,"log_dups_size":0,"ondisk_log_size":7661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31354880,"num_objects":7655,"num_object_clones":0,"num_object_copies":15310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5997,"num_write_kb":23988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7425","reported_seq":7544,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:59.575611+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:59.575611+0000","last_peered":"2024-09-10T15:54:59.575611+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:59.575611+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7425,"log_dups_size":0,"ondisk_log_size":7425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30396416,"num_objects":7421,"num_object_clones":0,"num_object_copies":14842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5693,"num_write_kb":22772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3619}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7412","reported_seq":7487,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:59.604049+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:59.604049+0000","last_peered":"2024-09-10T15:54:59.604049+0000","last_clean":"2024-09-10T15:54:59.604049+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:59.604049+0000","last_undegraded":"2024-09-10T15:54:59.604049+0000","last_fullsized":"2024-09-10T15:54:59.604049+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7412,"log_dups_size":0,"ondisk_log_size":7412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30359552,"num_objects":7412,"num_object_clones":0,"num_object_copies":14824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7412,"num_write_kb":29648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7259","reported_seq":7381,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:57.887361+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:57.887361+0000","last_peered":"2024-09-10T15:54:57.887361+0000","last_clean":"2024-09-10T15:54:57.887361+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:57.887361+0000","last_undegraded":"2024-09-10T15:54:57.887361+0000","last_fullsized":"2024-09-10T15:54:57.887361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7259,"log_dups_size":0,"ondisk_log_size":7259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29704192,"num_objects":7252,"num_object_clones":0,"num_object_copies":14504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4330,"num_write_kb":17320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7511","reported_seq":7641,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.884900+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:57.884900+0000","last_peered":"2024-09-10T15:54:57.884900+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:57.884900+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7511,"log_dups_size":0,"ondisk_log_size":7511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30744576,"num_objects":7506,"num_object_clones":0,"num_object_copies":15012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4229,"num_write_kb":16916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3638},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7266","reported_seq":7402,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.932697+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:57.932697+0000","last_peered":"2024-09-10T15:54:57.932697+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:57.932697+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7266,"log_dups_size":0,"ondisk_log_size":7266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29736960,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3916,"num_write_kb":15664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3426},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7431","reported_seq":7557,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:57.866297+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:57.866297+0000","last_peered":"2024-09-10T15:54:57.866297+0000","last_clean":"2024-09-10T15:54:57.866297+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:57.866297+0000","last_undegraded":"2024-09-10T15:54:57.866297+0000","last_fullsized":"2024-09-10T15:54:57.866297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7431,"log_dups_size":0,"ondisk_log_size":7431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30429184,"num_objects":7429,"num_object_clones":0,"num_object_copies":14858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4190,"num_write_kb":16760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7474","reported_seq":13940,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:59.601982+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:59.601982+0000","last_peered":"2024-09-10T15:54:59.601982+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:59.601982+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7474,"log_dups_size":0,"ondisk_log_size":7474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30584832,"num_objects":7467,"num_object_clones":0,"num_object_copies":14934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":821,"num_objects_unfound":0,"num_objects_dirty":7467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5451,"num_write_kb":21804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3171,"num_bytes_recovered":12980224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":821},{"shards":"2","objects":6644}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7340","reported_seq":7462,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.855577+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:57.855577+0000","last_peered":"2024-09-10T15:54:57.855577+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:57.855577+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7340,"log_dups_size":0,"ondisk_log_size":7340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4204,"num_write_kb":16816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3469},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7440","reported_seq":7560,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:59.509629+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:59.509629+0000","last_peered":"2024-09-10T15:54:59.509629+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:59.509629+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7440,"log_dups_size":0,"ondisk_log_size":7440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30453760,"num_objects":7435,"num_object_clones":0,"num_object_copies":14870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5414,"num_write_kb":21656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3603}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7311","reported_seq":7433,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:59.589627+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:59.589627+0000","last_peered":"2024-09-10T15:54:59.589627+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:59.589627+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7311,"log_dups_size":0,"ondisk_log_size":7311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29921280,"num_objects":7305,"num_object_clones":0,"num_object_copies":14610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5180,"num_write_kb":20720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3476}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7438","reported_seq":7560,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.923318+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:57.923318+0000","last_peered":"2024-09-10T15:54:57.923318+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:57.923318+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7438,"log_dups_size":0,"ondisk_log_size":7438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30457856,"num_objects":7436,"num_object_clones":0,"num_object_copies":14872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4630,"num_write_kb":18520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3603},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7433","reported_seq":7587,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.932186+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:57.932186+0000","last_peered":"2024-09-10T15:54:57.932186+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:57.932186+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7433,"log_dups_size":0,"ondisk_log_size":7433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30412800,"num_objects":7425,"num_object_clones":0,"num_object_copies":14850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3851,"num_write_kb":15404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3508},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7384","reported_seq":7519,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.924191+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:57.924191+0000","last_peered":"2024-09-10T15:54:57.924191+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:57.924191+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7384,"log_dups_size":0,"ondisk_log_size":7384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30228480,"num_objects":7380,"num_object_clones":0,"num_object_copies":14760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3990,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3503},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7569","reported_seq":7703,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.914578+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:57.914578+0000","last_peered":"2024-09-10T15:54:57.914578+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:57.914578+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7569,"log_dups_size":0,"ondisk_log_size":7569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30953955,"num_objects":7558,"num_object_clones":0,"num_object_copies":15116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4388,"num_write_kb":17552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3555},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":486281699,"num_objects":118722,"num_object_clones":0,"num_object_copies":237444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":821,"num_objects_unfound":0,"num_objects_dirty":118722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80324,"num_write_kb":321296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3173,"num_bytes_recovered":12988416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":985202688,"data_stored":985119589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118803,"ondisk_log_size":118803,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":338808,"kb_used_data":180840,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":94033032,"statfs":{"total":96636764160,"available":96289824768,"internally_reserved":0,"allocated":185180160,"data_stored":184852242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161738414},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,2,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149738,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687456,"kb_used_data":417040,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93684384,"statfs":{"total":96636764160,"available":95932809216,"internally_reserved":0,"allocated":427048960,"data_stored":426793302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276885165},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546560,"kb_used_data":362224,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93825280,"statfs":{"total":96636764160,"available":96077086720,"internally_reserved":0,"allocated":370917376,"data_stored":370661718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":427806720,"data_stored":427803107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":186028032,"data_stored":185952159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":371367936,"data_stored":371364323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:01.217 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:01.773 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:01.774 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:01.803 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-10T15:55:00.469375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":486281718,"num_objects":118724,"num_object_clones":0,"num_object_copies":237448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":821,"num_objects_unfound":0,"num_objects_dirty":118724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80326,"num_write_kb":321298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3175,"num_bytes_recovered":12988435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118806,"ondisk_log_size":118806,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1572824,"kb_used_data":960104,"kb_used_omap":12,"kb_used_meta":612659,"kb_avail":281542696,"statfs":{"total":289910292480,"available":288299720704,"internally_reserved":0,"allocated":983146496,"data_stored":982307262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":627362821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,5,6,14,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25509888,"num_objects":6228,"num_object_clones":0,"num_object_copies":12456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-574,"num_objects_unfound":0,"num_objects_dirty":6228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6228,"num_write_kb":24912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":574,"num_bytes_recovered":2351104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003155"},"pg_stats":[{"pgid":"2.f","version":"44'7449","reported_seq":7520,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.959511+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:57.959511+0000","last_peered":"2024-09-10T15:54:57.959511+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:57.959511+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7449,"log_dups_size":0,"ondisk_log_size":7449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30511104,"num_objects":7449,"num_object_clones":0,"num_object_copies":14898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7449,"num_write_kb":29796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3476},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7661","reported_seq":7763,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:59.320186+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:59.320186+0000","last_peered":"2024-09-10T15:54:59.320186+0000","last_clean":"2024-09-10T15:54:59.320186+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:59.320186+0000","last_undegraded":"2024-09-10T15:54:59.320186+0000","last_fullsized":"2024-09-10T15:54:59.320186+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7661,"log_dups_size":0,"ondisk_log_size":7661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31354880,"num_objects":7655,"num_object_clones":0,"num_object_copies":15310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5997,"num_write_kb":23988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7425","reported_seq":7544,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:59.575611+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:59.575611+0000","last_peered":"2024-09-10T15:54:59.575611+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:59.575611+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7425,"log_dups_size":0,"ondisk_log_size":7425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30396416,"num_objects":7421,"num_object_clones":0,"num_object_copies":14842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5693,"num_write_kb":22772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3619}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7412","reported_seq":7487,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:59.604049+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:59.604049+0000","last_peered":"2024-09-10T15:54:59.604049+0000","last_clean":"2024-09-10T15:54:59.604049+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:59.604049+0000","last_undegraded":"2024-09-10T15:54:59.604049+0000","last_fullsized":"2024-09-10T15:54:59.604049+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7412,"log_dups_size":0,"ondisk_log_size":7412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30359552,"num_objects":7412,"num_object_clones":0,"num_object_copies":14824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7412,"num_write_kb":29648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7259","reported_seq":7381,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:57.887361+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:57.887361+0000","last_peered":"2024-09-10T15:54:57.887361+0000","last_clean":"2024-09-10T15:54:57.887361+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:57.887361+0000","last_undegraded":"2024-09-10T15:54:57.887361+0000","last_fullsized":"2024-09-10T15:54:57.887361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7259,"log_dups_size":0,"ondisk_log_size":7259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29704192,"num_objects":7252,"num_object_clones":0,"num_object_copies":14504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4330,"num_write_kb":17320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7511","reported_seq":7641,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.884900+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:57.884900+0000","last_peered":"2024-09-10T15:54:57.884900+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:57.884900+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7511,"log_dups_size":0,"ondisk_log_size":7511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30744576,"num_objects":7506,"num_object_clones":0,"num_object_copies":15012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4229,"num_write_kb":16916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3638},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7266","reported_seq":7402,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.932697+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:57.932697+0000","last_peered":"2024-09-10T15:54:57.932697+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:57.932697+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7266,"log_dups_size":0,"ondisk_log_size":7266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29736960,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3916,"num_write_kb":15664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3426},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7431","reported_seq":7557,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:57.866297+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:57.866297+0000","last_peered":"2024-09-10T15:54:57.866297+0000","last_clean":"2024-09-10T15:54:57.866297+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:57.866297+0000","last_undegraded":"2024-09-10T15:54:57.866297+0000","last_fullsized":"2024-09-10T15:54:57.866297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7431,"log_dups_size":0,"ondisk_log_size":7431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30429184,"num_objects":7429,"num_object_clones":0,"num_object_copies":14858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4190,"num_write_kb":16760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7474","reported_seq":13940,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:59.601982+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:59.601982+0000","last_peered":"2024-09-10T15:54:59.601982+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:59.601982+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7474,"log_dups_size":0,"ondisk_log_size":7474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30584832,"num_objects":7467,"num_object_clones":0,"num_object_copies":14934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":821,"num_objects_unfound":0,"num_objects_dirty":7467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5451,"num_write_kb":21804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3171,"num_bytes_recovered":12980224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":821},{"shards":"2","objects":6644}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7340","reported_seq":7462,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.855577+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:57.855577+0000","last_peered":"2024-09-10T15:54:57.855577+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:57.855577+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7340,"log_dups_size":0,"ondisk_log_size":7340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4204,"num_write_kb":16816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3469},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7440","reported_seq":7560,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:59.509629+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:59.509629+0000","last_peered":"2024-09-10T15:54:59.509629+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:59.509629+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7440,"log_dups_size":0,"ondisk_log_size":7440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30453760,"num_objects":7435,"num_object_clones":0,"num_object_copies":14870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5414,"num_write_kb":21656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3603}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7311","reported_seq":7433,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:59.589627+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:59.589627+0000","last_peered":"2024-09-10T15:54:59.589627+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:59.589627+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7311,"log_dups_size":0,"ondisk_log_size":7311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29921280,"num_objects":7305,"num_object_clones":0,"num_object_copies":14610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5180,"num_write_kb":20720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3476}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7438","reported_seq":7560,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.923318+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:57.923318+0000","last_peered":"2024-09-10T15:54:57.923318+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:57.923318+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7438,"log_dups_size":0,"ondisk_log_size":7438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30457856,"num_objects":7436,"num_object_clones":0,"num_object_copies":14872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4630,"num_write_kb":18520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3603},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7433","reported_seq":7587,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.932186+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:57.932186+0000","last_peered":"2024-09-10T15:54:57.932186+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:57.932186+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7433,"log_dups_size":0,"ondisk_log_size":7433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30412800,"num_objects":7425,"num_object_clones":0,"num_object_copies":14850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3851,"num_write_kb":15404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3508},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7384","reported_seq":7519,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.924191+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:57.924191+0000","last_peered":"2024-09-10T15:54:57.924191+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:57.924191+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7384,"log_dups_size":0,"ondisk_log_size":7384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30228480,"num_objects":7380,"num_object_clones":0,"num_object_copies":14760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3990,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3503},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7569","reported_seq":7703,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.914578+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:57.914578+0000","last_peered":"2024-09-10T15:54:57.914578+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:57.914578+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7569,"log_dups_size":0,"ondisk_log_size":7569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30953955,"num_objects":7558,"num_object_clones":0,"num_object_copies":15116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4388,"num_write_kb":17552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3555},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":486281699,"num_objects":118722,"num_object_clones":0,"num_object_copies":237444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":821,"num_objects_unfound":0,"num_objects_dirty":118722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80324,"num_write_kb":321296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3173,"num_bytes_recovered":12988416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":985202688,"data_stored":985119589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118803,"ondisk_log_size":118803,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":338808,"kb_used_data":180840,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":94033032,"statfs":{"total":96636764160,"available":96289824768,"internally_reserved":0,"allocated":185180160,"data_stored":184852242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161738414},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,2,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149738,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687456,"kb_used_data":417040,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93684384,"statfs":{"total":96636764160,"available":95932809216,"internally_reserved":0,"allocated":427048960,"data_stored":426793302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276885165},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546560,"kb_used_data":362224,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93825280,"statfs":{"total":96636764160,"available":96077086720,"internally_reserved":0,"allocated":370917376,"data_stored":370661718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":427806720,"data_stored":427803107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":186028032,"data_stored":185952159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":371367936,"data_stored":371364323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:01.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:55:01.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'7449", 'reported_seq': 7520, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:57.959511+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:54:57.959511+0000', 'last_peered': '2024-09-10T15:54:57.959511+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:54:57.959511+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 7449, 'log_dups_size': 0, 'ondisk_log_size': 7449, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30511104, 'num_objects': 7449, 'num_object_clones': 0, 'num_object_copies': 14898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7449, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7449, 'num_write_kb': 29796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3476}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:01.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:55:01.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'7425", 'reported_seq': 7544, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:59.575611+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:54:59.575611+0000', 'last_peered': '2024-09-10T15:54:59.575611+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:54:59.575611+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 7425, 'log_dups_size': 0, 'ondisk_log_size': 7425, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30396416, 'num_objects': 7421, 'num_object_clones': 0, 'num_object_copies': 14842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7421, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5693, 'num_write_kb': 22772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 3619}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:01.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:55:01.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'7511", 'reported_seq': 7641, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:57.884900+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:54:57.884900+0000', 'last_peered': '2024-09-10T15:54:57.884900+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:54:57.884900+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 7511, 'log_dups_size': 0, 'ondisk_log_size': 7511, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30744576, 'num_objects': 7506, 'num_object_clones': 0, 'num_object_copies': 15012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7506, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4229, 'num_write_kb': 16916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3638}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:01.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:55:01.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'7266", 'reported_seq': 7402, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:57.932697+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:54:57.932697+0000', 'last_peered': '2024-09-10T15:54:57.932697+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:54:57.932697+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 7266, 'log_dups_size': 0, 'ondisk_log_size': 7266, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29736960, 'num_objects': 7260, 'num_object_clones': 0, 'num_object_copies': 14520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3916, 'num_write_kb': 15664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3426}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:01.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:55:01.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'7474", 'reported_seq': 13940, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:59.601982+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:54:59.601982+0000', 'last_peered': '2024-09-10T15:54:59.601982+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:54:59.601982+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 7474, 'log_dups_size': 0, 'ondisk_log_size': 7474, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30584832, 'num_objects': 7467, 'num_object_clones': 0, 'num_object_copies': 14934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 821, 'num_objects_unfound': 0, 'num_objects_dirty': 7467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5451, 'num_write_kb': 21804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3171, 'num_bytes_recovered': 12980224, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 821}, {'shards': '2', 'objects': 6644}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:01.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:55:01.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'7340", 'reported_seq': 7462, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:57.855577+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:54:57.855577+0000', 'last_peered': '2024-09-10T15:54:57.855577+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:54:57.855577+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 7340, 'log_dups_size': 0, 'ondisk_log_size': 7340, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30031872, 'num_objects': 7332, 'num_object_clones': 0, 'num_object_copies': 14664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7332, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4204, 'num_write_kb': 16816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3469}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:01.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:55:01.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'7440", 'reported_seq': 7560, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:59.509629+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:54:59.509629+0000', 'last_peered': '2024-09-10T15:54:59.509629+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:54:59.509629+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 7440, 'log_dups_size': 0, 'ondisk_log_size': 7440, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30453760, 'num_objects': 7435, 'num_object_clones': 0, 'num_object_copies': 14870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7435, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5414, 'num_write_kb': 21656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 3603}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:01.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:55:01.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'7311", 'reported_seq': 7433, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:59.589627+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:54:59.589627+0000', 'last_peered': '2024-09-10T15:54:59.589627+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:54:59.589627+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 7311, 'log_dups_size': 0, 'ondisk_log_size': 7311, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29921280, 'num_objects': 7305, 'num_object_clones': 0, 'num_object_copies': 14610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7305, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5180, 'num_write_kb': 20720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 3476}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:01.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:55:01.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'7438", 'reported_seq': 7560, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:57.923318+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:54:57.923318+0000', 'last_peered': '2024-09-10T15:54:57.923318+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:54:57.923318+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 7438, 'log_dups_size': 0, 'ondisk_log_size': 7438, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30457856, 'num_objects': 7436, 'num_object_clones': 0, 'num_object_copies': 14872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7436, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4630, 'num_write_kb': 18520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3603}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:01.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:55:01.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'7433", 'reported_seq': 7587, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:57.932186+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:54:57.932186+0000', 'last_peered': '2024-09-10T15:54:57.932186+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:54:57.932186+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 7433, 'log_dups_size': 0, 'ondisk_log_size': 7433, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30412800, 'num_objects': 7425, 'num_object_clones': 0, 'num_object_copies': 14850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7425, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3851, 'num_write_kb': 15404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3508}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:01.807 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:55:01.807 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'7384", 'reported_seq': 7519, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:57.924191+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:54:57.924191+0000', 'last_peered': '2024-09-10T15:54:57.924191+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:54:57.924191+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 7384, 'log_dups_size': 0, 'ondisk_log_size': 7384, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30228480, 'num_objects': 7380, 'num_object_clones': 0, 'num_object_copies': 14760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7380, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3990, 'num_write_kb': 15960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3503}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:01.807 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:55:01.807 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'7569", 'reported_seq': 7703, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:54:57.914578+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:54:57.914578+0000', 'last_peered': '2024-09-10T15:54:57.914578+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:54:57.914578+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 7569, 'log_dups_size': 0, 'ondisk_log_size': 7569, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30953955, 'num_objects': 7558, 'num_object_clones': 0, 'num_object_copies': 15116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7558, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4388, 'num_write_kb': 17552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3555}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:01.807 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:55:01.816 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 144 20 120452 120432 3.26646 1.85547 0.0654171 0.0239105 2024-09-10T15:55:02.501 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:02.501 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/237448 objects degraded (17.963%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":228,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":118724,"data_bytes":486281718,"bytes_used":1610571776,"bytes_avail":288299720704,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":237448,"degraded_ratio":0.17963090866210707,"misplaced_objects":821,"misplaced_total":237448,"misplaced_ratio":0.0034575991374953675,"recovering_objects_per_sec":47,"recovering_bytes_per_sec":195873,"recovering_keys_per_sec":0,"num_objects_recovered":574,"num_bytes_recovered":2351104,"num_keys_recovered":0,"write_bytes_sec":2125265,"read_op_per_sec":0,"write_op_per_sec":518},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:55:02.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:55:02.524 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:02.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 145 20 120942 120922 3.25714 1.91406 0.0713372 0.0239775 2024-09-10T15:55:03.119 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:03.119 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:03.145 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-10T15:55:02.470649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":486281718,"num_objects":118724,"num_object_clones":0,"num_object_copies":237448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":821,"num_objects_unfound":0,"num_objects_dirty":118724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80326,"num_write_kb":321298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3175,"num_bytes_recovered":12988435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118806,"ondisk_log_size":118806,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1572824,"kb_used_data":960104,"kb_used_omap":12,"kb_used_meta":612659,"kb_avail":281542696,"statfs":{"total":289910292480,"available":288299720704,"internally_reserved":0,"allocated":983146496,"data_stored":982307262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":627362821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,5,6,14,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21426176,"num_objects":5231,"num_object_clones":0,"num_object_copies":10462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-387,"num_objects_unfound":0,"num_objects_dirty":5231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5231,"num_write_kb":20924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":1585152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003927"},"pg_stats":[{"pgid":"2.f","version":"44'7449","reported_seq":7520,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.959511+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:54:57.959511+0000","last_peered":"2024-09-10T15:54:57.959511+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:54:57.959511+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7449,"log_dups_size":0,"ondisk_log_size":7449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30511104,"num_objects":7449,"num_object_clones":0,"num_object_copies":14898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7449,"num_write_kb":29796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3476},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7661","reported_seq":7763,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:59.320186+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:54:59.320186+0000","last_peered":"2024-09-10T15:54:59.320186+0000","last_clean":"2024-09-10T15:54:59.320186+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:54:59.320186+0000","last_undegraded":"2024-09-10T15:54:59.320186+0000","last_fullsized":"2024-09-10T15:54:59.320186+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7661,"log_dups_size":0,"ondisk_log_size":7661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31354880,"num_objects":7655,"num_object_clones":0,"num_object_copies":15310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5997,"num_write_kb":23988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7425","reported_seq":7544,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:59.575611+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:54:59.575611+0000","last_peered":"2024-09-10T15:54:59.575611+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:54:59.575611+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7425,"log_dups_size":0,"ondisk_log_size":7425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30396416,"num_objects":7421,"num_object_clones":0,"num_object_copies":14842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5693,"num_write_kb":22772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3619}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7412","reported_seq":7487,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:59.604049+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:54:59.604049+0000","last_peered":"2024-09-10T15:54:59.604049+0000","last_clean":"2024-09-10T15:54:59.604049+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:54:59.604049+0000","last_undegraded":"2024-09-10T15:54:59.604049+0000","last_fullsized":"2024-09-10T15:54:59.604049+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7412,"log_dups_size":0,"ondisk_log_size":7412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30359552,"num_objects":7412,"num_object_clones":0,"num_object_copies":14824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7412,"num_write_kb":29648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7259","reported_seq":7381,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:57.887361+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:54:57.887361+0000","last_peered":"2024-09-10T15:54:57.887361+0000","last_clean":"2024-09-10T15:54:57.887361+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:54:57.887361+0000","last_undegraded":"2024-09-10T15:54:57.887361+0000","last_fullsized":"2024-09-10T15:54:57.887361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7259,"log_dups_size":0,"ondisk_log_size":7259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29704192,"num_objects":7252,"num_object_clones":0,"num_object_copies":14504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4330,"num_write_kb":17320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7511","reported_seq":7641,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.884900+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:54:57.884900+0000","last_peered":"2024-09-10T15:54:57.884900+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:54:57.884900+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7511,"log_dups_size":0,"ondisk_log_size":7511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30744576,"num_objects":7506,"num_object_clones":0,"num_object_copies":15012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4229,"num_write_kb":16916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3638},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7266","reported_seq":7402,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.932697+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:54:57.932697+0000","last_peered":"2024-09-10T15:54:57.932697+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:54:57.932697+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7266,"log_dups_size":0,"ondisk_log_size":7266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29736960,"num_objects":7260,"num_object_clones":0,"num_object_copies":14520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3916,"num_write_kb":15664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3426},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7431","reported_seq":7557,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:54:57.866297+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:54:57.866297+0000","last_peered":"2024-09-10T15:54:57.866297+0000","last_clean":"2024-09-10T15:54:57.866297+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:54:57.866297+0000","last_undegraded":"2024-09-10T15:54:57.866297+0000","last_fullsized":"2024-09-10T15:54:57.866297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7431,"log_dups_size":0,"ondisk_log_size":7431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30429184,"num_objects":7429,"num_object_clones":0,"num_object_copies":14858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4190,"num_write_kb":16760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7474","reported_seq":13940,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:59.601982+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:54:59.601982+0000","last_peered":"2024-09-10T15:54:59.601982+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:54:59.601982+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7474,"log_dups_size":0,"ondisk_log_size":7474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30584832,"num_objects":7467,"num_object_clones":0,"num_object_copies":14934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":821,"num_objects_unfound":0,"num_objects_dirty":7467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5451,"num_write_kb":21804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3171,"num_bytes_recovered":12980224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":821},{"shards":"2","objects":6644}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7340","reported_seq":7462,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.855577+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:54:57.855577+0000","last_peered":"2024-09-10T15:54:57.855577+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:54:57.855577+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7340,"log_dups_size":0,"ondisk_log_size":7340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4204,"num_write_kb":16816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3469},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7440","reported_seq":7560,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:59.509629+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:54:59.509629+0000","last_peered":"2024-09-10T15:54:59.509629+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:54:59.509629+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7440,"log_dups_size":0,"ondisk_log_size":7440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30453760,"num_objects":7435,"num_object_clones":0,"num_object_copies":14870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5414,"num_write_kb":21656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3603}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7311","reported_seq":7433,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:59.589627+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:54:59.589627+0000","last_peered":"2024-09-10T15:54:59.589627+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:54:59.589627+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7311,"log_dups_size":0,"ondisk_log_size":7311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29921280,"num_objects":7305,"num_object_clones":0,"num_object_copies":14610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5180,"num_write_kb":20720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3476}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7438","reported_seq":7560,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.923318+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:54:57.923318+0000","last_peered":"2024-09-10T15:54:57.923318+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:54:57.923318+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7438,"log_dups_size":0,"ondisk_log_size":7438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30457856,"num_objects":7436,"num_object_clones":0,"num_object_copies":14872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4630,"num_write_kb":18520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3603},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7433","reported_seq":7587,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.932186+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:54:57.932186+0000","last_peered":"2024-09-10T15:54:57.932186+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:54:57.932186+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7433,"log_dups_size":0,"ondisk_log_size":7433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30412800,"num_objects":7425,"num_object_clones":0,"num_object_copies":14850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3851,"num_write_kb":15404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3508},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7384","reported_seq":7519,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.924191+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:54:57.924191+0000","last_peered":"2024-09-10T15:54:57.924191+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:54:57.924191+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7384,"log_dups_size":0,"ondisk_log_size":7384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30228480,"num_objects":7380,"num_object_clones":0,"num_object_copies":14760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3990,"num_write_kb":15960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3503},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7569","reported_seq":7703,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:54:57.914578+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:54:57.914578+0000","last_peered":"2024-09-10T15:54:57.914578+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:54:57.914578+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7569,"log_dups_size":0,"ondisk_log_size":7569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30953955,"num_objects":7558,"num_object_clones":0,"num_object_copies":15116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4388,"num_write_kb":17552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3555},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":486281699,"num_objects":118722,"num_object_clones":0,"num_object_copies":237444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":821,"num_objects_unfound":0,"num_objects_dirty":118722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80324,"num_write_kb":321296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3173,"num_bytes_recovered":12988416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":985202688,"data_stored":985119589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118803,"ondisk_log_size":118803,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":338808,"kb_used_data":180840,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":94033032,"statfs":{"total":96636764160,"available":96289824768,"internally_reserved":0,"allocated":185180160,"data_stored":184852242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161738414},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,2,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149738,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687456,"kb_used_data":417040,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93684384,"statfs":{"total":96636764160,"available":95932809216,"internally_reserved":0,"allocated":427048960,"data_stored":426793302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276885165},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546560,"kb_used_data":362224,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93825280,"statfs":{"total":96636764160,"available":96077086720,"internally_reserved":0,"allocated":370917376,"data_stored":370661718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":427806720,"data_stored":427803107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":186028032,"data_stored":185952159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":371367936,"data_stored":371364323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:03.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 146 20 121469 121449 3.24893 2.05859 0.0222843 0.0240384 2024-09-10T15:55:04.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 147 20 122006 121986 3.24109 2.09766 0.0449055 0.0240966 2024-09-10T15:55:05.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 148 20 122544 122524 3.23339 2.10156 0.0327125 0.0241546 2024-09-10T15:55:06.147 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:06.701 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:06.702 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:06.734 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-10T15:55:06.471641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":496976374,"num_objects":121335,"num_object_clones":0,"num_object_copies":242670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":618,"num_objects_unfound":0,"num_objects_dirty":121335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82937,"num_write_kb":331742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3378,"num_bytes_recovered":13819923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121417,"ondisk_log_size":121417,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1594632,"kb_used_data":981912,"kb_used_omap":12,"kb_used_meta":612659,"kb_avail":281520888,"statfs":{"total":289910292480,"available":288277389312,"internally_reserved":0,"allocated":1005477888,"data_stored":1004638654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":627362821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,1,13,8,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25382912,"num_objects":6197,"num_object_clones":0,"num_object_copies":12394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-590,"num_objects_unfound":0,"num_objects_dirty":6197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6197,"num_write_kb":24788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":590,"num_bytes_recovered":2416640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003825"},"pg_stats":[{"pgid":"2.f","version":"44'7621","reported_seq":7689,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.977548+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:02.977548+0000","last_peered":"2024-09-10T15:55:02.977548+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:02.977548+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7621,"log_dups_size":0,"ondisk_log_size":7621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31215616,"num_objects":7621,"num_object_clones":0,"num_object_copies":15242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7621,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3648},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7816","reported_seq":7917,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:04.528004+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:04.528004+0000","last_peered":"2024-09-10T15:55:04.528004+0000","last_clean":"2024-09-10T15:55:04.528004+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:04.528004+0000","last_undegraded":"2024-09-10T15:55:04.528004+0000","last_fullsized":"2024-09-10T15:55:04.528004+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7816,"log_dups_size":0,"ondisk_log_size":7816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6152,"num_write_kb":24608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7577","reported_seq":7696,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:04.546479+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:04.546479+0000","last_peered":"2024-09-10T15:55:04.546479+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:04.546479+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7577,"log_dups_size":0,"ondisk_log_size":7577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31019008,"num_objects":7573,"num_object_clones":0,"num_object_copies":15146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5845,"num_write_kb":23380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3771}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7589","reported_seq":7664,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:04.533588+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:04.533588+0000","last_peered":"2024-09-10T15:55:04.533588+0000","last_clean":"2024-09-10T15:55:04.533588+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:04.533588+0000","last_undegraded":"2024-09-10T15:55:04.533588+0000","last_fullsized":"2024-09-10T15:55:04.533588+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7589,"log_dups_size":0,"ondisk_log_size":7589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31084544,"num_objects":7589,"num_object_clones":0,"num_object_copies":15178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7589,"num_write_kb":30356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7413","reported_seq":7535,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:02.970469+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:02.970469+0000","last_peered":"2024-09-10T15:55:02.970469+0000","last_clean":"2024-09-10T15:55:02.970469+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:02.970469+0000","last_undegraded":"2024-09-10T15:55:02.970469+0000","last_fullsized":"2024-09-10T15:55:02.970469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7413,"log_dups_size":0,"ondisk_log_size":7413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4484,"num_write_kb":17936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7655","reported_seq":7785,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.793086+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:55:02.793086+0000","last_peered":"2024-09-10T15:55:02.793086+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:02.793086+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7655,"log_dups_size":0,"ondisk_log_size":7655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31334400,"num_objects":7650,"num_object_clones":0,"num_object_copies":15300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4373,"num_write_kb":17492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3782},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7429","reported_seq":7565,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.792616+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:02.792616+0000","last_peered":"2024-09-10T15:55:02.792616+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:02.792616+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7429,"log_dups_size":0,"ondisk_log_size":7429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30404608,"num_objects":7423,"num_object_clones":0,"num_object_copies":14846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4079,"num_write_kb":16316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3589},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7626","reported_seq":7752,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:02.940768+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:02.940768+0000","last_peered":"2024-09-10T15:55:02.940768+0000","last_clean":"2024-09-10T15:55:02.940768+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:02.940768+0000","last_undegraded":"2024-09-10T15:55:02.940768+0000","last_fullsized":"2024-09-10T15:55:02.940768+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7626,"log_dups_size":0,"ondisk_log_size":7626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31227904,"num_objects":7624,"num_object_clones":0,"num_object_copies":15248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4385,"num_write_kb":17540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7623","reported_seq":14496,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:04.602873+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:55:04.602873+0000","last_peered":"2024-09-10T15:55:04.602873+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:55:04.602873+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7623,"log_dups_size":0,"ondisk_log_size":7623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":7616,"num_object_clones":0,"num_object_copies":15232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":618,"num_objects_unfound":0,"num_objects_dirty":7616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5600,"num_write_kb":22400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3374,"num_bytes_recovered":13811712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":618},{"shards":"2","objects":6996}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7510","reported_seq":7632,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.904402+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:02.904402+0000","last_peered":"2024-09-10T15:55:02.904402+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:02.904402+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7510,"log_dups_size":0,"ondisk_log_size":7510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30728192,"num_objects":7502,"num_object_clones":0,"num_object_copies":15004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4374,"num_write_kb":17496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3639},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7623","reported_seq":7742,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:04.575033+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:04.575033+0000","last_peered":"2024-09-10T15:55:04.575033+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:04.575033+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7623,"log_dups_size":0,"ondisk_log_size":7623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31203328,"num_objects":7618,"num_object_clones":0,"num_object_copies":15236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5597,"num_write_kb":22388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3786}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7473","reported_seq":7595,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:04.589422+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:04.589422+0000","last_peered":"2024-09-10T15:55:04.589422+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:04.589422+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7473,"log_dups_size":0,"ondisk_log_size":7473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30584832,"num_objects":7467,"num_object_clones":0,"num_object_copies":14934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5342,"num_write_kb":21368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3638}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7593","reported_seq":7716,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.884854+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:02.884854+0000","last_peered":"2024-09-10T15:55:02.884854+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:02.884854+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7593,"log_dups_size":0,"ondisk_log_size":7593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31092736,"num_objects":7591,"num_object_clones":0,"num_object_copies":15182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4785,"num_write_kb":19140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3758},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7595","reported_seq":7754,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.929168+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:02.929168+0000","last_peered":"2024-09-10T15:55:02.929168+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:02.929168+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7595,"log_dups_size":0,"ondisk_log_size":7595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31076352,"num_objects":7587,"num_object_clones":0,"num_object_copies":15174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4013,"num_write_kb":16052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3670},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7531","reported_seq":7666,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.977229+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:02.977229+0000","last_peered":"2024-09-10T15:55:02.977229+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:02.977229+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7531,"log_dups_size":0,"ondisk_log_size":7531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30830592,"num_objects":7527,"num_object_clones":0,"num_object_copies":15054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4137,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3650},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7740","reported_seq":7874,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.947350+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:02.947350+0000","last_peered":"2024-09-10T15:55:02.947350+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:02.947350+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7740,"log_dups_size":0,"ondisk_log_size":7740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31654371,"num_objects":7729,"num_object_clones":0,"num_object_copies":15458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4559,"num_write_kb":18236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3726},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":496976355,"num_objects":121333,"num_object_clones":0,"num_object_copies":242666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":618,"num_objects_unfound":0,"num_objects_dirty":121333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82935,"num_write_kb":331740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3376,"num_bytes_recovered":13819904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007296512,"data_stored":1007213413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121414,"ondisk_log_size":121414,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":355816,"kb_used_data":189528,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":94016024,"statfs":{"total":96636764160,"available":96272408576,"internally_reserved":0,"allocated":194076672,"data_stored":193748754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170258094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149739,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699480,"kb_used_data":424904,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93672360,"statfs":{"total":96636764160,"available":95920496640,"internally_reserved":0,"allocated":435101696,"data_stored":434846038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281145005},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539336,"kb_used_data":367480,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93832504,"statfs":{"total":96636764160,"available":96084484096,"internally_reserved":0,"allocated":376299520,"data_stored":376043862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,1,4,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":435769344,"data_stored":435765731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":194797568,"data_stored":194721695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376729600,"data_stored":376725987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:06.735 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:06.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 149 20 123047 123027 3.22488 1.96484 0.042532 0.0242181 2024-09-10T15:55:07.300 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:07.300 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:07.330 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-10T15:55:06.471641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":496976374,"num_objects":121335,"num_object_clones":0,"num_object_copies":242670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":618,"num_objects_unfound":0,"num_objects_dirty":121335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82937,"num_write_kb":331742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3378,"num_bytes_recovered":13819923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121417,"ondisk_log_size":121417,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1594632,"kb_used_data":981912,"kb_used_omap":12,"kb_used_meta":612659,"kb_avail":281520888,"statfs":{"total":289910292480,"available":288277389312,"internally_reserved":0,"allocated":1005477888,"data_stored":1004638654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":627362821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,1,13,8,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25382912,"num_objects":6197,"num_object_clones":0,"num_object_copies":12394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-590,"num_objects_unfound":0,"num_objects_dirty":6197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6197,"num_write_kb":24788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":590,"num_bytes_recovered":2416640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003825"},"pg_stats":[{"pgid":"2.f","version":"44'7621","reported_seq":7689,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.977548+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:02.977548+0000","last_peered":"2024-09-10T15:55:02.977548+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:02.977548+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7621,"log_dups_size":0,"ondisk_log_size":7621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31215616,"num_objects":7621,"num_object_clones":0,"num_object_copies":15242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7621,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3648},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7816","reported_seq":7917,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:04.528004+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:04.528004+0000","last_peered":"2024-09-10T15:55:04.528004+0000","last_clean":"2024-09-10T15:55:04.528004+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:04.528004+0000","last_undegraded":"2024-09-10T15:55:04.528004+0000","last_fullsized":"2024-09-10T15:55:04.528004+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7816,"log_dups_size":0,"ondisk_log_size":7816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6152,"num_write_kb":24608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7577","reported_seq":7696,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:04.546479+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:04.546479+0000","last_peered":"2024-09-10T15:55:04.546479+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:04.546479+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7577,"log_dups_size":0,"ondisk_log_size":7577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31019008,"num_objects":7573,"num_object_clones":0,"num_object_copies":15146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5845,"num_write_kb":23380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3771}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7589","reported_seq":7664,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:04.533588+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:04.533588+0000","last_peered":"2024-09-10T15:55:04.533588+0000","last_clean":"2024-09-10T15:55:04.533588+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:04.533588+0000","last_undegraded":"2024-09-10T15:55:04.533588+0000","last_fullsized":"2024-09-10T15:55:04.533588+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7589,"log_dups_size":0,"ondisk_log_size":7589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31084544,"num_objects":7589,"num_object_clones":0,"num_object_copies":15178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7589,"num_write_kb":30356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7413","reported_seq":7535,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:02.970469+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:02.970469+0000","last_peered":"2024-09-10T15:55:02.970469+0000","last_clean":"2024-09-10T15:55:02.970469+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:02.970469+0000","last_undegraded":"2024-09-10T15:55:02.970469+0000","last_fullsized":"2024-09-10T15:55:02.970469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7413,"log_dups_size":0,"ondisk_log_size":7413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4484,"num_write_kb":17936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7655","reported_seq":7785,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.793086+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:55:02.793086+0000","last_peered":"2024-09-10T15:55:02.793086+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:02.793086+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7655,"log_dups_size":0,"ondisk_log_size":7655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31334400,"num_objects":7650,"num_object_clones":0,"num_object_copies":15300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4373,"num_write_kb":17492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3782},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7429","reported_seq":7565,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.792616+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:02.792616+0000","last_peered":"2024-09-10T15:55:02.792616+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:02.792616+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7429,"log_dups_size":0,"ondisk_log_size":7429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30404608,"num_objects":7423,"num_object_clones":0,"num_object_copies":14846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4079,"num_write_kb":16316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3589},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7626","reported_seq":7752,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:02.940768+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:02.940768+0000","last_peered":"2024-09-10T15:55:02.940768+0000","last_clean":"2024-09-10T15:55:02.940768+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:02.940768+0000","last_undegraded":"2024-09-10T15:55:02.940768+0000","last_fullsized":"2024-09-10T15:55:02.940768+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7626,"log_dups_size":0,"ondisk_log_size":7626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31227904,"num_objects":7624,"num_object_clones":0,"num_object_copies":15248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4385,"num_write_kb":17540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7623","reported_seq":14496,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:04.602873+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:55:04.602873+0000","last_peered":"2024-09-10T15:55:04.602873+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:55:04.602873+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7623,"log_dups_size":0,"ondisk_log_size":7623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":7616,"num_object_clones":0,"num_object_copies":15232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":618,"num_objects_unfound":0,"num_objects_dirty":7616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5600,"num_write_kb":22400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3374,"num_bytes_recovered":13811712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":618},{"shards":"2","objects":6996}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7510","reported_seq":7632,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.904402+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:02.904402+0000","last_peered":"2024-09-10T15:55:02.904402+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:02.904402+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7510,"log_dups_size":0,"ondisk_log_size":7510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30728192,"num_objects":7502,"num_object_clones":0,"num_object_copies":15004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4374,"num_write_kb":17496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3639},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7623","reported_seq":7742,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:04.575033+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:04.575033+0000","last_peered":"2024-09-10T15:55:04.575033+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:04.575033+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7623,"log_dups_size":0,"ondisk_log_size":7623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31203328,"num_objects":7618,"num_object_clones":0,"num_object_copies":15236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5597,"num_write_kb":22388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3786}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7473","reported_seq":7595,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:04.589422+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:04.589422+0000","last_peered":"2024-09-10T15:55:04.589422+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:04.589422+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7473,"log_dups_size":0,"ondisk_log_size":7473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30584832,"num_objects":7467,"num_object_clones":0,"num_object_copies":14934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5342,"num_write_kb":21368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3638}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7593","reported_seq":7716,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.884854+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:02.884854+0000","last_peered":"2024-09-10T15:55:02.884854+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:02.884854+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7593,"log_dups_size":0,"ondisk_log_size":7593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31092736,"num_objects":7591,"num_object_clones":0,"num_object_copies":15182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4785,"num_write_kb":19140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3758},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7595","reported_seq":7754,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.929168+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:02.929168+0000","last_peered":"2024-09-10T15:55:02.929168+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:02.929168+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7595,"log_dups_size":0,"ondisk_log_size":7595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31076352,"num_objects":7587,"num_object_clones":0,"num_object_copies":15174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4013,"num_write_kb":16052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3670},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7531","reported_seq":7666,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.977229+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:02.977229+0000","last_peered":"2024-09-10T15:55:02.977229+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:02.977229+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7531,"log_dups_size":0,"ondisk_log_size":7531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30830592,"num_objects":7527,"num_object_clones":0,"num_object_copies":15054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4137,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3650},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7740","reported_seq":7874,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:02.947350+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:02.947350+0000","last_peered":"2024-09-10T15:55:02.947350+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:02.947350+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7740,"log_dups_size":0,"ondisk_log_size":7740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31654371,"num_objects":7729,"num_object_clones":0,"num_object_copies":15458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4559,"num_write_kb":18236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3726},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":496976355,"num_objects":121333,"num_object_clones":0,"num_object_copies":242666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":618,"num_objects_unfound":0,"num_objects_dirty":121333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82935,"num_write_kb":331740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3376,"num_bytes_recovered":13819904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1007296512,"data_stored":1007213413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121414,"ondisk_log_size":121414,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":355816,"kb_used_data":189528,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":94016024,"statfs":{"total":96636764160,"available":96272408576,"internally_reserved":0,"allocated":194076672,"data_stored":193748754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170258094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149739,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699480,"kb_used_data":424904,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93672360,"statfs":{"total":96636764160,"available":95920496640,"internally_reserved":0,"allocated":435101696,"data_stored":434846038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281145005},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539336,"kb_used_data":367480,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93832504,"statfs":{"total":96636764160,"available":96084484096,"internally_reserved":0,"allocated":376299520,"data_stored":376043862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,1,4,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":435769344,"data_stored":435765731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":194797568,"data_stored":194721695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376729600,"data_stored":376725987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:07.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:55:07.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'7621", 'reported_seq': 7689, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:02.977548+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:55:02.977548+0000', 'last_peered': '2024-09-10T15:55:02.977548+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:55:02.977548+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 7621, 'log_dups_size': 0, 'ondisk_log_size': 7621, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31215616, 'num_objects': 7621, 'num_object_clones': 0, 'num_object_copies': 15242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7621, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7621, 'num_write_kb': 30484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3648}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:07.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:55:07.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'7577", 'reported_seq': 7696, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:04.546479+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:55:04.546479+0000', 'last_peered': '2024-09-10T15:55:04.546479+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:55:04.546479+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 7577, 'log_dups_size': 0, 'ondisk_log_size': 7577, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31019008, 'num_objects': 7573, 'num_object_clones': 0, 'num_object_copies': 15146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7573, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5845, 'num_write_kb': 23380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 3771}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:07.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:55:07.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'7655", 'reported_seq': 7785, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:02.793086+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:55:02.793086+0000', 'last_peered': '2024-09-10T15:55:02.793086+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:55:02.793086+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 7655, 'log_dups_size': 0, 'ondisk_log_size': 7655, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31334400, 'num_objects': 7650, 'num_object_clones': 0, 'num_object_copies': 15300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4373, 'num_write_kb': 17492, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3782}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:07.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:55:07.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'7429", 'reported_seq': 7565, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:02.792616+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:55:02.792616+0000', 'last_peered': '2024-09-10T15:55:02.792616+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:55:02.792616+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 7429, 'log_dups_size': 0, 'ondisk_log_size': 7429, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30404608, 'num_objects': 7423, 'num_object_clones': 0, 'num_object_copies': 14846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7423, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4079, 'num_write_kb': 16316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3589}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:07.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:55:07.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'7623", 'reported_seq': 14496, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:04.602873+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:55:04.602873+0000', 'last_peered': '2024-09-10T15:55:04.602873+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:55:04.602873+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 7623, 'log_dups_size': 0, 'ondisk_log_size': 7623, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31195136, 'num_objects': 7616, 'num_object_clones': 0, 'num_object_copies': 15232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 618, 'num_objects_unfound': 0, 'num_objects_dirty': 7616, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5600, 'num_write_kb': 22400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3374, 'num_bytes_recovered': 13811712, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 618}, {'shards': '2', 'objects': 6996}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:07.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:55:07.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'7510", 'reported_seq': 7632, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:02.904402+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:55:02.904402+0000', 'last_peered': '2024-09-10T15:55:02.904402+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:55:02.904402+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 7510, 'log_dups_size': 0, 'ondisk_log_size': 7510, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30728192, 'num_objects': 7502, 'num_object_clones': 0, 'num_object_copies': 15004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4374, 'num_write_kb': 17496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3639}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:07.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:55:07.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'7623", 'reported_seq': 7742, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:04.575033+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:55:04.575033+0000', 'last_peered': '2024-09-10T15:55:04.575033+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:55:04.575033+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 7623, 'log_dups_size': 0, 'ondisk_log_size': 7623, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31203328, 'num_objects': 7618, 'num_object_clones': 0, 'num_object_copies': 15236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7618, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5597, 'num_write_kb': 22388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 3786}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:07.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:55:07.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'7473", 'reported_seq': 7595, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:04.589422+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:55:04.589422+0000', 'last_peered': '2024-09-10T15:55:04.589422+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:55:04.589422+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 7473, 'log_dups_size': 0, 'ondisk_log_size': 7473, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30584832, 'num_objects': 7467, 'num_object_clones': 0, 'num_object_copies': 14934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5342, 'num_write_kb': 21368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 3638}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:07.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:55:07.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'7593", 'reported_seq': 7716, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:02.884854+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:55:02.884854+0000', 'last_peered': '2024-09-10T15:55:02.884854+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:55:02.884854+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 7593, 'log_dups_size': 0, 'ondisk_log_size': 7593, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31092736, 'num_objects': 7591, 'num_object_clones': 0, 'num_object_copies': 15182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4785, 'num_write_kb': 19140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3758}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:07.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:55:07.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'7595", 'reported_seq': 7754, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:02.929168+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:55:02.929168+0000', 'last_peered': '2024-09-10T15:55:02.929168+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:55:02.929168+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 7595, 'log_dups_size': 0, 'ondisk_log_size': 7595, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31076352, 'num_objects': 7587, 'num_object_clones': 0, 'num_object_copies': 15174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7587, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4013, 'num_write_kb': 16052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3670}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:07.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:55:07.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'7531", 'reported_seq': 7666, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:02.977229+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:55:02.977229+0000', 'last_peered': '2024-09-10T15:55:02.977229+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:55:02.977229+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 7531, 'log_dups_size': 0, 'ondisk_log_size': 7531, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30830592, 'num_objects': 7527, 'num_object_clones': 0, 'num_object_copies': 15054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7527, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4137, 'num_write_kb': 16548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3650}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:07.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:55:07.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'7740", 'reported_seq': 7874, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:02.947350+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:55:02.947350+0000', 'last_peered': '2024-09-10T15:55:02.947350+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:55:02.947350+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 7740, 'log_dups_size': 0, 'ondisk_log_size': 7740, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31654371, 'num_objects': 7729, 'num_object_clones': 0, 'num_object_copies': 15458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7729, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4559, 'num_write_kb': 18236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3726}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:07.334 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:55:07.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 150 20 123542 123522 3.21627 1.93359 0.0161437 0.0242822 2024-09-10T15:55:08.022 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:08.022 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/242670 objects degraded (17.577%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":233,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":121335,"data_bytes":496976374,"bytes_used":1632903168,"bytes_avail":288277389312,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":242670,"degraded_ratio":0.17576544278237935,"misplaced_objects":618,"misplaced_total":242670,"misplaced_ratio":0.002546668314995673,"recovering_objects_per_sec":49,"recovering_bytes_per_sec":201322,"recovering_keys_per_sec":0,"num_objects_recovered":590,"num_bytes_recovered":2416640,"num_keys_recovered":0,"write_bytes_sec":2114568,"read_op_per_sec":0,"write_op_per_sec":516},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:55:08.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:55:08.045 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:08.614 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:08.614 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:08.641 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-09-10T15:55:08.472143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":503628278,"num_objects":122959,"num_object_clones":0,"num_object_copies":245918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":618,"num_objects_unfound":0,"num_objects_dirty":122959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84561,"num_write_kb":338238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3378,"num_bytes_recovered":13819923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123041,"ondisk_log_size":123041,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1590096,"kb_used_data":998176,"kb_used_omap":12,"kb_used_meta":591859,"kb_avail":281525424,"statfs":{"total":289910292480,"available":288282034176,"internally_reserved":0,"allocated":1022132224,"data_stored":1021292990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":606063621},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,4,18,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28098560,"num_objects":6860,"num_object_clones":0,"num_object_copies":13720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-389,"num_objects_unfound":0,"num_objects_dirty":6860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6860,"num_write_kb":27440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":389,"num_bytes_recovered":1593344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003813"},"pg_stats":[{"pgid":"2.f","version":"44'7796","reported_seq":7864,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.989121+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:07.989121+0000","last_peered":"2024-09-10T15:55:07.989121+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:07.989121+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7796,"log_dups_size":0,"ondisk_log_size":7796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31932416,"num_objects":7796,"num_object_clones":0,"num_object_copies":15592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7796,"num_write_kb":31184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3823},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7816","reported_seq":7917,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:04.528004+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:04.528004+0000","last_peered":"2024-09-10T15:55:04.528004+0000","last_clean":"2024-09-10T15:55:04.528004+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:04.528004+0000","last_undegraded":"2024-09-10T15:55:04.528004+0000","last_fullsized":"2024-09-10T15:55:04.528004+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7816,"log_dups_size":0,"ondisk_log_size":7816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6152,"num_write_kb":24608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7577","reported_seq":7696,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:04.546479+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:04.546479+0000","last_peered":"2024-09-10T15:55:04.546479+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:04.546479+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7577,"log_dups_size":0,"ondisk_log_size":7577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31019008,"num_objects":7573,"num_object_clones":0,"num_object_copies":15146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5845,"num_write_kb":23380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3771}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7589","reported_seq":7664,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:04.533588+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:04.533588+0000","last_peered":"2024-09-10T15:55:04.533588+0000","last_clean":"2024-09-10T15:55:04.533588+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:04.533588+0000","last_undegraded":"2024-09-10T15:55:04.533588+0000","last_fullsized":"2024-09-10T15:55:04.533588+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7589,"log_dups_size":0,"ondisk_log_size":7589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31084544,"num_objects":7589,"num_object_clones":0,"num_object_copies":15178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7589,"num_write_kb":30356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7558","reported_seq":7680,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:07.987310+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:07.987310+0000","last_peered":"2024-09-10T15:55:07.987310+0000","last_clean":"2024-09-10T15:55:07.987310+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:07.987310+0000","last_undegraded":"2024-09-10T15:55:07.987310+0000","last_fullsized":"2024-09-10T15:55:07.987310+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7558,"log_dups_size":0,"ondisk_log_size":7558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30928896,"num_objects":7551,"num_object_clones":0,"num_object_copies":15102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4629,"num_write_kb":18516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7815","reported_seq":7945,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.938766+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:55:07.938766+0000","last_peered":"2024-09-10T15:55:07.938766+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:07.938766+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7815,"log_dups_size":0,"ondisk_log_size":7815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3942},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7576","reported_seq":7712,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.981001+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:07.981001+0000","last_peered":"2024-09-10T15:55:07.981001+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:07.981001+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7576,"log_dups_size":0,"ondisk_log_size":7576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4226,"num_write_kb":16904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3736},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7790","reported_seq":7915,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:07.952400+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:07.952400+0000","last_peered":"2024-09-10T15:55:07.952400+0000","last_clean":"2024-09-10T15:55:07.952400+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:07.952400+0000","last_undegraded":"2024-09-10T15:55:07.952400+0000","last_fullsized":"2024-09-10T15:55:07.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7790,"log_dups_size":0,"ondisk_log_size":7790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31899648,"num_objects":7788,"num_object_clones":0,"num_object_copies":15576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4549,"num_write_kb":18196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7623","reported_seq":14496,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:04.602873+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:55:04.602873+0000","last_peered":"2024-09-10T15:55:04.602873+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:55:04.602873+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7623,"log_dups_size":0,"ondisk_log_size":7623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":7616,"num_object_clones":0,"num_object_copies":15232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":618,"num_objects_unfound":0,"num_objects_dirty":7616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5600,"num_write_kb":22400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3374,"num_bytes_recovered":13811712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":618},{"shards":"2","objects":6996}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7668","reported_seq":7790,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.960236+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:07.960236+0000","last_peered":"2024-09-10T15:55:07.960236+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:07.960236+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7668,"log_dups_size":0,"ondisk_log_size":7668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31375360,"num_objects":7660,"num_object_clones":0,"num_object_copies":15320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4532,"num_write_kb":18128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7623","reported_seq":7742,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:04.575033+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:04.575033+0000","last_peered":"2024-09-10T15:55:04.575033+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:04.575033+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7623,"log_dups_size":0,"ondisk_log_size":7623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31203328,"num_objects":7618,"num_object_clones":0,"num_object_copies":15236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5597,"num_write_kb":22388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3786}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7473","reported_seq":7595,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:04.589422+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:04.589422+0000","last_peered":"2024-09-10T15:55:04.589422+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:04.589422+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7473,"log_dups_size":0,"ondisk_log_size":7473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30584832,"num_objects":7467,"num_object_clones":0,"num_object_copies":14934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5342,"num_write_kb":21368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3638}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7765","reported_seq":7888,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.853943+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:07.853943+0000","last_peered":"2024-09-10T15:55:07.853943+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:07.853943+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7765,"log_dups_size":0,"ondisk_log_size":7765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31797248,"num_objects":7763,"num_object_clones":0,"num_object_copies":15526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4957,"num_write_kb":19828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3930},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7761","reported_seq":7920,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.964550+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:07.964550+0000","last_peered":"2024-09-10T15:55:07.964550+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:07.964550+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7761,"log_dups_size":0,"ondisk_log_size":7761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4179,"num_write_kb":16716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3836},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7694","reported_seq":7829,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.900204+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:07.900204+0000","last_peered":"2024-09-10T15:55:07.900204+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:07.900204+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7694,"log_dups_size":0,"ondisk_log_size":7694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31498240,"num_objects":7690,"num_object_clones":0,"num_object_copies":15380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4300,"num_write_kb":17200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3813},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7914","reported_seq":8048,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.980628+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:07.980628+0000","last_peered":"2024-09-10T15:55:07.980628+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:07.980628+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7914,"log_dups_size":0,"ondisk_log_size":7914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32367075,"num_objects":7903,"num_object_clones":0,"num_object_copies":15806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3900},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":503628259,"num_objects":122957,"num_object_clones":0,"num_object_copies":245914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":618,"num_objects_unfound":0,"num_objects_dirty":122957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84559,"num_write_kb":338236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3376,"num_bytes_recovered":13819904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1024299008,"data_stored":1024215909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123038,"ondisk_log_size":123038,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":351812,"kb_used_data":198004,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":94020028,"statfs":{"total":96636764160,"available":96276508672,"internally_reserved":0,"allocated":202756096,"data_stored":202428178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157478574},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149740,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698948,"kb_used_data":432692,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93672892,"statfs":{"total":96636764160,"available":95921041408,"internally_reserved":0,"allocated":443076608,"data_stored":442820950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272625325},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,9,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539336,"kb_used_data":367480,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93832504,"statfs":{"total":96636764160,"available":96084484096,"internally_reserved":0,"allocated":376299520,"data_stored":376043862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,1,4,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443883520,"data_stored":443879907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":203685888,"data_stored":203610015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376729600,"data_stored":376725987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:08.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 151 20 124034 124014 3.2077 1.92188 0.0446454 0.0243487 2024-09-10T15:55:09.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 152 20 124579 124559 3.2006 2.12891 0.0609623 0.0244021 2024-09-10T15:55:10.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 153 20 125123 125103 3.19357 2.125 0.0356513 0.024456 2024-09-10T15:55:11.644 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:11.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 154 20 125654 125634 3.1863 2.07422 0.010681 0.0245107 2024-09-10T15:55:12.195 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:12.198 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:12.221 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-10T15:55:10.472650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":507720182,"num_objects":123958,"num_object_clones":0,"num_object_copies":247916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":408,"num_objects_unfound":0,"num_objects_dirty":123958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85560,"num_write_kb":342234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3588,"num_bytes_recovered":14680083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124040,"ondisk_log_size":124040,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1599284,"kb_used_data":1003204,"kb_used_omap":12,"kb_used_meta":596019,"kb_avail":281516236,"statfs":{"total":289910292480,"available":288272625664,"internally_reserved":0,"allocated":1027280896,"data_stored":1026441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":610323461},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,16,11,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25567232,"num_objects":6242,"num_object_clones":0,"num_object_copies":12484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-599,"num_objects_unfound":0,"num_objects_dirty":6242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6242,"num_write_kb":24968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":599,"num_bytes_recovered":2453504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003770"},"pg_stats":[{"pgid":"2.f","version":"44'7796","reported_seq":7864,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.989121+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:07.989121+0000","last_peered":"2024-09-10T15:55:07.989121+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:07.989121+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7796,"log_dups_size":0,"ondisk_log_size":7796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31932416,"num_objects":7796,"num_object_clones":0,"num_object_copies":15592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7796,"num_write_kb":31184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3823},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7981","reported_seq":8084,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:09.559587+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:09.559587+0000","last_peered":"2024-09-10T15:55:09.559587+0000","last_clean":"2024-09-10T15:55:09.559587+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:09.559587+0000","last_undegraded":"2024-09-10T15:55:09.559587+0000","last_fullsized":"2024-09-10T15:55:09.559587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7981,"log_dups_size":0,"ondisk_log_size":7981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32665600,"num_objects":7975,"num_object_clones":0,"num_object_copies":15950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6317,"num_write_kb":25268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7731","reported_seq":7850,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:09.519836+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:09.519836+0000","last_peered":"2024-09-10T15:55:09.519836+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:09.519836+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7731,"log_dups_size":0,"ondisk_log_size":7731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31649792,"num_objects":7727,"num_object_clones":0,"num_object_copies":15454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5999,"num_write_kb":23996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3925}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7753","reported_seq":7828,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:09.586251+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:09.586251+0000","last_peered":"2024-09-10T15:55:09.586251+0000","last_clean":"2024-09-10T15:55:09.586251+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:09.586251+0000","last_undegraded":"2024-09-10T15:55:09.586251+0000","last_fullsized":"2024-09-10T15:55:09.586251+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7753,"log_dups_size":0,"ondisk_log_size":7753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7753,"num_write_kb":31012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7558","reported_seq":7680,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:07.987310+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:07.987310+0000","last_peered":"2024-09-10T15:55:07.987310+0000","last_clean":"2024-09-10T15:55:07.987310+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:07.987310+0000","last_undegraded":"2024-09-10T15:55:07.987310+0000","last_fullsized":"2024-09-10T15:55:07.987310+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7558,"log_dups_size":0,"ondisk_log_size":7558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30928896,"num_objects":7551,"num_object_clones":0,"num_object_copies":15102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4629,"num_write_kb":18516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7815","reported_seq":7945,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.938766+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:55:07.938766+0000","last_peered":"2024-09-10T15:55:07.938766+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:07.938766+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7815,"log_dups_size":0,"ondisk_log_size":7815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3942},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7576","reported_seq":7712,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.981001+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:07.981001+0000","last_peered":"2024-09-10T15:55:07.981001+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:07.981001+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7576,"log_dups_size":0,"ondisk_log_size":7576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4226,"num_write_kb":16904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3736},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7790","reported_seq":7915,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:07.952400+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:07.952400+0000","last_peered":"2024-09-10T15:55:07.952400+0000","last_clean":"2024-09-10T15:55:07.952400+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:07.952400+0000","last_undegraded":"2024-09-10T15:55:07.952400+0000","last_fullsized":"2024-09-10T15:55:07.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7790,"log_dups_size":0,"ondisk_log_size":7790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31899648,"num_objects":7788,"num_object_clones":0,"num_object_copies":15576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4549,"num_write_kb":18196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7815","reported_seq":15108,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:09.632509+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:55:09.632509+0000","last_peered":"2024-09-10T15:55:09.632509+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:55:09.632509+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7815,"log_dups_size":0,"ondisk_log_size":7815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31981568,"num_objects":7808,"num_object_clones":0,"num_object_copies":15616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":408,"num_objects_unfound":0,"num_objects_dirty":7808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5792,"num_write_kb":23168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3584,"num_bytes_recovered":14671872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":408},{"shards":"2","objects":7398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7668","reported_seq":7790,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.960236+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:07.960236+0000","last_peered":"2024-09-10T15:55:07.960236+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:07.960236+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7668,"log_dups_size":0,"ondisk_log_size":7668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31375360,"num_objects":7660,"num_object_clones":0,"num_object_copies":15320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4532,"num_write_kb":18128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7776","reported_seq":7894,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:09.593618+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:09.593618+0000","last_peered":"2024-09-10T15:55:09.593618+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:09.593618+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7776,"log_dups_size":0,"ondisk_log_size":7776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31830016,"num_objects":7771,"num_object_clones":0,"num_object_copies":15542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5750,"num_write_kb":23000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7644","reported_seq":7764,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:09.540680+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:09.540680+0000","last_peered":"2024-09-10T15:55:09.540680+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:09.540680+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7644,"log_dups_size":0,"ondisk_log_size":7644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31285248,"num_objects":7638,"num_object_clones":0,"num_object_copies":15276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5513,"num_write_kb":22052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3809}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7765","reported_seq":7888,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.853943+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:07.853943+0000","last_peered":"2024-09-10T15:55:07.853943+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:07.853943+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7765,"log_dups_size":0,"ondisk_log_size":7765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31797248,"num_objects":7763,"num_object_clones":0,"num_object_copies":15526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4957,"num_write_kb":19828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3930},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7761","reported_seq":7920,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.964550+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:07.964550+0000","last_peered":"2024-09-10T15:55:07.964550+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:07.964550+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7761,"log_dups_size":0,"ondisk_log_size":7761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4179,"num_write_kb":16716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3836},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7694","reported_seq":7829,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.900204+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:07.900204+0000","last_peered":"2024-09-10T15:55:07.900204+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:07.900204+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7694,"log_dups_size":0,"ondisk_log_size":7694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31498240,"num_objects":7690,"num_object_clones":0,"num_object_copies":15380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4300,"num_write_kb":17200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3813},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7914","reported_seq":8048,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.980628+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:07.980628+0000","last_peered":"2024-09-10T15:55:07.980628+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:07.980628+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7914,"log_dups_size":0,"ondisk_log_size":7914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32367075,"num_objects":7903,"num_object_clones":0,"num_object_copies":15806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3900},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":507720163,"num_objects":123956,"num_object_clones":0,"num_object_copies":247912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":408,"num_objects_unfound":0,"num_objects_dirty":123956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85558,"num_write_kb":342232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3586,"num_bytes_recovered":14680064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1029636096,"data_stored":1029552997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124037,"ondisk_log_size":124037,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":351812,"kb_used_data":198004,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":94020028,"statfs":{"total":96636764160,"available":96276508672,"internally_reserved":0,"allocated":202756096,"data_stored":202428178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157478574},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149740,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698948,"kb_used_data":432692,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93672892,"statfs":{"total":96636764160,"available":95921041408,"internally_reserved":0,"allocated":443076608,"data_stored":442820950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272625325},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,9,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548524,"kb_used_data":372508,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93823316,"statfs":{"total":96636764160,"available":96075075584,"internally_reserved":0,"allocated":381448192,"data_stored":381192534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443883520,"data_stored":443879907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":203685888,"data_stored":203610015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":382066688,"data_stored":382063075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:12.223 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:12.796 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:12.798 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:12.817 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 155 20 126130 126110 3.17774 1.85938 0.00785076 0.0245744 2024-09-10T15:55:12.822 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-10T15:55:12.473153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":507720182,"num_objects":123958,"num_object_clones":0,"num_object_copies":247916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":408,"num_objects_unfound":0,"num_objects_dirty":123958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85560,"num_write_kb":342234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3588,"num_bytes_recovered":14680083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124040,"ondisk_log_size":124040,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1599284,"kb_used_data":1003204,"kb_used_omap":12,"kb_used_meta":596019,"kb_avail":281516236,"statfs":{"total":289910292480,"available":288272625664,"internally_reserved":0,"allocated":1027280896,"data_stored":1026441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":610323461},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,16,11,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-413,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5234,"num_write_kb":20936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":1691648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003778"},"pg_stats":[{"pgid":"2.f","version":"44'7796","reported_seq":7864,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.989121+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:07.989121+0000","last_peered":"2024-09-10T15:55:07.989121+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:07.989121+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7796,"log_dups_size":0,"ondisk_log_size":7796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31932416,"num_objects":7796,"num_object_clones":0,"num_object_copies":15592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7796,"num_write_kb":31184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3823},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7981","reported_seq":8084,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:09.559587+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:09.559587+0000","last_peered":"2024-09-10T15:55:09.559587+0000","last_clean":"2024-09-10T15:55:09.559587+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:09.559587+0000","last_undegraded":"2024-09-10T15:55:09.559587+0000","last_fullsized":"2024-09-10T15:55:09.559587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7981,"log_dups_size":0,"ondisk_log_size":7981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32665600,"num_objects":7975,"num_object_clones":0,"num_object_copies":15950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6317,"num_write_kb":25268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7731","reported_seq":7850,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:09.519836+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:09.519836+0000","last_peered":"2024-09-10T15:55:09.519836+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:09.519836+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7731,"log_dups_size":0,"ondisk_log_size":7731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31649792,"num_objects":7727,"num_object_clones":0,"num_object_copies":15454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5999,"num_write_kb":23996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3925}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7753","reported_seq":7828,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:09.586251+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:09.586251+0000","last_peered":"2024-09-10T15:55:09.586251+0000","last_clean":"2024-09-10T15:55:09.586251+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:09.586251+0000","last_undegraded":"2024-09-10T15:55:09.586251+0000","last_fullsized":"2024-09-10T15:55:09.586251+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7753,"log_dups_size":0,"ondisk_log_size":7753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7753,"num_write_kb":31012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7558","reported_seq":7680,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:07.987310+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:07.987310+0000","last_peered":"2024-09-10T15:55:07.987310+0000","last_clean":"2024-09-10T15:55:07.987310+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:07.987310+0000","last_undegraded":"2024-09-10T15:55:07.987310+0000","last_fullsized":"2024-09-10T15:55:07.987310+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7558,"log_dups_size":0,"ondisk_log_size":7558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30928896,"num_objects":7551,"num_object_clones":0,"num_object_copies":15102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4629,"num_write_kb":18516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7815","reported_seq":7945,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.938766+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:55:07.938766+0000","last_peered":"2024-09-10T15:55:07.938766+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:07.938766+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7815,"log_dups_size":0,"ondisk_log_size":7815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3942},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7576","reported_seq":7712,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.981001+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:07.981001+0000","last_peered":"2024-09-10T15:55:07.981001+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:07.981001+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7576,"log_dups_size":0,"ondisk_log_size":7576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4226,"num_write_kb":16904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3736},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7790","reported_seq":7915,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:07.952400+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:07.952400+0000","last_peered":"2024-09-10T15:55:07.952400+0000","last_clean":"2024-09-10T15:55:07.952400+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:07.952400+0000","last_undegraded":"2024-09-10T15:55:07.952400+0000","last_fullsized":"2024-09-10T15:55:07.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7790,"log_dups_size":0,"ondisk_log_size":7790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31899648,"num_objects":7788,"num_object_clones":0,"num_object_copies":15576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4549,"num_write_kb":18196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7815","reported_seq":15108,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:09.632509+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:55:09.632509+0000","last_peered":"2024-09-10T15:55:09.632509+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:55:09.632509+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7815,"log_dups_size":0,"ondisk_log_size":7815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31981568,"num_objects":7808,"num_object_clones":0,"num_object_copies":15616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":408,"num_objects_unfound":0,"num_objects_dirty":7808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5792,"num_write_kb":23168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3584,"num_bytes_recovered":14671872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":408},{"shards":"2","objects":7398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7668","reported_seq":7790,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.960236+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:07.960236+0000","last_peered":"2024-09-10T15:55:07.960236+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:07.960236+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7668,"log_dups_size":0,"ondisk_log_size":7668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31375360,"num_objects":7660,"num_object_clones":0,"num_object_copies":15320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4532,"num_write_kb":18128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7776","reported_seq":7894,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:09.593618+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:09.593618+0000","last_peered":"2024-09-10T15:55:09.593618+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:09.593618+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7776,"log_dups_size":0,"ondisk_log_size":7776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31830016,"num_objects":7771,"num_object_clones":0,"num_object_copies":15542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5750,"num_write_kb":23000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7644","reported_seq":7764,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:09.540680+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:09.540680+0000","last_peered":"2024-09-10T15:55:09.540680+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:09.540680+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7644,"log_dups_size":0,"ondisk_log_size":7644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31285248,"num_objects":7638,"num_object_clones":0,"num_object_copies":15276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5513,"num_write_kb":22052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3809}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7765","reported_seq":7888,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.853943+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:07.853943+0000","last_peered":"2024-09-10T15:55:07.853943+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:07.853943+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7765,"log_dups_size":0,"ondisk_log_size":7765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31797248,"num_objects":7763,"num_object_clones":0,"num_object_copies":15526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4957,"num_write_kb":19828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3930},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7761","reported_seq":7920,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.964550+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:07.964550+0000","last_peered":"2024-09-10T15:55:07.964550+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:07.964550+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7761,"log_dups_size":0,"ondisk_log_size":7761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4179,"num_write_kb":16716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3836},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7694","reported_seq":7829,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.900204+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:07.900204+0000","last_peered":"2024-09-10T15:55:07.900204+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:07.900204+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7694,"log_dups_size":0,"ondisk_log_size":7694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31498240,"num_objects":7690,"num_object_clones":0,"num_object_copies":15380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4300,"num_write_kb":17200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3813},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7914","reported_seq":8048,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.980628+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:07.980628+0000","last_peered":"2024-09-10T15:55:07.980628+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:07.980628+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7914,"log_dups_size":0,"ondisk_log_size":7914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32367075,"num_objects":7903,"num_object_clones":0,"num_object_copies":15806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3900},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":507720163,"num_objects":123956,"num_object_clones":0,"num_object_copies":247912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":408,"num_objects_unfound":0,"num_objects_dirty":123956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85558,"num_write_kb":342232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3586,"num_bytes_recovered":14680064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1029636096,"data_stored":1029552997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124037,"ondisk_log_size":124037,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":351812,"kb_used_data":198004,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":94020028,"statfs":{"total":96636764160,"available":96276508672,"internally_reserved":0,"allocated":202756096,"data_stored":202428178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157478574},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149740,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698948,"kb_used_data":432692,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93672892,"statfs":{"total":96636764160,"available":95921041408,"internally_reserved":0,"allocated":443076608,"data_stored":442820950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272625325},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,9,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548524,"kb_used_data":372508,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93823316,"statfs":{"total":96636764160,"available":96075075584,"internally_reserved":0,"allocated":381448192,"data_stored":381192534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443883520,"data_stored":443879907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":203685888,"data_stored":203610015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":382066688,"data_stored":382063075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:12.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:55:12.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'7796", 'reported_seq': 7864, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:07.989121+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:55:07.989121+0000', 'last_peered': '2024-09-10T15:55:07.989121+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:55:07.989121+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 7796, 'log_dups_size': 0, 'ondisk_log_size': 7796, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31932416, 'num_objects': 7796, 'num_object_clones': 0, 'num_object_copies': 15592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7796, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7796, 'num_write_kb': 31184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3823}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:12.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:55:12.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'7731", 'reported_seq': 7850, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:09.519836+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:55:09.519836+0000', 'last_peered': '2024-09-10T15:55:09.519836+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:55:09.519836+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 7731, 'log_dups_size': 0, 'ondisk_log_size': 7731, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31649792, 'num_objects': 7727, 'num_object_clones': 0, 'num_object_copies': 15454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7727, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5999, 'num_write_kb': 23996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 3925}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:12.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:55:12.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'7815", 'reported_seq': 7945, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:07.938766+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:55:07.938766+0000', 'last_peered': '2024-09-10T15:55:07.938766+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:55:07.938766+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 7815, 'log_dups_size': 0, 'ondisk_log_size': 7815, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31989760, 'num_objects': 7810, 'num_object_clones': 0, 'num_object_copies': 15620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7810, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4533, 'num_write_kb': 18132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3942}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:12.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:55:12.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'7576", 'reported_seq': 7712, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:07.981001+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:55:07.981001+0000', 'last_peered': '2024-09-10T15:55:07.981001+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:55:07.981001+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 7576, 'log_dups_size': 0, 'ondisk_log_size': 7576, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31006720, 'num_objects': 7570, 'num_object_clones': 0, 'num_object_copies': 15140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4226, 'num_write_kb': 16904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3736}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:12.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:55:12.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'7815", 'reported_seq': 15108, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:09.632509+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:55:09.632509+0000', 'last_peered': '2024-09-10T15:55:09.632509+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:55:09.632509+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 7815, 'log_dups_size': 0, 'ondisk_log_size': 7815, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31981568, 'num_objects': 7808, 'num_object_clones': 0, 'num_object_copies': 15616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 408, 'num_objects_unfound': 0, 'num_objects_dirty': 7808, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5792, 'num_write_kb': 23168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3584, 'num_bytes_recovered': 14671872, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 408}, {'shards': '2', 'objects': 7398}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:12.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:55:12.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'7668", 'reported_seq': 7790, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:07.960236+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:55:07.960236+0000', 'last_peered': '2024-09-10T15:55:07.960236+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:55:07.960236+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 7668, 'log_dups_size': 0, 'ondisk_log_size': 7668, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31375360, 'num_objects': 7660, 'num_object_clones': 0, 'num_object_copies': 15320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7660, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4532, 'num_write_kb': 18128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3797}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:12.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:55:12.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'7776", 'reported_seq': 7894, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:09.593618+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:55:09.593618+0000', 'last_peered': '2024-09-10T15:55:09.593618+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:55:09.593618+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 7776, 'log_dups_size': 0, 'ondisk_log_size': 7776, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31830016, 'num_objects': 7771, 'num_object_clones': 0, 'num_object_copies': 15542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7771, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5750, 'num_write_kb': 23000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:12.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:55:12.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'7644", 'reported_seq': 7764, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:09.540680+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:55:09.540680+0000', 'last_peered': '2024-09-10T15:55:09.540680+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:55:09.540680+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 7644, 'log_dups_size': 0, 'ondisk_log_size': 7644, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31285248, 'num_objects': 7638, 'num_object_clones': 0, 'num_object_copies': 15276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7638, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5513, 'num_write_kb': 22052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 3809}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:12.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:55:12.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'7765", 'reported_seq': 7888, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:07.853943+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:55:07.853943+0000', 'last_peered': '2024-09-10T15:55:07.853943+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:55:07.853943+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 7765, 'log_dups_size': 0, 'ondisk_log_size': 7765, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31797248, 'num_objects': 7763, 'num_object_clones': 0, 'num_object_copies': 15526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7763, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4957, 'num_write_kb': 19828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3930}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:12.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:55:12.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'7761", 'reported_seq': 7920, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:07.964550+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:55:07.964550+0000', 'last_peered': '2024-09-10T15:55:07.964550+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:55:07.964550+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 7761, 'log_dups_size': 0, 'ondisk_log_size': 7761, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31756288, 'num_objects': 7753, 'num_object_clones': 0, 'num_object_copies': 15506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4179, 'num_write_kb': 16716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3836}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:12.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:55:12.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'7694", 'reported_seq': 7829, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:07.900204+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:55:07.900204+0000', 'last_peered': '2024-09-10T15:55:07.900204+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:55:07.900204+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 7694, 'log_dups_size': 0, 'ondisk_log_size': 7694, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31498240, 'num_objects': 7690, 'num_object_clones': 0, 'num_object_copies': 15380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4300, 'num_write_kb': 17200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3813}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:12.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:55:12.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'7914", 'reported_seq': 8048, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:07.980628+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:55:07.980628+0000', 'last_peered': '2024-09-10T15:55:07.980628+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:55:07.980628+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 7914, 'log_dups_size': 0, 'ondisk_log_size': 7914, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32367075, 'num_objects': 7903, 'num_object_clones': 0, 'num_object_copies': 15806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4733, 'num_write_kb': 18932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3900}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:12.826 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:55:13.512 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:13.512 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/247916 objects degraded (17.205%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":239,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":123958,"data_bytes":507720182,"bytes_used":1637666816,"bytes_avail":288272625664,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":247916,"degraded_ratio":0.17204617693089597,"misplaced_objects":408,"misplaced_total":247916,"misplaced_ratio":0.0016457187111763663,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":140926,"recovering_keys_per_sec":0,"num_objects_recovered":413,"num_bytes_recovered":1691648,"num_keys_recovered":0,"write_bytes_sec":1785976,"read_op_per_sec":0,"write_op_per_sec":436},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:55:13.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:55:13.535 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:13.818 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 156 20 126599 126579 3.16911 1.83203 0.0414982 0.0246424 2024-09-10T15:55:14.086 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:14.087 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:14.109 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-10T15:55:12.473153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":507720182,"num_objects":123958,"num_object_clones":0,"num_object_copies":247916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":408,"num_objects_unfound":0,"num_objects_dirty":123958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85560,"num_write_kb":342234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3588,"num_bytes_recovered":14680083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124040,"ondisk_log_size":124040,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1599284,"kb_used_data":1003204,"kb_used_omap":12,"kb_used_meta":596019,"kb_avail":281516236,"statfs":{"total":289910292480,"available":288272625664,"internally_reserved":0,"allocated":1027280896,"data_stored":1026441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":610323461},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,16,11,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-413,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5234,"num_write_kb":20936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":1691648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003778"},"pg_stats":[{"pgid":"2.f","version":"44'7796","reported_seq":7864,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.989121+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:07.989121+0000","last_peered":"2024-09-10T15:55:07.989121+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:07.989121+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7796,"log_dups_size":0,"ondisk_log_size":7796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31932416,"num_objects":7796,"num_object_clones":0,"num_object_copies":15592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7796,"num_write_kb":31184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3823},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'7981","reported_seq":8084,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:09.559587+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:09.559587+0000","last_peered":"2024-09-10T15:55:09.559587+0000","last_clean":"2024-09-10T15:55:09.559587+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:09.559587+0000","last_undegraded":"2024-09-10T15:55:09.559587+0000","last_fullsized":"2024-09-10T15:55:09.559587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":7981,"log_dups_size":0,"ondisk_log_size":7981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32665600,"num_objects":7975,"num_object_clones":0,"num_object_copies":15950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6317,"num_write_kb":25268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7731","reported_seq":7850,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:09.519836+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:09.519836+0000","last_peered":"2024-09-10T15:55:09.519836+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:09.519836+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7731,"log_dups_size":0,"ondisk_log_size":7731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31649792,"num_objects":7727,"num_object_clones":0,"num_object_copies":15454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5999,"num_write_kb":23996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":3925}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7753","reported_seq":7828,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:09.586251+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:09.586251+0000","last_peered":"2024-09-10T15:55:09.586251+0000","last_clean":"2024-09-10T15:55:09.586251+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:09.586251+0000","last_undegraded":"2024-09-10T15:55:09.586251+0000","last_fullsized":"2024-09-10T15:55:09.586251+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7753,"log_dups_size":0,"ondisk_log_size":7753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7753,"num_write_kb":31012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7558","reported_seq":7680,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:07.987310+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:07.987310+0000","last_peered":"2024-09-10T15:55:07.987310+0000","last_clean":"2024-09-10T15:55:07.987310+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:07.987310+0000","last_undegraded":"2024-09-10T15:55:07.987310+0000","last_fullsized":"2024-09-10T15:55:07.987310+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7558,"log_dups_size":0,"ondisk_log_size":7558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30928896,"num_objects":7551,"num_object_clones":0,"num_object_copies":15102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4629,"num_write_kb":18516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7815","reported_seq":7945,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.938766+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:55:07.938766+0000","last_peered":"2024-09-10T15:55:07.938766+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:07.938766+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7815,"log_dups_size":0,"ondisk_log_size":7815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3942},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7576","reported_seq":7712,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.981001+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:07.981001+0000","last_peered":"2024-09-10T15:55:07.981001+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:07.981001+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7576,"log_dups_size":0,"ondisk_log_size":7576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4226,"num_write_kb":16904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3736},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7790","reported_seq":7915,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:07.952400+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:07.952400+0000","last_peered":"2024-09-10T15:55:07.952400+0000","last_clean":"2024-09-10T15:55:07.952400+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:07.952400+0000","last_undegraded":"2024-09-10T15:55:07.952400+0000","last_fullsized":"2024-09-10T15:55:07.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7790,"log_dups_size":0,"ondisk_log_size":7790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31899648,"num_objects":7788,"num_object_clones":0,"num_object_copies":15576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4549,"num_write_kb":18196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7815","reported_seq":15108,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:09.632509+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:55:09.632509+0000","last_peered":"2024-09-10T15:55:09.632509+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:55:09.632509+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7815,"log_dups_size":0,"ondisk_log_size":7815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31981568,"num_objects":7808,"num_object_clones":0,"num_object_copies":15616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":408,"num_objects_unfound":0,"num_objects_dirty":7808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5792,"num_write_kb":23168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3584,"num_bytes_recovered":14671872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":408},{"shards":"2","objects":7398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7668","reported_seq":7790,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.960236+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:07.960236+0000","last_peered":"2024-09-10T15:55:07.960236+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:07.960236+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7668,"log_dups_size":0,"ondisk_log_size":7668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31375360,"num_objects":7660,"num_object_clones":0,"num_object_copies":15320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4532,"num_write_kb":18128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7776","reported_seq":7894,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:09.593618+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:09.593618+0000","last_peered":"2024-09-10T15:55:09.593618+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:09.593618+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7776,"log_dups_size":0,"ondisk_log_size":7776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31830016,"num_objects":7771,"num_object_clones":0,"num_object_copies":15542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5750,"num_write_kb":23000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":3939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7644","reported_seq":7764,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:09.540680+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:09.540680+0000","last_peered":"2024-09-10T15:55:09.540680+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:09.540680+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7644,"log_dups_size":0,"ondisk_log_size":7644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31285248,"num_objects":7638,"num_object_clones":0,"num_object_copies":15276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5513,"num_write_kb":22052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3809}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7765","reported_seq":7888,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.853943+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:07.853943+0000","last_peered":"2024-09-10T15:55:07.853943+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:07.853943+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7765,"log_dups_size":0,"ondisk_log_size":7765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31797248,"num_objects":7763,"num_object_clones":0,"num_object_copies":15526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4957,"num_write_kb":19828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3930},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7761","reported_seq":7920,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.964550+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:07.964550+0000","last_peered":"2024-09-10T15:55:07.964550+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:07.964550+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7761,"log_dups_size":0,"ondisk_log_size":7761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4179,"num_write_kb":16716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3836},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7694","reported_seq":7829,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.900204+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:07.900204+0000","last_peered":"2024-09-10T15:55:07.900204+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:07.900204+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7694,"log_dups_size":0,"ondisk_log_size":7694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31498240,"num_objects":7690,"num_object_clones":0,"num_object_copies":15380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4300,"num_write_kb":17200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3813},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'7914","reported_seq":8048,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:07.980628+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:07.980628+0000","last_peered":"2024-09-10T15:55:07.980628+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:07.980628+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":7914,"log_dups_size":0,"ondisk_log_size":7914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32367075,"num_objects":7903,"num_object_clones":0,"num_object_copies":15806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3900},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":507720163,"num_objects":123956,"num_object_clones":0,"num_object_copies":247912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":408,"num_objects_unfound":0,"num_objects_dirty":123956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85558,"num_write_kb":342232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3586,"num_bytes_recovered":14680064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1029636096,"data_stored":1029552997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124037,"ondisk_log_size":124037,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":351812,"kb_used_data":198004,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":94020028,"statfs":{"total":96636764160,"available":96276508672,"internally_reserved":0,"allocated":202756096,"data_stored":202428178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157478574},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149740,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698948,"kb_used_data":432692,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93672892,"statfs":{"total":96636764160,"available":95921041408,"internally_reserved":0,"allocated":443076608,"data_stored":442820950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272625325},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,9,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548524,"kb_used_data":372508,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93823316,"statfs":{"total":96636764160,"available":96075075584,"internally_reserved":0,"allocated":381448192,"data_stored":381192534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443883520,"data_stored":443879907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":203685888,"data_stored":203610015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":382066688,"data_stored":382063075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:14.818 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 157 20 127124 127104 3.16198 2.05078 0.0235776 0.0247005 2024-09-10T15:55:15.819 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 158 20 127665 127645 3.15533 2.11328 0.0442608 0.0247528 2024-09-10T15:55:16.819 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 159 20 128196 128176 3.14853 2.07422 0.0296424 0.0248062 2024-09-10T15:55:17.110 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:17.671 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:17.671 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:17.695 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-10T15:55:16.474108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":518169078,"num_objects":126509,"num_object_clones":0,"num_object_copies":253018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":197,"num_objects_unfound":0,"num_objects_dirty":126509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88111,"num_write_kb":352438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3799,"num_bytes_recovered":15544339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126591,"ondisk_log_size":126591,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1655720,"kb_used_data":1024952,"kb_used_omap":12,"kb_used_meta":630707,"kb_avail":281459800,"statfs":{"total":289910292480,"available":288214835200,"internally_reserved":0,"allocated":1049550848,"data_stored":1048711614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":645843973},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,7,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25198592,"num_objects":6152,"num_object_clones":0,"num_object_copies":12304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-624,"num_objects_unfound":0,"num_objects_dirty":6152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6152,"num_write_kb":24608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":624,"num_bytes_recovered":2555904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002966"},"pg_stats":[{"pgid":"2.f","version":"44'7961","reported_seq":8033,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:13.030440+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:13.030440+0000","last_peered":"2024-09-10T15:55:13.030440+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:13.030440+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7961,"log_dups_size":0,"ondisk_log_size":7961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32608256,"num_objects":7961,"num_object_clones":0,"num_object_copies":15922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7961,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3988},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'8133","reported_seq":8236,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:14.546108+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:14.546108+0000","last_peered":"2024-09-10T15:55:14.546108+0000","last_clean":"2024-09-10T15:55:14.546108+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:14.546108+0000","last_undegraded":"2024-09-10T15:55:14.546108+0000","last_fullsized":"2024-09-10T15:55:14.546108+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8133,"log_dups_size":0,"ondisk_log_size":8133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6469,"num_write_kb":25876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7877","reported_seq":7996,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:14.538958+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:14.538958+0000","last_peered":"2024-09-10T15:55:14.538958+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:14.538958+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7877,"log_dups_size":0,"ondisk_log_size":7877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32247808,"num_objects":7873,"num_object_clones":0,"num_object_copies":15746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6145,"num_write_kb":24580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7911","reported_seq":7986,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:14.610422+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:14.610422+0000","last_peered":"2024-09-10T15:55:14.610422+0000","last_clean":"2024-09-10T15:55:14.610422+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:14.610422+0000","last_undegraded":"2024-09-10T15:55:14.610422+0000","last_fullsized":"2024-09-10T15:55:14.610422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7911,"log_dups_size":0,"ondisk_log_size":7911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32403456,"num_objects":7911,"num_object_clones":0,"num_object_copies":15822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7911,"num_write_kb":31644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7723","reported_seq":7845,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:13.013550+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:13.013550+0000","last_peered":"2024-09-10T15:55:13.013550+0000","last_clean":"2024-09-10T15:55:13.013550+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:13.013550+0000","last_undegraded":"2024-09-10T15:55:13.013550+0000","last_fullsized":"2024-09-10T15:55:13.013550+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7723,"log_dups_size":0,"ondisk_log_size":7723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31604736,"num_objects":7716,"num_object_clones":0,"num_object_copies":15432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4794,"num_write_kb":19176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7981","reported_seq":8111,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:12.959921+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:55:12.959921+0000","last_peered":"2024-09-10T15:55:12.959921+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:12.959921+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7981,"log_dups_size":0,"ondisk_log_size":7981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32669696,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4699,"num_write_kb":18796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4108},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7749","reported_seq":7885,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:12.978439+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:12.978439+0000","last_peered":"2024-09-10T15:55:12.978439+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:12.978439+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7749,"log_dups_size":0,"ondisk_log_size":7749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31715328,"num_objects":7743,"num_object_clones":0,"num_object_copies":15486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4399,"num_write_kb":17596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3909},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7947","reported_seq":8073,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:12.946148+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:12.946148+0000","last_peered":"2024-09-10T15:55:12.946148+0000","last_clean":"2024-09-10T15:55:12.946148+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:12.946148+0000","last_undegraded":"2024-09-10T15:55:12.946148+0000","last_fullsized":"2024-09-10T15:55:12.946148+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7947,"log_dups_size":0,"ondisk_log_size":7947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32542720,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4706,"num_write_kb":18824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7985","reported_seq":15699,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:14.635490+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:55:14.635490+0000","last_peered":"2024-09-10T15:55:14.635490+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:55:14.635490+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7985,"log_dups_size":0,"ondisk_log_size":7985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32677888,"num_objects":7978,"num_object_clones":0,"num_object_copies":15956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":197,"num_objects_unfound":0,"num_objects_dirty":7978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5962,"num_write_kb":23848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3795,"num_bytes_recovered":15536128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":197},{"shards":"2","objects":7779}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7816","reported_seq":7938,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:12.963769+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:12.963769+0000","last_peered":"2024-09-10T15:55:12.963769+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:12.963769+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7816,"log_dups_size":0,"ondisk_log_size":7816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31981568,"num_objects":7808,"num_object_clones":0,"num_object_copies":15616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4680,"num_write_kb":18720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3945},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7932","reported_seq":8052,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:14.592838+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:14.592838+0000","last_peered":"2024-09-10T15:55:14.592838+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:14.592838+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7932,"log_dups_size":0,"ondisk_log_size":7932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32468992,"num_objects":7927,"num_object_clones":0,"num_object_copies":15854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5906,"num_write_kb":23624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7798","reported_seq":7920,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:14.617154+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:14.617154+0000","last_peered":"2024-09-10T15:55:14.617154+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:14.617154+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7798,"log_dups_size":0,"ondisk_log_size":7798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5667,"num_write_kb":22668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3963}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7924","reported_seq":8047,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:13.032821+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:13.032821+0000","last_peered":"2024-09-10T15:55:13.032821+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:13.032821+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7924,"log_dups_size":0,"ondisk_log_size":7924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32448512,"num_objects":7922,"num_object_clones":0,"num_object_copies":15844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5116,"num_write_kb":20464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4089},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7928","reported_seq":8087,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:12.970016+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:12.970016+0000","last_peered":"2024-09-10T15:55:12.970016+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:12.970016+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7928,"log_dups_size":0,"ondisk_log_size":7928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32440320,"num_objects":7920,"num_object_clones":0,"num_object_copies":15840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4346,"num_write_kb":17384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4003},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7849","reported_seq":7984,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:12.960117+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:12.960117+0000","last_peered":"2024-09-10T15:55:12.960117+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:12.960117+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7849,"log_dups_size":0,"ondisk_log_size":7849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32133120,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4455,"num_write_kb":17820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3968},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'8074","reported_seq":8208,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:13.014133+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:13.014133+0000","last_peered":"2024-09-10T15:55:13.014133+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:13.014133+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8074,"log_dups_size":0,"ondisk_log_size":8074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33022435,"num_objects":8063,"num_object_clones":0,"num_object_copies":16126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4893,"num_write_kb":19572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4060},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":518169059,"num_objects":126507,"num_object_clones":0,"num_object_copies":253014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":197,"num_objects_unfound":0,"num_objects_dirty":126507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88109,"num_write_kb":352436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3797,"num_bytes_recovered":15544320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1051668480,"data_stored":1051585381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126588,"ondisk_log_size":126588,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":386968,"kb_used_data":206792,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93984872,"statfs":{"total":96636764160,"available":96240508928,"internally_reserved":0,"allocated":211755008,"data_stored":211427090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184479406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149741,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710860,"kb_used_data":440444,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93660980,"statfs":{"total":96636764160,"available":95908843520,"internally_reserved":0,"allocated":451014656,"data_stored":450758998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276885165},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557892,"kb_used_data":377716,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93813948,"statfs":{"total":96636764160,"available":96065482752,"internally_reserved":0,"allocated":386781184,"data_stored":386525526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":451833856,"data_stored":451830243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":212586496,"data_stored":212510623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387248128,"data_stored":387244515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:17.697 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:17.819 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:2024-09-10T15:55:17.819018+0000 min lat: 0.00198913 max lat: 2.66872 avg lat: 0.024866 2024-09-10T15:55:17.819 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:55:17.819 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 160 20 128689 128669 3.14088 1.92578 0.0555095 0.024866 2024-09-10T15:55:18.150 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:18.151 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:18.182 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-10T15:55:16.474108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":518169078,"num_objects":126509,"num_object_clones":0,"num_object_copies":253018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":197,"num_objects_unfound":0,"num_objects_dirty":126509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88111,"num_write_kb":352438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3799,"num_bytes_recovered":15544339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126591,"ondisk_log_size":126591,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1655720,"kb_used_data":1024952,"kb_used_omap":12,"kb_used_meta":630707,"kb_avail":281459800,"statfs":{"total":289910292480,"available":288214835200,"internally_reserved":0,"allocated":1049550848,"data_stored":1048711614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":645843973},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,7,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25198592,"num_objects":6152,"num_object_clones":0,"num_object_copies":12304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-624,"num_objects_unfound":0,"num_objects_dirty":6152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6152,"num_write_kb":24608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":624,"num_bytes_recovered":2555904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002966"},"pg_stats":[{"pgid":"2.f","version":"44'7961","reported_seq":8033,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:13.030440+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:13.030440+0000","last_peered":"2024-09-10T15:55:13.030440+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:13.030440+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7961,"log_dups_size":0,"ondisk_log_size":7961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32608256,"num_objects":7961,"num_object_clones":0,"num_object_copies":15922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7961,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3988},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'8133","reported_seq":8236,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:14.546108+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:14.546108+0000","last_peered":"2024-09-10T15:55:14.546108+0000","last_clean":"2024-09-10T15:55:14.546108+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:14.546108+0000","last_undegraded":"2024-09-10T15:55:14.546108+0000","last_fullsized":"2024-09-10T15:55:14.546108+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8133,"log_dups_size":0,"ondisk_log_size":8133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6469,"num_write_kb":25876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7877","reported_seq":7996,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:14.538958+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:14.538958+0000","last_peered":"2024-09-10T15:55:14.538958+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:14.538958+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7877,"log_dups_size":0,"ondisk_log_size":7877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32247808,"num_objects":7873,"num_object_clones":0,"num_object_copies":15746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6145,"num_write_kb":24580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7911","reported_seq":7986,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:14.610422+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:14.610422+0000","last_peered":"2024-09-10T15:55:14.610422+0000","last_clean":"2024-09-10T15:55:14.610422+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:14.610422+0000","last_undegraded":"2024-09-10T15:55:14.610422+0000","last_fullsized":"2024-09-10T15:55:14.610422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7911,"log_dups_size":0,"ondisk_log_size":7911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32403456,"num_objects":7911,"num_object_clones":0,"num_object_copies":15822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7911,"num_write_kb":31644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7723","reported_seq":7845,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:13.013550+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:13.013550+0000","last_peered":"2024-09-10T15:55:13.013550+0000","last_clean":"2024-09-10T15:55:13.013550+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:13.013550+0000","last_undegraded":"2024-09-10T15:55:13.013550+0000","last_fullsized":"2024-09-10T15:55:13.013550+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7723,"log_dups_size":0,"ondisk_log_size":7723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31604736,"num_objects":7716,"num_object_clones":0,"num_object_copies":15432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4794,"num_write_kb":19176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'7981","reported_seq":8111,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:12.959921+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:55:12.959921+0000","last_peered":"2024-09-10T15:55:12.959921+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:12.959921+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":7981,"log_dups_size":0,"ondisk_log_size":7981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32669696,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4699,"num_write_kb":18796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4108},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7749","reported_seq":7885,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:12.978439+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:12.978439+0000","last_peered":"2024-09-10T15:55:12.978439+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:12.978439+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7749,"log_dups_size":0,"ondisk_log_size":7749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31715328,"num_objects":7743,"num_object_clones":0,"num_object_copies":15486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4399,"num_write_kb":17596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3909},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'7947","reported_seq":8073,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:12.946148+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:12.946148+0000","last_peered":"2024-09-10T15:55:12.946148+0000","last_clean":"2024-09-10T15:55:12.946148+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:12.946148+0000","last_undegraded":"2024-09-10T15:55:12.946148+0000","last_fullsized":"2024-09-10T15:55:12.946148+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":7947,"log_dups_size":0,"ondisk_log_size":7947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32542720,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4706,"num_write_kb":18824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7985","reported_seq":15699,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:14.635490+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:55:14.635490+0000","last_peered":"2024-09-10T15:55:14.635490+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:55:14.635490+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7985,"log_dups_size":0,"ondisk_log_size":7985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32677888,"num_objects":7978,"num_object_clones":0,"num_object_copies":15956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":197,"num_objects_unfound":0,"num_objects_dirty":7978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5962,"num_write_kb":23848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3795,"num_bytes_recovered":15536128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":197},{"shards":"2","objects":7779}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7816","reported_seq":7938,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:12.963769+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:12.963769+0000","last_peered":"2024-09-10T15:55:12.963769+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:12.963769+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7816,"log_dups_size":0,"ondisk_log_size":7816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31981568,"num_objects":7808,"num_object_clones":0,"num_object_copies":15616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4680,"num_write_kb":18720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3945},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7932","reported_seq":8052,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:14.592838+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:14.592838+0000","last_peered":"2024-09-10T15:55:14.592838+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:14.592838+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7932,"log_dups_size":0,"ondisk_log_size":7932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32468992,"num_objects":7927,"num_object_clones":0,"num_object_copies":15854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5906,"num_write_kb":23624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7798","reported_seq":7920,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:14.617154+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:14.617154+0000","last_peered":"2024-09-10T15:55:14.617154+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:14.617154+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7798,"log_dups_size":0,"ondisk_log_size":7798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5667,"num_write_kb":22668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3963}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'7924","reported_seq":8047,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:13.032821+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:13.032821+0000","last_peered":"2024-09-10T15:55:13.032821+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:13.032821+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":7924,"log_dups_size":0,"ondisk_log_size":7924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32448512,"num_objects":7922,"num_object_clones":0,"num_object_copies":15844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5116,"num_write_kb":20464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4089},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'7928","reported_seq":8087,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:12.970016+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:12.970016+0000","last_peered":"2024-09-10T15:55:12.970016+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:12.970016+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":7928,"log_dups_size":0,"ondisk_log_size":7928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32440320,"num_objects":7920,"num_object_clones":0,"num_object_copies":15840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4346,"num_write_kb":17384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4003},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'7849","reported_seq":7984,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:12.960117+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:12.960117+0000","last_peered":"2024-09-10T15:55:12.960117+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:12.960117+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":7849,"log_dups_size":0,"ondisk_log_size":7849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32133120,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4455,"num_write_kb":17820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3968},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'8074","reported_seq":8208,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:13.014133+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:13.014133+0000","last_peered":"2024-09-10T15:55:13.014133+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:13.014133+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8074,"log_dups_size":0,"ondisk_log_size":8074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33022435,"num_objects":8063,"num_object_clones":0,"num_object_copies":16126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4893,"num_write_kb":19572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4060},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":518169059,"num_objects":126507,"num_object_clones":0,"num_object_copies":253014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":197,"num_objects_unfound":0,"num_objects_dirty":126507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88109,"num_write_kb":352436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3797,"num_bytes_recovered":15544320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1051668480,"data_stored":1051585381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126588,"ondisk_log_size":126588,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":386968,"kb_used_data":206792,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93984872,"statfs":{"total":96636764160,"available":96240508928,"internally_reserved":0,"allocated":211755008,"data_stored":211427090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184479406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149741,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710860,"kb_used_data":440444,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93660980,"statfs":{"total":96636764160,"available":95908843520,"internally_reserved":0,"allocated":451014656,"data_stored":450758998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276885165},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557892,"kb_used_data":377716,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93813948,"statfs":{"total":96636764160,"available":96065482752,"internally_reserved":0,"allocated":386781184,"data_stored":386525526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":451833856,"data_stored":451830243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":212586496,"data_stored":212510623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387248128,"data_stored":387244515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:18.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:55:18.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'7961", 'reported_seq': 8033, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:13.030440+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:55:13.030440+0000', 'last_peered': '2024-09-10T15:55:13.030440+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:55:13.030440+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 7961, 'log_dups_size': 0, 'ondisk_log_size': 7961, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32608256, 'num_objects': 7961, 'num_object_clones': 0, 'num_object_copies': 15922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7961, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7961, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3988}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:18.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:55:18.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'7877", 'reported_seq': 7996, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:14.538958+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:55:14.538958+0000', 'last_peered': '2024-09-10T15:55:14.538958+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:55:14.538958+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 7877, 'log_dups_size': 0, 'ondisk_log_size': 7877, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32247808, 'num_objects': 7873, 'num_object_clones': 0, 'num_object_copies': 15746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7873, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6145, 'num_write_kb': 24580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 4071}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:18.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:55:18.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'7981", 'reported_seq': 8111, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:12.959921+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:55:12.959921+0000', 'last_peered': '2024-09-10T15:55:12.959921+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:55:12.959921+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 7981, 'log_dups_size': 0, 'ondisk_log_size': 7981, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32669696, 'num_objects': 7976, 'num_object_clones': 0, 'num_object_copies': 15952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7976, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4699, 'num_write_kb': 18796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4108}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:18.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:55:18.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'7749", 'reported_seq': 7885, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:12.978439+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:55:12.978439+0000', 'last_peered': '2024-09-10T15:55:12.978439+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:55:12.978439+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 7749, 'log_dups_size': 0, 'ondisk_log_size': 7749, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31715328, 'num_objects': 7743, 'num_object_clones': 0, 'num_object_copies': 15486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7743, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4399, 'num_write_kb': 17596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3909}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:18.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:55:18.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'7985", 'reported_seq': 15699, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:14.635490+0000', 'last_change': '2024-09-10T15:53:19.895502+0000', 'last_active': '2024-09-10T15:55:14.635490+0000', 'last_peered': '2024-09-10T15:55:14.635490+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:55:14.635490+0000', 'last_undegraded': '2024-09-10T15:53:19.503929+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 7985, 'log_dups_size': 0, 'ondisk_log_size': 7985, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32677888, 'num_objects': 7978, 'num_object_clones': 0, 'num_object_copies': 15956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 197, 'num_objects_unfound': 0, 'num_objects_dirty': 7978, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5962, 'num_write_kb': 23848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3795, 'num_bytes_recovered': 15536128, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 197}, {'shards': '2', 'objects': 7779}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:18.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:55:18.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'7816", 'reported_seq': 7938, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:12.963769+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:55:12.963769+0000', 'last_peered': '2024-09-10T15:55:12.963769+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:55:12.963769+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 7816, 'log_dups_size': 0, 'ondisk_log_size': 7816, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31981568, 'num_objects': 7808, 'num_object_clones': 0, 'num_object_copies': 15616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7808, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4680, 'num_write_kb': 18720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3945}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:18.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:55:18.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'7932", 'reported_seq': 8052, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:14.592838+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:55:14.592838+0000', 'last_peered': '2024-09-10T15:55:14.592838+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:55:14.592838+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 7932, 'log_dups_size': 0, 'ondisk_log_size': 7932, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32468992, 'num_objects': 7927, 'num_object_clones': 0, 'num_object_copies': 15854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7927, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5906, 'num_write_kb': 23624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 4095}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:18.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:55:18.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'7798", 'reported_seq': 7920, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:14.617154+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:55:14.617154+0000', 'last_peered': '2024-09-10T15:55:14.617154+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:55:14.617154+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 7798, 'log_dups_size': 0, 'ondisk_log_size': 7798, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31916032, 'num_objects': 7792, 'num_object_clones': 0, 'num_object_copies': 15584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7792, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5667, 'num_write_kb': 22668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 3963}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:18.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:55:18.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'7924", 'reported_seq': 8047, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:13.032821+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:55:13.032821+0000', 'last_peered': '2024-09-10T15:55:13.032821+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:55:13.032821+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 7924, 'log_dups_size': 0, 'ondisk_log_size': 7924, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32448512, 'num_objects': 7922, 'num_object_clones': 0, 'num_object_copies': 15844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7922, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5116, 'num_write_kb': 20464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4089}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:18.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:55:18.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'7928", 'reported_seq': 8087, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:12.970016+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:55:12.970016+0000', 'last_peered': '2024-09-10T15:55:12.970016+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:55:12.970016+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 7928, 'log_dups_size': 0, 'ondisk_log_size': 7928, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32440320, 'num_objects': 7920, 'num_object_clones': 0, 'num_object_copies': 15840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4346, 'num_write_kb': 17384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4003}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:18.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:55:18.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'7849", 'reported_seq': 7984, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:12.960117+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:55:12.960117+0000', 'last_peered': '2024-09-10T15:55:12.960117+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:55:12.960117+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 7849, 'log_dups_size': 0, 'ondisk_log_size': 7849, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32133120, 'num_objects': 7845, 'num_object_clones': 0, 'num_object_copies': 15690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4455, 'num_write_kb': 17820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3968}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:18.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:55:18.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'8074", 'reported_seq': 8208, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:13.014133+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:55:13.014133+0000', 'last_peered': '2024-09-10T15:55:13.014133+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:55:13.014133+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 8074, 'log_dups_size': 0, 'ondisk_log_size': 8074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33022435, 'num_objects': 8063, 'num_object_clones': 0, 'num_object_copies': 16126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8063, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4893, 'num_write_kb': 19572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4060}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:18.186 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:55:18.819 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 161 20 129112 129092 3.13164 1.65234 0.0222754 0.0249392 2024-09-10T15:55:18.878 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:18.878 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42653/256284 objects degraded (16.643%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":244,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":128142,"data_bytes":524857846,"bytes_used":1725984768,"bytes_avail":288184307712,"bytes_total":289910292480,"degraded_objects":42653,"degraded_total":256284,"degraded_ratio":0.16642864946699754,"misplaced_objects":197,"misplaced_total":256284,"misplaced_ratio":0.00076867849729206664,"recovering_objects_per_sec":35,"recovering_bytes_per_sec":143666,"recovering_keys_per_sec":0,"num_objects_recovered":421,"num_bytes_recovered":1724416,"num_keys_recovered":0,"write_bytes_sec":2322885,"read_op_per_sec":0,"write_op_per_sec":567},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:55:18.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:55:18.911 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:19.450 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:19.453 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:19.483 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-10T15:55:18.474588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":524857846,"num_objects":128142,"num_object_clones":0,"num_object_copies":256284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":197,"num_objects_unfound":0,"num_objects_dirty":128142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89744,"num_write_kb":358970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3799,"num_bytes_recovered":15544339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128224,"ondisk_log_size":128224,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1685532,"kb_used_data":1032552,"kb_used_omap":12,"kb_used_meta":652915,"kb_avail":281429988,"statfs":{"total":289910292480,"available":288184307712,"internally_reserved":0,"allocated":1057333248,"data_stored":1056494014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":668584965},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,7,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27881472,"num_objects":6807,"num_object_clones":0,"num_object_copies":13614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-421,"num_objects_unfound":0,"num_objects_dirty":6807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6807,"num_write_kb":27228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":1724416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002946"},"pg_stats":[{"pgid":"2.f","version":"44'8111","reported_seq":8183,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:18.011681+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:18.011681+0000","last_peered":"2024-09-10T15:55:18.011681+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:18.011681+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8111,"log_dups_size":0,"ondisk_log_size":8111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33222656,"num_objects":8111,"num_object_clones":0,"num_object_copies":16222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8111,"num_write_kb":32444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4138},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'8133","reported_seq":8236,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:14.546108+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:14.546108+0000","last_peered":"2024-09-10T15:55:14.546108+0000","last_clean":"2024-09-10T15:55:14.546108+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:14.546108+0000","last_undegraded":"2024-09-10T15:55:14.546108+0000","last_fullsized":"2024-09-10T15:55:14.546108+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8133,"log_dups_size":0,"ondisk_log_size":8133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6469,"num_write_kb":25876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'7877","reported_seq":7996,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:14.538958+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:14.538958+0000","last_peered":"2024-09-10T15:55:14.538958+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:14.538958+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":7877,"log_dups_size":0,"ondisk_log_size":7877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32247808,"num_objects":7873,"num_object_clones":0,"num_object_copies":15746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6145,"num_write_kb":24580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'7911","reported_seq":7986,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:14.610422+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:14.610422+0000","last_peered":"2024-09-10T15:55:14.610422+0000","last_clean":"2024-09-10T15:55:14.610422+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:14.610422+0000","last_undegraded":"2024-09-10T15:55:14.610422+0000","last_fullsized":"2024-09-10T15:55:14.610422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":7911,"log_dups_size":0,"ondisk_log_size":7911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32403456,"num_objects":7911,"num_object_clones":0,"num_object_copies":15822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7911,"num_write_kb":31644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7883","reported_seq":8005,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:18.030720+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:18.030720+0000","last_peered":"2024-09-10T15:55:18.030720+0000","last_clean":"2024-09-10T15:55:18.030720+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:18.030720+0000","last_undegraded":"2024-09-10T15:55:18.030720+0000","last_fullsized":"2024-09-10T15:55:18.030720+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7883,"log_dups_size":0,"ondisk_log_size":7883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32260096,"num_objects":7876,"num_object_clones":0,"num_object_copies":15752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4954,"num_write_kb":19816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'8154","reported_seq":8284,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.960284+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:55:17.960284+0000","last_peered":"2024-09-10T15:55:17.960284+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:17.960284+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8154,"log_dups_size":0,"ondisk_log_size":8154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33378304,"num_objects":8149,"num_object_clones":0,"num_object_copies":16298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4872,"num_write_kb":19488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7901","reported_seq":8037,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.983808+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:17.983808+0000","last_peered":"2024-09-10T15:55:17.983808+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:17.983808+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7901,"log_dups_size":0,"ondisk_log_size":7901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32337920,"num_objects":7895,"num_object_clones":0,"num_object_copies":15790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4551,"num_write_kb":18204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4061},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'8120","reported_seq":8247,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:18.025699+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:18.025699+0000","last_peered":"2024-09-10T15:55:18.025699+0000","last_clean":"2024-09-10T15:55:18.025699+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:18.025699+0000","last_undegraded":"2024-09-10T15:55:18.025699+0000","last_fullsized":"2024-09-10T15:55:18.025699+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8120,"log_dups_size":0,"ondisk_log_size":8120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33251328,"num_objects":8118,"num_object_clones":0,"num_object_copies":16236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4879,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.476005+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.476005+0000","last_peered":"2024-09-10T15:53:19.476005+0000","last_clean":"2024-09-10T15:53:19.476005+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.476005+0000","last_undegraded":"2024-09-10T15:53:19.476005+0000","last_fullsized":"2024-09-10T15:53:19.476005+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'7985","reported_seq":15699,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:14.635490+0000","last_change":"2024-09-10T15:53:19.895502+0000","last_active":"2024-09-10T15:55:14.635490+0000","last_peered":"2024-09-10T15:55:14.635490+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:55:14.635490+0000","last_undegraded":"2024-09-10T15:53:19.503929+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":7985,"log_dups_size":0,"ondisk_log_size":7985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32677888,"num_objects":7978,"num_object_clones":0,"num_object_copies":15956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":197,"num_objects_unfound":0,"num_objects_dirty":7978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5962,"num_write_kb":23848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3795,"num_bytes_recovered":15536128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":197},{"shards":"2","objects":7779}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7982","reported_seq":8104,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.998555+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:17.998555+0000","last_peered":"2024-09-10T15:55:17.998555+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:17.998555+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7982,"log_dups_size":0,"ondisk_log_size":7982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32661504,"num_objects":7974,"num_object_clones":0,"num_object_copies":15948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4846,"num_write_kb":19384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4111},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'7932","reported_seq":8052,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:14.592838+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:14.592838+0000","last_peered":"2024-09-10T15:55:14.592838+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:14.592838+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":7932,"log_dups_size":0,"ondisk_log_size":7932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32468992,"num_objects":7927,"num_object_clones":0,"num_object_copies":15854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5906,"num_write_kb":23624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7798","reported_seq":7920,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:14.617154+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:14.617154+0000","last_peered":"2024-09-10T15:55:14.617154+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:14.617154+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7798,"log_dups_size":0,"ondisk_log_size":7798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5667,"num_write_kb":22668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":3963}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'8094","reported_seq":8217,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:18.012124+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:18.012124+0000","last_peered":"2024-09-10T15:55:18.012124+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:18.012124+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8094,"log_dups_size":0,"ondisk_log_size":8094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33144832,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5286,"num_write_kb":21144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4259},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'8073","reported_seq":8232,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.992164+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:17.992164+0000","last_peered":"2024-09-10T15:55:17.992164+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:17.992164+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8073,"log_dups_size":0,"ondisk_log_size":8073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33034240,"num_objects":8065,"num_object_clones":0,"num_object_copies":16130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4491,"num_write_kb":17964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4148},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'8014","reported_seq":8149,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.963874+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:17.963874+0000","last_peered":"2024-09-10T15:55:17.963874+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:17.963874+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8014,"log_dups_size":0,"ondisk_log_size":8014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32808960,"num_objects":8010,"num_object_clones":0,"num_object_copies":16020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4620,"num_write_kb":18480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4133},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'8253","reported_seq":8387,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:18.001167+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:18.001167+0000","last_peered":"2024-09-10T15:55:18.001167+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:18.001167+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8253,"log_dups_size":0,"ondisk_log_size":8253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33755619,"num_objects":8242,"num_object_clones":0,"num_object_copies":16484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5072,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4239},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":524857827,"num_objects":128140,"num_object_clones":0,"num_object_copies":256280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42653,"num_objects_misplaced":197,"num_objects_unfound":0,"num_objects_dirty":128140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89742,"num_write_kb":358968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3797,"num_bytes_recovered":15544320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1059553280,"data_stored":1059470181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128221,"ondisk_log_size":128221,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":386968,"kb_used_data":206792,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93984872,"statfs":{"total":96636764160,"available":96240508928,"internally_reserved":0,"allocated":211755008,"data_stored":211427090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184479406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149742,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":740672,"kb_used_data":448044,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93631168,"statfs":{"total":96636764160,"available":95878316032,"internally_reserved":0,"allocated":458797056,"data_stored":458541398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299626157},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557892,"kb_used_data":377716,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93813948,"statfs":{"total":96636764160,"available":96065482752,"internally_reserved":0,"allocated":386781184,"data_stored":386525526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":459718656,"data_stored":459715043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":212586496,"data_stored":212510623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387248128,"data_stored":387244515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:19.819 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 162 19 129586 129567 3.12376 1.85547 0.028367 0.0249997 2024-09-10T15:55:20.819 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 163 20 129795 129775 3.10958 0.8125 0.0306518 0.0251177 2024-09-10T15:55:21.819 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 164 20 130334 130314 3.10346 2.10547 0.0457318 0.0251661 2024-09-10T15:55:22.485 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:22.820 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 165 20 130852 130832 3.09691 2.02344 0.0686664 0.0252195 2024-09-10T15:55:23.033 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:23.034 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:23.074 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-10T15:55:22.475428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":528577014,"num_objects":129050,"num_object_clones":0,"num_object_copies":258100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90652,"num_write_kb":362602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3998,"num_bytes_recovered":16359443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129132,"ondisk_log_size":129132,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1725472,"kb_used_data":1046124,"kb_used_omap":12,"kb_used_meta":679283,"kb_avail":281390048,"statfs":{"total":289910292480,"available":288143409152,"internally_reserved":0,"allocated":1071230976,"data_stored":1070391742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":695585797},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,9,6,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14241792,"num_objects":3477,"num_object_clones":0,"num_object_copies":6954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":-408,"num_objects_unfound":0,"num_objects_dirty":3477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3477,"num_write_kb":13908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":1679360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001802"},"pg_stats":[{"pgid":"2.f","version":"44'8111","reported_seq":8183,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:18.011681+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:18.011681+0000","last_peered":"2024-09-10T15:55:18.011681+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:18.011681+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8111,"log_dups_size":0,"ondisk_log_size":8111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33222656,"num_objects":8111,"num_object_clones":0,"num_object_copies":16222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8111,"num_write_kb":32444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4138},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'8287","reported_seq":8390,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:19.571953+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:19.571953+0000","last_peered":"2024-09-10T15:55:19.571953+0000","last_clean":"2024-09-10T15:55:19.571953+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:19.571953+0000","last_undegraded":"2024-09-10T15:55:19.571953+0000","last_fullsized":"2024-09-10T15:55:19.571953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8287,"log_dups_size":0,"ondisk_log_size":8287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33918976,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6623,"num_write_kb":26492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'8014","reported_seq":8132,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:19.522996+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:19.522996+0000","last_peered":"2024-09-10T15:55:19.522996+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:19.522996+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8014,"log_dups_size":0,"ondisk_log_size":8014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32808960,"num_objects":8010,"num_object_clones":0,"num_object_copies":16020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6282,"num_write_kb":25128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4208}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'8052","reported_seq":8127,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:19.392290+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:19.392290+0000","last_peered":"2024-09-10T15:55:19.392290+0000","last_clean":"2024-09-10T15:55:19.392290+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:19.392290+0000","last_undegraded":"2024-09-10T15:55:19.392290+0000","last_fullsized":"2024-09-10T15:55:19.392290+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8052,"log_dups_size":0,"ondisk_log_size":8052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32980992,"num_objects":8052,"num_object_clones":0,"num_object_copies":16104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8052,"num_write_kb":32208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7883","reported_seq":8005,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:18.030720+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:18.030720+0000","last_peered":"2024-09-10T15:55:18.030720+0000","last_clean":"2024-09-10T15:55:18.030720+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:18.030720+0000","last_undegraded":"2024-09-10T15:55:18.030720+0000","last_fullsized":"2024-09-10T15:55:18.030720+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7883,"log_dups_size":0,"ondisk_log_size":7883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32260096,"num_objects":7876,"num_object_clones":0,"num_object_copies":15752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4954,"num_write_kb":19816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'8154","reported_seq":8284,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.960284+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:55:17.960284+0000","last_peered":"2024-09-10T15:55:17.960284+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:17.960284+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8154,"log_dups_size":0,"ondisk_log_size":8154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33378304,"num_objects":8149,"num_object_clones":0,"num_object_copies":16298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4872,"num_write_kb":19488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7901","reported_seq":8037,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.983808+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:17.983808+0000","last_peered":"2024-09-10T15:55:17.983808+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:17.983808+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7901,"log_dups_size":0,"ondisk_log_size":7901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32337920,"num_objects":7895,"num_object_clones":0,"num_object_copies":15790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4551,"num_write_kb":18204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4061},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'8120","reported_seq":8247,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:18.025699+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:18.025699+0000","last_peered":"2024-09-10T15:55:18.025699+0000","last_clean":"2024-09-10T15:55:18.025699+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:18.025699+0000","last_undegraded":"2024-09-10T15:55:18.025699+0000","last_fullsized":"2024-09-10T15:55:18.025699+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8120,"log_dups_size":0,"ondisk_log_size":8120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33251328,"num_objects":8118,"num_object_clones":0,"num_object_copies":16236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4879,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.478695+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.478695+0000","last_peered":"2024-09-10T15:53:19.478695+0000","last_clean":"2024-09-10T15:53:19.478695+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.478695+0000","last_undegraded":"2024-09-10T15:53:19.478695+0000","last_fullsized":"2024-09-10T15:53:19.478695+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'8134","reported_seq":16248,"reported_epoch":44,"state":"active+undersized+remapped","last_fresh":"2024-09-10T15:55:19.634746+0000","last_change":"2024-09-10T15:55:19.342351+0000","last_active":"2024-09-10T15:55:19.634746+0000","last_peered":"2024-09-10T15:55:19.634746+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:55:19.634746+0000","last_undegraded":"2024-09-10T15:55:19.634746+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8134,"log_dups_size":0,"ondisk_log_size":8134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6111,"num_write_kb":24444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":8127}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7982","reported_seq":8104,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.998555+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:17.998555+0000","last_peered":"2024-09-10T15:55:17.998555+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:17.998555+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7982,"log_dups_size":0,"ondisk_log_size":7982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32661504,"num_objects":7974,"num_object_clones":0,"num_object_copies":15948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4846,"num_write_kb":19384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4111},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'8091","reported_seq":8210,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:19.586604+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:19.586604+0000","last_peered":"2024-09-10T15:55:19.586604+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:19.586604+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8091,"log_dups_size":0,"ondisk_log_size":8091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33120256,"num_objects":8086,"num_object_clones":0,"num_object_copies":16172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6065,"num_write_kb":24260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4254}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7966","reported_seq":8088,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:19.623764+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:19.623764+0000","last_peered":"2024-09-10T15:55:19.623764+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:19.623764+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7966,"log_dups_size":0,"ondisk_log_size":7966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32604160,"num_objects":7960,"num_object_clones":0,"num_object_copies":15920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5835,"num_write_kb":23340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'8094","reported_seq":8217,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:18.012124+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:18.012124+0000","last_peered":"2024-09-10T15:55:18.012124+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:18.012124+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8094,"log_dups_size":0,"ondisk_log_size":8094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33144832,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5286,"num_write_kb":21144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4259},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'8073","reported_seq":8232,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.992164+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:17.992164+0000","last_peered":"2024-09-10T15:55:17.992164+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:17.992164+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8073,"log_dups_size":0,"ondisk_log_size":8073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33034240,"num_objects":8065,"num_object_clones":0,"num_object_copies":16130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4491,"num_write_kb":17964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4148},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'8014","reported_seq":8149,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.963874+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:17.963874+0000","last_peered":"2024-09-10T15:55:17.963874+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:17.963874+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8014,"log_dups_size":0,"ondisk_log_size":8014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32808960,"num_objects":8010,"num_object_clones":0,"num_object_copies":16020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4620,"num_write_kb":18480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4133},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'8253","reported_seq":8387,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:18.001167+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:18.001167+0000","last_peered":"2024-09-10T15:55:18.001167+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:18.001167+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8253,"log_dups_size":0,"ondisk_log_size":8253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33755619,"num_objects":8242,"num_object_clones":0,"num_object_copies":16484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5072,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4239},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":528576995,"num_objects":129048,"num_object_clones":0,"num_object_copies":258096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90650,"num_write_kb":362600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3996,"num_bytes_recovered":16359424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073348608,"data_stored":1073265509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129129,"ondisk_log_size":129129,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399644,"kb_used_data":215308,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93972196,"statfs":{"total":96636764160,"available":96227528704,"internally_reserved":0,"allocated":220475392,"data_stored":220147474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188739246},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149742,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":740672,"kb_used_data":448044,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93631168,"statfs":{"total":96636764160,"available":95878316032,"internally_reserved":0,"allocated":458797056,"data_stored":458541398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299626157},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585156,"kb_used_data":382772,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93786684,"statfs":{"total":96636764160,"available":96037564416,"internally_reserved":0,"allocated":391958528,"data_stored":391702870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":459718656,"data_stored":459715043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221339648,"data_stored":221263775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392290304,"data_stored":392286691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:23.075 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:23.554 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:23.578 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-10T15:55:22.475428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":528577014,"num_objects":129050,"num_object_clones":0,"num_object_copies":258100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90652,"num_write_kb":362602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3998,"num_bytes_recovered":16359443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129132,"ondisk_log_size":129132,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1725472,"kb_used_data":1046124,"kb_used_omap":12,"kb_used_meta":679283,"kb_avail":281390048,"statfs":{"total":289910292480,"available":288143409152,"internally_reserved":0,"allocated":1071230976,"data_stored":1070391742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":695585797},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,9,6,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14241792,"num_objects":3477,"num_object_clones":0,"num_object_copies":6954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":-408,"num_objects_unfound":0,"num_objects_dirty":3477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3477,"num_write_kb":13908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":1679360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001802"},"pg_stats":[{"pgid":"2.f","version":"44'8111","reported_seq":8183,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:18.011681+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:18.011681+0000","last_peered":"2024-09-10T15:55:18.011681+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:18.011681+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8111,"log_dups_size":0,"ondisk_log_size":8111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33222656,"num_objects":8111,"num_object_clones":0,"num_object_copies":16222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8111,"num_write_kb":32444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4138},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'8287","reported_seq":8390,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:19.571953+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:19.571953+0000","last_peered":"2024-09-10T15:55:19.571953+0000","last_clean":"2024-09-10T15:55:19.571953+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:19.571953+0000","last_undegraded":"2024-09-10T15:55:19.571953+0000","last_fullsized":"2024-09-10T15:55:19.571953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8287,"log_dups_size":0,"ondisk_log_size":8287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33918976,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6623,"num_write_kb":26492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'8014","reported_seq":8132,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:19.522996+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:19.522996+0000","last_peered":"2024-09-10T15:55:19.522996+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:19.522996+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8014,"log_dups_size":0,"ondisk_log_size":8014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32808960,"num_objects":8010,"num_object_clones":0,"num_object_copies":16020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6282,"num_write_kb":25128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4208}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'8052","reported_seq":8127,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:19.392290+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:19.392290+0000","last_peered":"2024-09-10T15:55:19.392290+0000","last_clean":"2024-09-10T15:55:19.392290+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:19.392290+0000","last_undegraded":"2024-09-10T15:55:19.392290+0000","last_fullsized":"2024-09-10T15:55:19.392290+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8052,"log_dups_size":0,"ondisk_log_size":8052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32980992,"num_objects":8052,"num_object_clones":0,"num_object_copies":16104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8052,"num_write_kb":32208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'7883","reported_seq":8005,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:18.030720+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:18.030720+0000","last_peered":"2024-09-10T15:55:18.030720+0000","last_clean":"2024-09-10T15:55:18.030720+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:18.030720+0000","last_undegraded":"2024-09-10T15:55:18.030720+0000","last_fullsized":"2024-09-10T15:55:18.030720+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":7883,"log_dups_size":0,"ondisk_log_size":7883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32260096,"num_objects":7876,"num_object_clones":0,"num_object_copies":15752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4954,"num_write_kb":19816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'8154","reported_seq":8284,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.960284+0000","last_change":"2024-09-10T15:53:19.898325+0000","last_active":"2024-09-10T15:55:17.960284+0000","last_peered":"2024-09-10T15:55:17.960284+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:17.960284+0000","last_undegraded":"2024-09-10T15:53:19.535041+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8154,"log_dups_size":0,"ondisk_log_size":8154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33378304,"num_objects":8149,"num_object_clones":0,"num_object_copies":16298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4872,"num_write_kb":19488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":3868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'7901","reported_seq":8037,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.983808+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:17.983808+0000","last_peered":"2024-09-10T15:55:17.983808+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:17.983808+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":7901,"log_dups_size":0,"ondisk_log_size":7901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32337920,"num_objects":7895,"num_object_clones":0,"num_object_copies":15790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4551,"num_write_kb":18204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4061},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'8120","reported_seq":8247,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:18.025699+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:18.025699+0000","last_peered":"2024-09-10T15:55:18.025699+0000","last_clean":"2024-09-10T15:55:18.025699+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:18.025699+0000","last_undegraded":"2024-09-10T15:55:18.025699+0000","last_fullsized":"2024-09-10T15:55:18.025699+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8120,"log_dups_size":0,"ondisk_log_size":8120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33251328,"num_objects":8118,"num_object_clones":0,"num_object_copies":16236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4879,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.478695+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:53:19.478695+0000","last_peered":"2024-09-10T15:53:19.478695+0000","last_clean":"2024-09-10T15:53:19.478695+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:53:19.478695+0000","last_undegraded":"2024-09-10T15:53:19.478695+0000","last_fullsized":"2024-09-10T15:53:19.478695+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'8134","reported_seq":16248,"reported_epoch":44,"state":"active+undersized+remapped","last_fresh":"2024-09-10T15:55:19.634746+0000","last_change":"2024-09-10T15:55:19.342351+0000","last_active":"2024-09-10T15:55:19.634746+0000","last_peered":"2024-09-10T15:55:19.634746+0000","last_clean":"2024-09-10T15:53:17.326712+0000","last_became_active":"2024-09-10T15:53:19.871150+0000","last_became_peered":"2024-09-10T15:53:19.871150+0000","last_unstale":"2024-09-10T15:55:19.634746+0000","last_undegraded":"2024-09-10T15:55:19.634746+0000","last_fullsized":"2024-09-10T15:53:19.470440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8134,"log_dups_size":0,"ondisk_log_size":8134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6111,"num_write_kb":24444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":8127}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'7982","reported_seq":8104,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.998555+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:17.998555+0000","last_peered":"2024-09-10T15:55:17.998555+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:17.998555+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":7982,"log_dups_size":0,"ondisk_log_size":7982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32661504,"num_objects":7974,"num_object_clones":0,"num_object_copies":15948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4846,"num_write_kb":19384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4111},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.892514+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:53:19.892514+0000","last_peered":"2024-09-10T15:53:19.892514+0000","last_clean":"2024-09-10T15:53:19.892514+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:53:19.892514+0000","last_undegraded":"2024-09-10T15:53:19.892514+0000","last_fullsized":"2024-09-10T15:53:19.892514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'8091","reported_seq":8210,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:19.586604+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:19.586604+0000","last_peered":"2024-09-10T15:55:19.586604+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:19.586604+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8091,"log_dups_size":0,"ondisk_log_size":8091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33120256,"num_objects":8086,"num_object_clones":0,"num_object_copies":16172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6065,"num_write_kb":24260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4254}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":73,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.558580+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:53:19.558580+0000","last_peered":"2024-09-10T15:53:19.558580+0000","last_clean":"2024-09-10T15:53:19.558580+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:53:19.558580+0000","last_undegraded":"2024-09-10T15:53:19.558580+0000","last_fullsized":"2024-09-10T15:53:19.558580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7966","reported_seq":8088,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:19.623764+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:19.623764+0000","last_peered":"2024-09-10T15:55:19.623764+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:19.623764+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7966,"log_dups_size":0,"ondisk_log_size":7966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32604160,"num_objects":7960,"num_object_clones":0,"num_object_copies":15920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5835,"num_write_kb":23340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'8094","reported_seq":8217,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:18.012124+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:18.012124+0000","last_peered":"2024-09-10T15:55:18.012124+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:18.012124+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8094,"log_dups_size":0,"ondisk_log_size":8094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33144832,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5286,"num_write_kb":21144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4259},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599583+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:53:19.599583+0000","last_peered":"2024-09-10T15:53:19.599583+0000","last_clean":"2024-09-10T15:53:19.599583+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:53:19.599583+0000","last_undegraded":"2024-09-10T15:53:19.599583+0000","last_fullsized":"2024-09-10T15:53:19.599583+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'8073","reported_seq":8232,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.992164+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:17.992164+0000","last_peered":"2024-09-10T15:55:17.992164+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:17.992164+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8073,"log_dups_size":0,"ondisk_log_size":8073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33034240,"num_objects":8065,"num_object_clones":0,"num_object_copies":16130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4491,"num_write_kb":17964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4148},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'8014","reported_seq":8149,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:17.963874+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:17.963874+0000","last_peered":"2024-09-10T15:55:17.963874+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:17.963874+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8014,"log_dups_size":0,"ondisk_log_size":8014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32808960,"num_objects":8010,"num_object_clones":0,"num_object_copies":16020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4620,"num_write_kb":18480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4133},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.599049+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:53:19.599049+0000","last_peered":"2024-09-10T15:53:19.599049+0000","last_clean":"2024-09-10T15:53:19.599049+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:53:19.599049+0000","last_undegraded":"2024-09-10T15:53:19.599049+0000","last_fullsized":"2024-09-10T15:53:19.599049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.473733+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:53:19.473733+0000","last_peered":"2024-09-10T15:53:19.473733+0000","last_clean":"2024-09-10T15:53:19.473733+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:53:19.473733+0000","last_undegraded":"2024-09-10T15:53:19.473733+0000","last_fullsized":"2024-09-10T15:53:19.473733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'8253","reported_seq":8387,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:18.001167+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:18.001167+0000","last_peered":"2024-09-10T15:55:18.001167+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:18.001167+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8253,"log_dups_size":0,"ondisk_log_size":8253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33755619,"num_objects":8242,"num_object_clones":0,"num_object_copies":16484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5072,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4239},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":528576995,"num_objects":129048,"num_object_clones":0,"num_object_copies":258096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90650,"num_write_kb":362600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3996,"num_bytes_recovered":16359424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073348608,"data_stored":1073265509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129129,"ondisk_log_size":129129,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399644,"kb_used_data":215308,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93972196,"statfs":{"total":96636764160,"available":96227528704,"internally_reserved":0,"allocated":220475392,"data_stored":220147474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188739246},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149742,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":740672,"kb_used_data":448044,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93631168,"statfs":{"total":96636764160,"available":95878316032,"internally_reserved":0,"allocated":458797056,"data_stored":458541398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299626157},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585156,"kb_used_data":382772,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93786684,"statfs":{"total":96636764160,"available":96037564416,"internally_reserved":0,"allocated":391958528,"data_stored":391702870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":459718656,"data_stored":459715043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221339648,"data_stored":221263775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392290304,"data_stored":392286691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:23.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:55:23.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'8111", 'reported_seq': 8183, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:18.011681+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:55:18.011681+0000', 'last_peered': '2024-09-10T15:55:18.011681+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:55:18.011681+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 8111, 'log_dups_size': 0, 'ondisk_log_size': 8111, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33222656, 'num_objects': 8111, 'num_object_clones': 0, 'num_object_copies': 16222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8111, 'num_write_kb': 32444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4138}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:23.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:55:23.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'8014", 'reported_seq': 8132, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:19.522996+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:55:19.522996+0000', 'last_peered': '2024-09-10T15:55:19.522996+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:55:19.522996+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 8014, 'log_dups_size': 0, 'ondisk_log_size': 8014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32808960, 'num_objects': 8010, 'num_object_clones': 0, 'num_object_copies': 16020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8010, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6282, 'num_write_kb': 25128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 4208}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:23.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:55:23.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'8154", 'reported_seq': 8284, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:17.960284+0000', 'last_change': '2024-09-10T15:53:19.898325+0000', 'last_active': '2024-09-10T15:55:17.960284+0000', 'last_peered': '2024-09-10T15:55:17.960284+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:55:17.960284+0000', 'last_undegraded': '2024-09-10T15:53:19.535041+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 8154, 'log_dups_size': 0, 'ondisk_log_size': 8154, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33378304, 'num_objects': 8149, 'num_object_clones': 0, 'num_object_copies': 16298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3870, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8149, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4872, 'num_write_kb': 19488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4281}, {'shards': '0,2', 'objects': 3868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:23.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:55:23.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'7901", 'reported_seq': 8037, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:17.983808+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:55:17.983808+0000', 'last_peered': '2024-09-10T15:55:17.983808+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:55:17.983808+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 7901, 'log_dups_size': 0, 'ondisk_log_size': 7901, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32337920, 'num_objects': 7895, 'num_object_clones': 0, 'num_object_copies': 15790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7895, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4551, 'num_write_kb': 18204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4061}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:23.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-10T15:55:23.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'8134", 'reported_seq': 16248, 'reported_epoch': 44, 'state': 'active+undersized+remapped', 'last_fresh': '2024-09-10T15:55:19.634746+0000', 'last_change': '2024-09-10T15:55:19.342351+0000', 'last_active': '2024-09-10T15:55:19.634746+0000', 'last_peered': '2024-09-10T15:55:19.634746+0000', 'last_clean': '2024-09-10T15:53:17.326712+0000', 'last_became_active': '2024-09-10T15:53:19.871150+0000', 'last_became_peered': '2024-09-10T15:53:19.871150+0000', 'last_unstale': '2024-09-10T15:55:19.634746+0000', 'last_undegraded': '2024-09-10T15:55:19.634746+0000', 'last_fullsized': '2024-09-10T15:53:19.470440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2072", 'last_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:52.645684+0000', 'objects_scrubbed': 2016, 'log_size': 8134, 'log_dups_size': 0, 'ondisk_log_size': 8134, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 750, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33288192, 'num_objects': 8127, 'num_object_clones': 0, 'num_object_copies': 16254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6111, 'num_write_kb': 24444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3994, 'num_bytes_recovered': 16351232, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [{'shards': '2', 'objects': 8127}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:23.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:55:23.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'7982", 'reported_seq': 8104, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:17.998555+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:55:17.998555+0000', 'last_peered': '2024-09-10T15:55:17.998555+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:55:17.998555+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 7982, 'log_dups_size': 0, 'ondisk_log_size': 7982, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32661504, 'num_objects': 7974, 'num_object_clones': 0, 'num_object_copies': 15948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7974, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4846, 'num_write_kb': 19384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4111}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:23.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:55:23.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'8091", 'reported_seq': 8210, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:19.586604+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:55:19.586604+0000', 'last_peered': '2024-09-10T15:55:19.586604+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:55:19.586604+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 8091, 'log_dups_size': 0, 'ondisk_log_size': 8091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33120256, 'num_objects': 8086, 'num_object_clones': 0, 'num_object_copies': 16172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8086, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6065, 'num_write_kb': 24260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 4254}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:23.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:55:23.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'7966", 'reported_seq': 8088, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:19.623764+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:55:19.623764+0000', 'last_peered': '2024-09-10T15:55:19.623764+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:55:19.623764+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 7966, 'log_dups_size': 0, 'ondisk_log_size': 7966, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32604160, 'num_objects': 7960, 'num_object_clones': 0, 'num_object_copies': 15920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7960, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5835, 'num_write_kb': 23340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 4131}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:23.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:55:23.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'8094", 'reported_seq': 8217, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:18.012124+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:55:18.012124+0000', 'last_peered': '2024-09-10T15:55:18.012124+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:55:18.012124+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 8094, 'log_dups_size': 0, 'ondisk_log_size': 8094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33144832, 'num_objects': 8092, 'num_object_clones': 0, 'num_object_copies': 16184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8092, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5286, 'num_write_kb': 21144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4259}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:23.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:55:23.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'8073", 'reported_seq': 8232, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:17.992164+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:55:17.992164+0000', 'last_peered': '2024-09-10T15:55:17.992164+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:55:17.992164+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 8073, 'log_dups_size': 0, 'ondisk_log_size': 8073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33034240, 'num_objects': 8065, 'num_object_clones': 0, 'num_object_copies': 16130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4491, 'num_write_kb': 17964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4148}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:23.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:55:23.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'8014", 'reported_seq': 8149, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:17.963874+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:55:17.963874+0000', 'last_peered': '2024-09-10T15:55:17.963874+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:55:17.963874+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 8014, 'log_dups_size': 0, 'ondisk_log_size': 8014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32808960, 'num_objects': 8010, 'num_object_clones': 0, 'num_object_copies': 16020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8010, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4620, 'num_write_kb': 18480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4133}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:23.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:55:23.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'8253", 'reported_seq': 8387, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:18.001167+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:55:18.001167+0000', 'last_peered': '2024-09-10T15:55:18.001167+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:55:18.001167+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 8253, 'log_dups_size': 0, 'ondisk_log_size': 8253, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33755619, 'num_objects': 8242, 'num_object_clones': 0, 'num_object_copies': 16484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8242, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5072, 'num_write_kb': 20288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4239}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:23.582 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:55:23.820 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 166 20 131279 131259 3.0883 1.66797 0.0276026 0.025289 2024-09-10T15:55:24.284 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:24.284 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42651/258100 objects degraded (16.525%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":250,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":129050,"data_bytes":528577014,"bytes_used":1766883328,"bytes_avail":288143409152,"bytes_total":289910292480,"degraded_objects":42651,"degraded_total":258100,"degraded_ratio":0.16524990313831847,"recovering_objects_per_sec":51,"recovering_bytes_per_sec":209872,"recovering_keys_per_sec":0,"num_objects_recovered":410,"num_bytes_recovered":1679360,"num_keys_recovered":0,"write_bytes_sec":1779823,"read_op_per_sec":0,"write_op_per_sec":434},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:55:24.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:55:24.311 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:24.820 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 167 20 131736 131716 3.0805 1.78516 0.0351923 0.0253528 2024-09-10T15:55:24.858 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:24.861 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:24.883 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-09-10T15:55:24.475883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":534692342,"num_objects":130543,"num_object_clones":0,"num_object_copies":261086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3758,"num_objects_unfound":0,"num_objects_dirty":130543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92145,"num_write_kb":368574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4108,"num_bytes_recovered":16810003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130625,"ondisk_log_size":130625,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1732204,"kb_used_data":1059772,"kb_used_omap":12,"kb_used_meta":672371,"kb_avail":281383316,"statfs":{"total":289910292480,"available":288136515584,"internally_reserved":0,"allocated":1085206528,"data_stored":1084346504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":688507909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,1,9,7,13,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16523264,"num_objects":4034,"num_object_clones":0,"num_object_copies":8068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3872,"num_objects_misplaced":3561,"num_objects_unfound":0,"num_objects_dirty":4034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4034,"num_write_kb":16136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001774"},"pg_stats":[{"pgid":"2.f","version":"46'8239","reported_seq":8315,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.070028+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:23.070028+0000","last_peered":"2024-09-10T15:55:23.070028+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:23.070028+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8239,"log_dups_size":0,"ondisk_log_size":8239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33746944,"num_objects":8239,"num_object_clones":0,"num_object_copies":16478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8239,"num_write_kb":32956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4266},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'8287","reported_seq":8390,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:19.571953+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:19.571953+0000","last_peered":"2024-09-10T15:55:19.571953+0000","last_clean":"2024-09-10T15:55:19.571953+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:19.571953+0000","last_undegraded":"2024-09-10T15:55:19.571953+0000","last_fullsized":"2024-09-10T15:55:19.571953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8287,"log_dups_size":0,"ondisk_log_size":8287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33918976,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6623,"num_write_kb":26492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'8014","reported_seq":8132,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:19.522996+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:19.522996+0000","last_peered":"2024-09-10T15:55:19.522996+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:19.522996+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8014,"log_dups_size":0,"ondisk_log_size":8014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32808960,"num_objects":8010,"num_object_clones":0,"num_object_copies":16020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6282,"num_write_kb":25128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4208}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'8052","reported_seq":8127,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:55:19.392290+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:19.392290+0000","last_peered":"2024-09-10T15:55:19.392290+0000","last_clean":"2024-09-10T15:55:19.392290+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:19.392290+0000","last_undegraded":"2024-09-10T15:55:19.392290+0000","last_fullsized":"2024-09-10T15:55:19.392290+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8052,"log_dups_size":0,"ondisk_log_size":8052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32980992,"num_objects":8052,"num_object_clones":0,"num_object_copies":16104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8052,"num_write_kb":32208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8010","reported_seq":8136,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:22.974097+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:22.974097+0000","last_peered":"2024-09-10T15:55:22.974097+0000","last_clean":"2024-09-10T15:55:22.974097+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:22.974097+0000","last_undegraded":"2024-09-10T15:55:22.974097+0000","last_fullsized":"2024-09-10T15:55:22.974097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8010,"log_dups_size":0,"ondisk_log_size":8010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32780288,"num_objects":8003,"num_object_clones":0,"num_object_copies":16006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5081,"num_write_kb":20324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8294","reported_seq":8649,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:23.108500+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:23.108500+0000","last_peered":"2024-09-10T15:55:23.108500+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:23.108500+0000","last_undegraded":"2024-09-10T15:55:23.108500+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8294,"log_dups_size":0,"ondisk_log_size":8294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33951744,"num_objects":8289,"num_object_clones":0,"num_object_copies":16578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3758,"num_objects_unfound":0,"num_objects_dirty":8289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5012,"num_write_kb":20048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":450560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4531},{"shards":"0,2","objects":3758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8043","reported_seq":8183,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.027003+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:23.027003+0000","last_peered":"2024-09-10T15:55:23.027003+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:23.027003+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8043,"log_dups_size":0,"ondisk_log_size":8043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32919552,"num_objects":8037,"num_object_clones":0,"num_object_copies":16074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4693,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4203},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8255","reported_seq":8385,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.013191+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:23.013191+0000","last_peered":"2024-09-10T15:55:23.013191+0000","last_clean":"2024-09-10T15:55:23.013191+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:23.013191+0000","last_undegraded":"2024-09-10T15:55:23.013191+0000","last_fullsized":"2024-09-10T15:55:23.013191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8255,"log_dups_size":0,"ondisk_log_size":8255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33804288,"num_objects":8253,"num_object_clones":0,"num_object_copies":16506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5014,"num_write_kb":20056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8248","reported_seq":16375,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.668016+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:23.668016+0000","last_peered":"2024-09-10T15:55:23.668016+0000","last_clean":"2024-09-10T15:55:23.668016+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:23.668016+0000","last_undegraded":"2024-09-10T15:55:23.668016+0000","last_fullsized":"2024-09-10T15:55:23.668016+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8248,"log_dups_size":0,"ondisk_log_size":8248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33755136,"num_objects":8241,"num_object_clones":0,"num_object_copies":16482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6225,"num_write_kb":24900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.470714+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:53:19.470714+0000","last_peered":"2024-09-10T15:53:19.470714+0000","last_clean":"2024-09-10T15:53:19.470714+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:53:19.470714+0000","last_undegraded":"2024-09-10T15:53:19.470714+0000","last_fullsized":"2024-09-10T15:53:19.470714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8119","reported_seq":8245,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.035731+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:23.035731+0000","last_peered":"2024-09-10T15:55:23.035731+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:23.035731+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8119,"log_dups_size":0,"ondisk_log_size":8119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33222656,"num_objects":8111,"num_object_clones":0,"num_object_copies":16222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4983,"num_write_kb":19932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4248},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'8091","reported_seq":8210,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:19.586604+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:19.586604+0000","last_peered":"2024-09-10T15:55:19.586604+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:19.586604+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8091,"log_dups_size":0,"ondisk_log_size":8091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33120256,"num_objects":8086,"num_object_clones":0,"num_object_copies":16172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6065,"num_write_kb":24260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4254}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'7966","reported_seq":8088,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:19.623764+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:19.623764+0000","last_peered":"2024-09-10T15:55:19.623764+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:19.623764+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":7966,"log_dups_size":0,"ondisk_log_size":7966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32604160,"num_objects":7960,"num_object_clones":0,"num_object_copies":15920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5835,"num_write_kb":23340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8243","reported_seq":8369,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.113391+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:23.113391+0000","last_peered":"2024-09-10T15:55:23.113391+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:23.113391+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8243,"log_dups_size":0,"ondisk_log_size":8243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33755136,"num_objects":8241,"num_object_clones":0,"num_object_copies":16482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5435,"num_write_kb":21740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4408},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8206","reported_seq":8368,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.113240+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:23.113240+0000","last_peered":"2024-09-10T15:55:23.113240+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:23.113240+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8206,"log_dups_size":0,"ondisk_log_size":8206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33579008,"num_objects":8198,"num_object_clones":0,"num_object_copies":16396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4624,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T15:53:19.469321+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:53:19.469321+0000","last_peered":"2024-09-10T15:53:19.469321+0000","last_clean":"2024-09-10T15:53:19.469321+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:53:19.469321+0000","last_undegraded":"2024-09-10T15:53:19.469321+0000","last_fullsized":"2024-09-10T15:53:19.469321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8158","reported_seq":8297,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.084275+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:23.084275+0000","last_peered":"2024-09-10T15:55:23.084275+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:23.084275+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8158,"log_dups_size":0,"ondisk_log_size":8158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33398784,"num_objects":8154,"num_object_clones":0,"num_object_copies":16308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4764,"num_write_kb":19056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4277},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8397","reported_seq":8534,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.092696+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:23.092696+0000","last_peered":"2024-09-10T15:55:23.092696+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:23.092696+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8397,"log_dups_size":0,"ondisk_log_size":8397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34345443,"num_objects":8386,"num_object_clones":0,"num_object_copies":16772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5216,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4383},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":534692323,"num_objects":130541,"num_object_clones":0,"num_object_copies":261082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3758,"num_objects_unfound":0,"num_objects_dirty":130541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92143,"num_write_kb":368572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4106,"num_bytes_recovered":16809984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1087754240,"data_stored":1087671141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130622,"ondisk_log_size":130622,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412452,"kb_used_data":222548,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93959388,"statfs":{"total":96636764160,"available":96214413312,"internally_reserved":0,"allocated":227889152,"data_stored":227550839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":194440878},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149743,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":734596,"kb_used_data":454452,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93637244,"statfs":{"total":96636764160,"available":95884537856,"internally_reserved":0,"allocated":465358848,"data_stored":465092795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286846637},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585156,"kb_used_data":382772,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93786684,"statfs":{"total":96636764160,"available":96037564416,"internally_reserved":0,"allocated":391958528,"data_stored":391702870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466567168,"data_stored":466563555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":228896768,"data_stored":228820895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392290304,"data_stored":392286691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:25.821 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 168 20 132264 132244 3.07442 2.0625 0.0506866 0.0254039 2024-09-10T15:55:26.821 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 169 20 132782 132762 3.0682 2.02344 0.0305454 0.0254545 2024-09-10T15:55:27.821 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 170 20 133309 133289 3.06226 2.05859 0.0319055 0.0255043 2024-09-10T15:55:27.886 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:28.448 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:28.474 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-10T15:55:26.476386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":537461238,"num_objects":131219,"num_object_clones":0,"num_object_copies":262438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3758,"num_objects_unfound":0,"num_objects_dirty":131219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92821,"num_write_kb":371278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4108,"num_bytes_recovered":16810003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131301,"ondisk_log_size":131301,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1776776,"kb_used_data":1064088,"kb_used_omap":12,"kb_used_meta":712627,"kb_avail":281338744,"statfs":{"total":289910292480,"available":288090873856,"internally_reserved":0,"allocated":1089626112,"data_stored":1088755693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":729730053},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,10,6,12,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12603392,"num_objects":3077,"num_object_clones":0,"num_object_copies":6154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3872,"num_objects_misplaced":3561,"num_objects_unfound":0,"num_objects_dirty":3077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3077,"num_write_kb":12308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001798"},"pg_stats":[{"pgid":"2.f","version":"46'8239","reported_seq":8315,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.070028+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:23.070028+0000","last_peered":"2024-09-10T15:55:23.070028+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:23.070028+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8239,"log_dups_size":0,"ondisk_log_size":8239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33746944,"num_objects":8239,"num_object_clones":0,"num_object_copies":16478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8239,"num_write_kb":32956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4266},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8416","reported_seq":8523,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:24.677054+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:24.677054+0000","last_peered":"2024-09-10T15:55:24.677054+0000","last_clean":"2024-09-10T15:55:24.677054+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:24.677054+0000","last_undegraded":"2024-09-10T15:55:24.677054+0000","last_fullsized":"2024-09-10T15:55:24.677054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8416,"log_dups_size":0,"ondisk_log_size":8416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8143","reported_seq":8266,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:24.666209+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:24.666209+0000","last_peered":"2024-09-10T15:55:24.666209+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:24.666209+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8143,"log_dups_size":0,"ondisk_log_size":8143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33337344,"num_objects":8139,"num_object_clones":0,"num_object_copies":16278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6411,"num_write_kb":25644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4337}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8203","reported_seq":8282,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:24.547451+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:24.547451+0000","last_peered":"2024-09-10T15:55:24.547451+0000","last_clean":"2024-09-10T15:55:24.547451+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:24.547451+0000","last_undegraded":"2024-09-10T15:55:24.547451+0000","last_fullsized":"2024-09-10T15:55:24.547451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8203,"log_dups_size":0,"ondisk_log_size":8203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33599488,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8203,"num_write_kb":32812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8010","reported_seq":8136,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:22.974097+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:22.974097+0000","last_peered":"2024-09-10T15:55:22.974097+0000","last_clean":"2024-09-10T15:55:22.974097+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:22.974097+0000","last_undegraded":"2024-09-10T15:55:22.974097+0000","last_fullsized":"2024-09-10T15:55:22.974097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8010,"log_dups_size":0,"ondisk_log_size":8010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32780288,"num_objects":8003,"num_object_clones":0,"num_object_copies":16006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5081,"num_write_kb":20324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8294","reported_seq":8649,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:23.108500+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:23.108500+0000","last_peered":"2024-09-10T15:55:23.108500+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:23.108500+0000","last_undegraded":"2024-09-10T15:55:23.108500+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8294,"log_dups_size":0,"ondisk_log_size":8294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33951744,"num_objects":8289,"num_object_clones":0,"num_object_copies":16578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3758,"num_objects_unfound":0,"num_objects_dirty":8289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5012,"num_write_kb":20048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":450560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4531},{"shards":"0,2","objects":3758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8043","reported_seq":8183,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.027003+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:23.027003+0000","last_peered":"2024-09-10T15:55:23.027003+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:23.027003+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8043,"log_dups_size":0,"ondisk_log_size":8043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32919552,"num_objects":8037,"num_object_clones":0,"num_object_copies":16074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4693,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4203},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8255","reported_seq":8385,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.013191+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:23.013191+0000","last_peered":"2024-09-10T15:55:23.013191+0000","last_clean":"2024-09-10T15:55:23.013191+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:23.013191+0000","last_undegraded":"2024-09-10T15:55:23.013191+0000","last_fullsized":"2024-09-10T15:55:23.013191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8255,"log_dups_size":0,"ondisk_log_size":8255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33804288,"num_objects":8253,"num_object_clones":0,"num_object_copies":16506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5014,"num_write_kb":20056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8248","reported_seq":16375,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.668016+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:23.668016+0000","last_peered":"2024-09-10T15:55:23.668016+0000","last_clean":"2024-09-10T15:55:23.668016+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:23.668016+0000","last_undegraded":"2024-09-10T15:55:23.668016+0000","last_fullsized":"2024-09-10T15:55:23.668016+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8248,"log_dups_size":0,"ondisk_log_size":8248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33755136,"num_objects":8241,"num_object_clones":0,"num_object_copies":16482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6225,"num_write_kb":24900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8119","reported_seq":8245,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.035731+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:23.035731+0000","last_peered":"2024-09-10T15:55:23.035731+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:23.035731+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8119,"log_dups_size":0,"ondisk_log_size":8119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33222656,"num_objects":8111,"num_object_clones":0,"num_object_copies":16222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4983,"num_write_kb":19932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4248},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8213","reported_seq":8337,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:24.648120+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:24.648120+0000","last_peered":"2024-09-10T15:55:24.648120+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:24.648120+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8213,"log_dups_size":0,"ondisk_log_size":8213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":8208,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6187,"num_write_kb":24748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4376}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8111","reported_seq":8237,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:24.546195+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:24.546195+0000","last_peered":"2024-09-10T15:55:24.546195+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:24.546195+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8111,"log_dups_size":0,"ondisk_log_size":8111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33198080,"num_objects":8105,"num_object_clones":0,"num_object_copies":16210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5980,"num_write_kb":23920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4276}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8243","reported_seq":8369,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.113391+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:23.113391+0000","last_peered":"2024-09-10T15:55:23.113391+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:23.113391+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8243,"log_dups_size":0,"ondisk_log_size":8243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33755136,"num_objects":8241,"num_object_clones":0,"num_object_copies":16482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5435,"num_write_kb":21740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4408},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8206","reported_seq":8368,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.113240+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:23.113240+0000","last_peered":"2024-09-10T15:55:23.113240+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:23.113240+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8206,"log_dups_size":0,"ondisk_log_size":8206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33579008,"num_objects":8198,"num_object_clones":0,"num_object_copies":16396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4624,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8158","reported_seq":8297,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.084275+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:23.084275+0000","last_peered":"2024-09-10T15:55:23.084275+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:23.084275+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8158,"log_dups_size":0,"ondisk_log_size":8158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33398784,"num_objects":8154,"num_object_clones":0,"num_object_copies":16308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4764,"num_write_kb":19056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4277},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8397","reported_seq":8534,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:23.092696+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:23.092696+0000","last_peered":"2024-09-10T15:55:23.092696+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:23.092696+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8397,"log_dups_size":0,"ondisk_log_size":8397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34345443,"num_objects":8386,"num_object_clones":0,"num_object_copies":16772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5216,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4383},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":537461219,"num_objects":131217,"num_object_clones":0,"num_object_copies":262434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3758,"num_objects_unfound":0,"num_objects_dirty":131217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92819,"num_write_kb":371276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4106,"num_bytes_recovered":16809984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1092182016,"data_stored":1092098917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131298,"ondisk_log_size":131298,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412452,"kb_used_data":222548,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93959388,"statfs":{"total":96636764160,"available":96214413312,"internally_reserved":0,"allocated":227889152,"data_stored":227550839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":194440878},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149743,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":734596,"kb_used_data":454452,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93637244,"statfs":{"total":96636764160,"available":95884537856,"internally_reserved":0,"allocated":465358848,"data_stored":465092795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286846637},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629728,"kb_used_data":387088,"kb_used_omap":4,"kb_used_meta":242619,"kb_avail":93742112,"statfs":{"total":96636764160,"available":95991922688,"internally_reserved":0,"allocated":396378112,"data_stored":396112059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":248442538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466567168,"data_stored":466563555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":228896768,"data_stored":228820895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396718080,"data_stored":396714467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:28.475 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:28.821 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 171 20 133783 133763 3.05518 1.85156 0.0541227 0.0255627 2024-09-10T15:55:29.026 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:29.027 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:29.048 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-10T15:55:28.476863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":543732214,"num_objects":132750,"num_object_clones":0,"num_object_copies":265500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3671,"num_objects_unfound":0,"num_objects_dirty":132750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94352,"num_write_kb":377402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":17166355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132832,"ondisk_log_size":132832,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1793788,"kb_used_data":1072780,"kb_used_omap":12,"kb_used_meta":720947,"kb_avail":281321732,"statfs":{"total":289910292480,"available":288073453568,"internally_reserved":0,"allocated":1098526720,"data_stored":1097656301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":738249733},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,5,5,3,13,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15155200,"num_objects":3700,"num_object_clones":0,"num_object_copies":7400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3870,"num_objects_misplaced":3671,"num_objects_unfound":0,"num_objects_dirty":3700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3700,"num_write_kb":14800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.811761"},"pg_stats":[{"pgid":"2.f","version":"46'8381","reported_seq":8457,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.172908+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:28.172908+0000","last_peered":"2024-09-10T15:55:28.172908+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:28.172908+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8381,"log_dups_size":0,"ondisk_log_size":8381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34328576,"num_objects":8381,"num_object_clones":0,"num_object_copies":16762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8381,"num_write_kb":33524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4408},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8416","reported_seq":8523,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:24.677054+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:24.677054+0000","last_peered":"2024-09-10T15:55:24.677054+0000","last_clean":"2024-09-10T15:55:24.677054+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:24.677054+0000","last_undegraded":"2024-09-10T15:55:24.677054+0000","last_fullsized":"2024-09-10T15:55:24.677054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8416,"log_dups_size":0,"ondisk_log_size":8416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8143","reported_seq":8266,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:24.666209+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:24.666209+0000","last_peered":"2024-09-10T15:55:24.666209+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:24.666209+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8143,"log_dups_size":0,"ondisk_log_size":8143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33337344,"num_objects":8139,"num_object_clones":0,"num_object_copies":16278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6411,"num_write_kb":25644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4337}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8203","reported_seq":8282,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:24.547451+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:24.547451+0000","last_peered":"2024-09-10T15:55:24.547451+0000","last_clean":"2024-09-10T15:55:24.547451+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:24.547451+0000","last_undegraded":"2024-09-10T15:55:24.547451+0000","last_fullsized":"2024-09-10T15:55:24.547451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8203,"log_dups_size":0,"ondisk_log_size":8203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33599488,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8203,"num_write_kb":32812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8169","reported_seq":8295,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:28.154555+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:28.154555+0000","last_peered":"2024-09-10T15:55:28.154555+0000","last_clean":"2024-09-10T15:55:28.154555+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:28.154555+0000","last_undegraded":"2024-09-10T15:55:28.154555+0000","last_fullsized":"2024-09-10T15:55:28.154555+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8169,"log_dups_size":0,"ondisk_log_size":8169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33431552,"num_objects":8162,"num_object_clones":0,"num_object_copies":16324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5240,"num_write_kb":20960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8439","reported_seq":8968,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:28.183751+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:28.183751+0000","last_peered":"2024-09-10T15:55:28.183751+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:28.183751+0000","last_undegraded":"2024-09-10T15:55:28.183751+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8439,"log_dups_size":0,"ondisk_log_size":8439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34545664,"num_objects":8434,"num_object_clones":0,"num_object_copies":16868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3671,"num_objects_unfound":0,"num_objects_dirty":8434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5157,"num_write_kb":20628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4763},{"shards":"0,2","objects":3671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8174","reported_seq":8314,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.151977+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:28.151977+0000","last_peered":"2024-09-10T15:55:28.151977+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:28.151977+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8174,"log_dups_size":0,"ondisk_log_size":8174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33456128,"num_objects":8168,"num_object_clones":0,"num_object_copies":16336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":19296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4334},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8425","reported_seq":8556,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:28.103971+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:28.103971+0000","last_peered":"2024-09-10T15:55:28.103971+0000","last_clean":"2024-09-10T15:55:28.103971+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:28.103971+0000","last_undegraded":"2024-09-10T15:55:28.103971+0000","last_fullsized":"2024-09-10T15:55:28.103971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8425,"log_dups_size":0,"ondisk_log_size":8425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34500608,"num_objects":8423,"num_object_clones":0,"num_object_copies":16846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5184,"num_write_kb":20736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8248","reported_seq":16375,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.668016+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:23.668016+0000","last_peered":"2024-09-10T15:55:23.668016+0000","last_clean":"2024-09-10T15:55:23.668016+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:23.668016+0000","last_undegraded":"2024-09-10T15:55:23.668016+0000","last_fullsized":"2024-09-10T15:55:23.668016+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8248,"log_dups_size":0,"ondisk_log_size":8248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33755136,"num_objects":8241,"num_object_clones":0,"num_object_copies":16482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6225,"num_write_kb":24900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8282","reported_seq":8408,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.113289+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:28.113289+0000","last_peered":"2024-09-10T15:55:28.113289+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:28.113289+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8282,"log_dups_size":0,"ondisk_log_size":8282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33890304,"num_objects":8274,"num_object_clones":0,"num_object_copies":16548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5146,"num_write_kb":20584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4411},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8213","reported_seq":8337,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:24.648120+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:24.648120+0000","last_peered":"2024-09-10T15:55:24.648120+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:24.648120+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8213,"log_dups_size":0,"ondisk_log_size":8213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":8208,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6187,"num_write_kb":24748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4376}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8111","reported_seq":8237,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:24.546195+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:24.546195+0000","last_peered":"2024-09-10T15:55:24.546195+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:24.546195+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8111,"log_dups_size":0,"ondisk_log_size":8111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33198080,"num_objects":8105,"num_object_clones":0,"num_object_copies":16210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5980,"num_write_kb":23920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4276}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8404","reported_seq":8531,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.124870+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:28.124870+0000","last_peered":"2024-09-10T15:55:28.124870+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:28.124870+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8404,"log_dups_size":0,"ondisk_log_size":8404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34414592,"num_objects":8402,"num_object_clones":0,"num_object_copies":16804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5596,"num_write_kb":22384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4569},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8367","reported_seq":8530,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.067794+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:28.067794+0000","last_peered":"2024-09-10T15:55:28.067794+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:28.067794+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8367,"log_dups_size":0,"ondisk_log_size":8367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34238464,"num_objects":8359,"num_object_clones":0,"num_object_copies":16718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4785,"num_write_kb":19140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8311","reported_seq":8449,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.168476+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:28.168476+0000","last_peered":"2024-09-10T15:55:28.168476+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:28.168476+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8311,"log_dups_size":0,"ondisk_log_size":8311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34025472,"num_objects":8307,"num_object_clones":0,"num_object_copies":16614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4917,"num_write_kb":19668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4430},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8543","reported_seq":8680,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.164699+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:28.164699+0000","last_peered":"2024-09-10T15:55:28.164699+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:28.164699+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8543,"log_dups_size":0,"ondisk_log_size":8543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34943459,"num_objects":8532,"num_object_clones":0,"num_object_copies":17064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5362,"num_write_kb":21448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4529},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":543732195,"num_objects":132748,"num_object_clones":0,"num_object_copies":265496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3671,"num_objects_unfound":0,"num_objects_dirty":132748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94350,"num_write_kb":377400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4193,"num_bytes_recovered":17166336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1099685888,"data_stored":1099602789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132829,"ondisk_log_size":132829,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412452,"kb_used_data":222548,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93959388,"statfs":{"total":96636764160,"available":96214413312,"internally_reserved":0,"allocated":227889152,"data_stored":227550839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":194440878},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149744,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751608,"kb_used_data":463144,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93620232,"statfs":{"total":96636764160,"available":95867117568,"internally_reserved":0,"allocated":474259456,"data_stored":473993403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295366317},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,1,1,5,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629728,"kb_used_data":387088,"kb_used_omap":4,"kb_used_meta":242619,"kb_avail":93742112,"statfs":{"total":96636764160,"available":95991922688,"internally_reserved":0,"allocated":396378112,"data_stored":396112059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":248442538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":474071040,"data_stored":474067427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":228896768,"data_stored":228820895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396718080,"data_stored":396714467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:29.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:55:29.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'8381", 'reported_seq': 8457, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:28.172908+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:55:28.172908+0000', 'last_peered': '2024-09-10T15:55:28.172908+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:55:28.172908+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 8381, 'log_dups_size': 0, 'ondisk_log_size': 8381, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34328576, 'num_objects': 8381, 'num_object_clones': 0, 'num_object_copies': 16762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8381, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8381, 'num_write_kb': 33524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4408}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:29.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:55:29.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'8143", 'reported_seq': 8266, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:24.666209+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:55:24.666209+0000', 'last_peered': '2024-09-10T15:55:24.666209+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:55:24.666209+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 8143, 'log_dups_size': 0, 'ondisk_log_size': 8143, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33337344, 'num_objects': 8139, 'num_object_clones': 0, 'num_object_copies': 16278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8139, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6411, 'num_write_kb': 25644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 4337}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:29.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:55:29.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'8439", 'reported_seq': 8968, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:55:28.183751+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:55:28.183751+0000', 'last_peered': '2024-09-10T15:55:28.183751+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:55:28.183751+0000', 'last_undegraded': '2024-09-10T15:55:28.183751+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 8439, 'log_dups_size': 0, 'ondisk_log_size': 8439, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34545664, 'num_objects': 8434, 'num_object_clones': 0, 'num_object_copies': 16868, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3671, 'num_objects_unfound': 0, 'num_objects_dirty': 8434, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5157, 'num_write_kb': 20628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 197, 'num_bytes_recovered': 806912, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4763}, {'shards': '0,2', 'objects': 3671}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:29.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:55:29.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'8174", 'reported_seq': 8314, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:28.151977+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:55:28.151977+0000', 'last_peered': '2024-09-10T15:55:28.151977+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:55:28.151977+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 8174, 'log_dups_size': 0, 'ondisk_log_size': 8174, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33456128, 'num_objects': 8168, 'num_object_clones': 0, 'num_object_copies': 16336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4824, 'num_write_kb': 19296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4334}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:29.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:55:29.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'8282", 'reported_seq': 8408, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:28.113289+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:55:28.113289+0000', 'last_peered': '2024-09-10T15:55:28.113289+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:55:28.113289+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 8282, 'log_dups_size': 0, 'ondisk_log_size': 8282, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33890304, 'num_objects': 8274, 'num_object_clones': 0, 'num_object_copies': 16548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8274, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5146, 'num_write_kb': 20584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4411}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:29.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:55:29.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'8213", 'reported_seq': 8337, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:24.648120+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:55:24.648120+0000', 'last_peered': '2024-09-10T15:55:24.648120+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:55:24.648120+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 8213, 'log_dups_size': 0, 'ondisk_log_size': 8213, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33619968, 'num_objects': 8208, 'num_object_clones': 0, 'num_object_copies': 16416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8208, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6187, 'num_write_kb': 24748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 4376}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:29.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:55:29.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'8111", 'reported_seq': 8237, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:24.546195+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:55:24.546195+0000', 'last_peered': '2024-09-10T15:55:24.546195+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:55:24.546195+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 8111, 'log_dups_size': 0, 'ondisk_log_size': 8111, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33198080, 'num_objects': 8105, 'num_object_clones': 0, 'num_object_copies': 16210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5980, 'num_write_kb': 23920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 4276}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:29.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:55:29.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'8404", 'reported_seq': 8531, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:28.124870+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:55:28.124870+0000', 'last_peered': '2024-09-10T15:55:28.124870+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:55:28.124870+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 8404, 'log_dups_size': 0, 'ondisk_log_size': 8404, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34414592, 'num_objects': 8402, 'num_object_clones': 0, 'num_object_copies': 16804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8402, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5596, 'num_write_kb': 22384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4569}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:29.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:55:29.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'8367", 'reported_seq': 8530, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:28.067794+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:55:28.067794+0000', 'last_peered': '2024-09-10T15:55:28.067794+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:55:28.067794+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 8367, 'log_dups_size': 0, 'ondisk_log_size': 8367, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34238464, 'num_objects': 8359, 'num_object_clones': 0, 'num_object_copies': 16718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4785, 'num_write_kb': 19140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4442}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:29.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:55:29.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'8311", 'reported_seq': 8449, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:28.168476+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:55:28.168476+0000', 'last_peered': '2024-09-10T15:55:28.168476+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:55:28.168476+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 8311, 'log_dups_size': 0, 'ondisk_log_size': 8311, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34025472, 'num_objects': 8307, 'num_object_clones': 0, 'num_object_copies': 16614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8307, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4917, 'num_write_kb': 19668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4430}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:29.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:55:29.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'8543", 'reported_seq': 8680, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:28.164699+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:55:28.164699+0000', 'last_peered': '2024-09-10T15:55:28.164699+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:55:28.164699+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 8543, 'log_dups_size': 0, 'ondisk_log_size': 8543, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34943459, 'num_objects': 8532, 'num_object_clones': 0, 'num_object_copies': 17064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8532, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5362, 'num_write_kb': 21448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4529}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:29.052 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:55:29.728 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:29.728 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/265500 objects degraded (14.607%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":255,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":132750,"data_bytes":543732214,"bytes_used":1836838912,"bytes_avail":288073453568,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":265500,"degraded_ratio":0.1460677966101695,"misplaced_objects":3671,"misplaced_total":265500,"misplaced_ratio":0.013826741996233521,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":91572,"recovering_keys_per_sec":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"write_bytes_sec":1719883,"read_op_per_sec":0,"write_op_per_sec":419},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:55:29.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:55:29.753 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:29.821 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 172 20 134240 134220 3.0478 1.78516 0.0413821 0.0256255 2024-09-10T15:55:30.322 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:30.322 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:30.347 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-10T15:55:28.476863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":543732214,"num_objects":132750,"num_object_clones":0,"num_object_copies":265500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3671,"num_objects_unfound":0,"num_objects_dirty":132750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94352,"num_write_kb":377402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":17166355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132832,"ondisk_log_size":132832,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1793788,"kb_used_data":1072780,"kb_used_omap":12,"kb_used_meta":720947,"kb_avail":281321732,"statfs":{"total":289910292480,"available":288073453568,"internally_reserved":0,"allocated":1098526720,"data_stored":1097656301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":738249733},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,5,5,3,13,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15155200,"num_objects":3700,"num_object_clones":0,"num_object_copies":7400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3870,"num_objects_misplaced":3671,"num_objects_unfound":0,"num_objects_dirty":3700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3700,"num_write_kb":14800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.811761"},"pg_stats":[{"pgid":"2.f","version":"46'8381","reported_seq":8457,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.172908+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:28.172908+0000","last_peered":"2024-09-10T15:55:28.172908+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:28.172908+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8381,"log_dups_size":0,"ondisk_log_size":8381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34328576,"num_objects":8381,"num_object_clones":0,"num_object_copies":16762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8381,"num_write_kb":33524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4408},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8416","reported_seq":8523,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:24.677054+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:24.677054+0000","last_peered":"2024-09-10T15:55:24.677054+0000","last_clean":"2024-09-10T15:55:24.677054+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:24.677054+0000","last_undegraded":"2024-09-10T15:55:24.677054+0000","last_fullsized":"2024-09-10T15:55:24.677054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8416,"log_dups_size":0,"ondisk_log_size":8416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8143","reported_seq":8266,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:24.666209+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:24.666209+0000","last_peered":"2024-09-10T15:55:24.666209+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:24.666209+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8143,"log_dups_size":0,"ondisk_log_size":8143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33337344,"num_objects":8139,"num_object_clones":0,"num_object_copies":16278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6411,"num_write_kb":25644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4337}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8203","reported_seq":8282,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:24.547451+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:24.547451+0000","last_peered":"2024-09-10T15:55:24.547451+0000","last_clean":"2024-09-10T15:55:24.547451+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:24.547451+0000","last_undegraded":"2024-09-10T15:55:24.547451+0000","last_fullsized":"2024-09-10T15:55:24.547451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8203,"log_dups_size":0,"ondisk_log_size":8203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33599488,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8203,"num_write_kb":32812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8169","reported_seq":8295,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:28.154555+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:28.154555+0000","last_peered":"2024-09-10T15:55:28.154555+0000","last_clean":"2024-09-10T15:55:28.154555+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:28.154555+0000","last_undegraded":"2024-09-10T15:55:28.154555+0000","last_fullsized":"2024-09-10T15:55:28.154555+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8169,"log_dups_size":0,"ondisk_log_size":8169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33431552,"num_objects":8162,"num_object_clones":0,"num_object_copies":16324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5240,"num_write_kb":20960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8439","reported_seq":8968,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:28.183751+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:28.183751+0000","last_peered":"2024-09-10T15:55:28.183751+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:28.183751+0000","last_undegraded":"2024-09-10T15:55:28.183751+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8439,"log_dups_size":0,"ondisk_log_size":8439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34545664,"num_objects":8434,"num_object_clones":0,"num_object_copies":16868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3671,"num_objects_unfound":0,"num_objects_dirty":8434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5157,"num_write_kb":20628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4763},{"shards":"0,2","objects":3671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8174","reported_seq":8314,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.151977+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:28.151977+0000","last_peered":"2024-09-10T15:55:28.151977+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:28.151977+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8174,"log_dups_size":0,"ondisk_log_size":8174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33456128,"num_objects":8168,"num_object_clones":0,"num_object_copies":16336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":19296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4334},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8425","reported_seq":8556,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:28.103971+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:28.103971+0000","last_peered":"2024-09-10T15:55:28.103971+0000","last_clean":"2024-09-10T15:55:28.103971+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:28.103971+0000","last_undegraded":"2024-09-10T15:55:28.103971+0000","last_fullsized":"2024-09-10T15:55:28.103971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8425,"log_dups_size":0,"ondisk_log_size":8425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34500608,"num_objects":8423,"num_object_clones":0,"num_object_copies":16846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5184,"num_write_kb":20736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8248","reported_seq":16375,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.668016+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:23.668016+0000","last_peered":"2024-09-10T15:55:23.668016+0000","last_clean":"2024-09-10T15:55:23.668016+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:23.668016+0000","last_undegraded":"2024-09-10T15:55:23.668016+0000","last_fullsized":"2024-09-10T15:55:23.668016+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8248,"log_dups_size":0,"ondisk_log_size":8248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33755136,"num_objects":8241,"num_object_clones":0,"num_object_copies":16482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6225,"num_write_kb":24900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8282","reported_seq":8408,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.113289+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:28.113289+0000","last_peered":"2024-09-10T15:55:28.113289+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:28.113289+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8282,"log_dups_size":0,"ondisk_log_size":8282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33890304,"num_objects":8274,"num_object_clones":0,"num_object_copies":16548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5146,"num_write_kb":20584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4411},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8213","reported_seq":8337,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:24.648120+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:24.648120+0000","last_peered":"2024-09-10T15:55:24.648120+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:24.648120+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8213,"log_dups_size":0,"ondisk_log_size":8213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":8208,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6187,"num_write_kb":24748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4376}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8111","reported_seq":8237,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:24.546195+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:24.546195+0000","last_peered":"2024-09-10T15:55:24.546195+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:24.546195+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8111,"log_dups_size":0,"ondisk_log_size":8111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33198080,"num_objects":8105,"num_object_clones":0,"num_object_copies":16210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5980,"num_write_kb":23920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4276}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8404","reported_seq":8531,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.124870+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:28.124870+0000","last_peered":"2024-09-10T15:55:28.124870+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:28.124870+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8404,"log_dups_size":0,"ondisk_log_size":8404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34414592,"num_objects":8402,"num_object_clones":0,"num_object_copies":16804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5596,"num_write_kb":22384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4569},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8367","reported_seq":8530,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.067794+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:28.067794+0000","last_peered":"2024-09-10T15:55:28.067794+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:28.067794+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8367,"log_dups_size":0,"ondisk_log_size":8367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34238464,"num_objects":8359,"num_object_clones":0,"num_object_copies":16718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4785,"num_write_kb":19140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8311","reported_seq":8449,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.168476+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:28.168476+0000","last_peered":"2024-09-10T15:55:28.168476+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:28.168476+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8311,"log_dups_size":0,"ondisk_log_size":8311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34025472,"num_objects":8307,"num_object_clones":0,"num_object_copies":16614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4917,"num_write_kb":19668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4430},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8543","reported_seq":8680,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.164699+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:28.164699+0000","last_peered":"2024-09-10T15:55:28.164699+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:28.164699+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8543,"log_dups_size":0,"ondisk_log_size":8543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34943459,"num_objects":8532,"num_object_clones":0,"num_object_copies":17064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5362,"num_write_kb":21448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4529},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":543732195,"num_objects":132748,"num_object_clones":0,"num_object_copies":265496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3671,"num_objects_unfound":0,"num_objects_dirty":132748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94350,"num_write_kb":377400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4193,"num_bytes_recovered":17166336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1099685888,"data_stored":1099602789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132829,"ondisk_log_size":132829,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412452,"kb_used_data":222548,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93959388,"statfs":{"total":96636764160,"available":96214413312,"internally_reserved":0,"allocated":227889152,"data_stored":227550839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":194440878},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149744,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751608,"kb_used_data":463144,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93620232,"statfs":{"total":96636764160,"available":95867117568,"internally_reserved":0,"allocated":474259456,"data_stored":473993403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295366317},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,1,1,5,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629728,"kb_used_data":387088,"kb_used_omap":4,"kb_used_meta":242619,"kb_avail":93742112,"statfs":{"total":96636764160,"available":95991922688,"internally_reserved":0,"allocated":396378112,"data_stored":396112059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":248442538},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":474071040,"data_stored":474067427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":228896768,"data_stored":228820895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396718080,"data_stored":396714467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:30.821 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 173 20 134734 134714 3.04134 1.92969 0.0842423 0.0256807 2024-09-10T15:55:31.821 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 174 20 135258 135238 3.03562 2.04688 0.0632161 0.0257286 2024-09-10T15:55:32.821 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 175 20 135778 135758 3.02988 2.03125 0.0224201 0.0257757 2024-09-10T15:55:33.350 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:33.822 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 176 20 136267 136247 3.02351 1.91016 0.0342261 0.0258322 2024-09-10T15:55:33.887 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:33.888 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:33.913 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-10T15:55:32.477880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":547684854,"num_objects":133715,"num_object_clones":0,"num_object_copies":267430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3671,"num_objects_unfound":0,"num_objects_dirty":133715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95317,"num_write_kb":381262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":17166355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133797,"ondisk_log_size":133797,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1822012,"kb_used_data":1087116,"kb_used_omap":12,"kb_used_meta":734835,"kb_avail":281293508,"statfs":{"total":289910292480,"available":288044552192,"internally_reserved":0,"allocated":1113206784,"data_stored":1112336365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":752471045},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,8,11,10,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19107840,"num_objects":4665,"num_object_clones":0,"num_object_copies":9330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3870,"num_objects_misplaced":3671,"num_objects_unfound":0,"num_objects_dirty":4665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4665,"num_write_kb":18660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.811887"},"pg_stats":[{"pgid":"2.f","version":"46'8381","reported_seq":8457,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.172908+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:28.172908+0000","last_peered":"2024-09-10T15:55:28.172908+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:28.172908+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8381,"log_dups_size":0,"ondisk_log_size":8381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34328576,"num_objects":8381,"num_object_clones":0,"num_object_copies":16762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8381,"num_write_kb":33524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4408},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8558","reported_seq":8665,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:29.671326+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:29.671326+0000","last_peered":"2024-09-10T15:55:29.671326+0000","last_clean":"2024-09-10T15:55:29.671326+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:29.671326+0000","last_undegraded":"2024-09-10T15:55:29.671326+0000","last_fullsized":"2024-09-10T15:55:29.671326+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8558,"log_dups_size":0,"ondisk_log_size":8558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35028992,"num_objects":8552,"num_object_clones":0,"num_object_copies":17104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6894,"num_write_kb":27576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8316","reported_seq":8439,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:29.652063+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:29.652063+0000","last_peered":"2024-09-10T15:55:29.652063+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:29.652063+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8316,"log_dups_size":0,"ondisk_log_size":8316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6584,"num_write_kb":26336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8363","reported_seq":8442,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:29.417051+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:29.417051+0000","last_peered":"2024-09-10T15:55:29.417051+0000","last_clean":"2024-09-10T15:55:29.417051+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:29.417051+0000","last_undegraded":"2024-09-10T15:55:29.417051+0000","last_fullsized":"2024-09-10T15:55:29.417051+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8363,"log_dups_size":0,"ondisk_log_size":8363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34254848,"num_objects":8363,"num_object_clones":0,"num_object_copies":16726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8363,"num_write_kb":33452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8169","reported_seq":8295,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:28.154555+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:28.154555+0000","last_peered":"2024-09-10T15:55:28.154555+0000","last_clean":"2024-09-10T15:55:28.154555+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:28.154555+0000","last_undegraded":"2024-09-10T15:55:28.154555+0000","last_fullsized":"2024-09-10T15:55:28.154555+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8169,"log_dups_size":0,"ondisk_log_size":8169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33431552,"num_objects":8162,"num_object_clones":0,"num_object_copies":16324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5240,"num_write_kb":20960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8439","reported_seq":8968,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:28.183751+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:28.183751+0000","last_peered":"2024-09-10T15:55:28.183751+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:28.183751+0000","last_undegraded":"2024-09-10T15:55:28.183751+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8439,"log_dups_size":0,"ondisk_log_size":8439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34545664,"num_objects":8434,"num_object_clones":0,"num_object_copies":16868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3671,"num_objects_unfound":0,"num_objects_dirty":8434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5157,"num_write_kb":20628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":806912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4763},{"shards":"0,2","objects":3671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8174","reported_seq":8314,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.151977+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:28.151977+0000","last_peered":"2024-09-10T15:55:28.151977+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:28.151977+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8174,"log_dups_size":0,"ondisk_log_size":8174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33456128,"num_objects":8168,"num_object_clones":0,"num_object_copies":16336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":19296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4334},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8425","reported_seq":8556,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:28.103971+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:28.103971+0000","last_peered":"2024-09-10T15:55:28.103971+0000","last_clean":"2024-09-10T15:55:28.103971+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:28.103971+0000","last_undegraded":"2024-09-10T15:55:28.103971+0000","last_fullsized":"2024-09-10T15:55:28.103971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8425,"log_dups_size":0,"ondisk_log_size":8425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34500608,"num_objects":8423,"num_object_clones":0,"num_object_copies":16846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5184,"num_write_kb":20736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8403","reported_seq":16530,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:28.659179+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:28.659179+0000","last_peered":"2024-09-10T15:55:28.659179+0000","last_clean":"2024-09-10T15:55:28.659179+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:28.659179+0000","last_undegraded":"2024-09-10T15:55:28.659179+0000","last_fullsized":"2024-09-10T15:55:28.659179+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8403,"log_dups_size":0,"ondisk_log_size":8403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34390016,"num_objects":8396,"num_object_clones":0,"num_object_copies":16792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6380,"num_write_kb":25520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8282","reported_seq":8408,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.113289+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:28.113289+0000","last_peered":"2024-09-10T15:55:28.113289+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:28.113289+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8282,"log_dups_size":0,"ondisk_log_size":8282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33890304,"num_objects":8274,"num_object_clones":0,"num_object_copies":16548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5146,"num_write_kb":20584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4411},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8387","reported_seq":8510,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:29.530843+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:29.530843+0000","last_peered":"2024-09-10T15:55:29.530843+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:29.530843+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8387,"log_dups_size":0,"ondisk_log_size":8387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34332672,"num_objects":8382,"num_object_clones":0,"num_object_copies":16764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6361,"num_write_kb":25444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4550}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8272","reported_seq":8396,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:29.680137+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:29.680137+0000","last_peered":"2024-09-10T15:55:29.680137+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:29.680137+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8272,"log_dups_size":0,"ondisk_log_size":8272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33857536,"num_objects":8266,"num_object_clones":0,"num_object_copies":16532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6141,"num_write_kb":24564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8404","reported_seq":8531,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.124870+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:28.124870+0000","last_peered":"2024-09-10T15:55:28.124870+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:28.124870+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8404,"log_dups_size":0,"ondisk_log_size":8404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34414592,"num_objects":8402,"num_object_clones":0,"num_object_copies":16804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5596,"num_write_kb":22384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4569},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8367","reported_seq":8530,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.067794+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:28.067794+0000","last_peered":"2024-09-10T15:55:28.067794+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:28.067794+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8367,"log_dups_size":0,"ondisk_log_size":8367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34238464,"num_objects":8359,"num_object_clones":0,"num_object_copies":16718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4785,"num_write_kb":19140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8311","reported_seq":8449,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.168476+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:28.168476+0000","last_peered":"2024-09-10T15:55:28.168476+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:28.168476+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8311,"log_dups_size":0,"ondisk_log_size":8311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34025472,"num_objects":8307,"num_object_clones":0,"num_object_copies":16614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4917,"num_write_kb":19668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4430},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8543","reported_seq":8680,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:28.164699+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:28.164699+0000","last_peered":"2024-09-10T15:55:28.164699+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:28.164699+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8543,"log_dups_size":0,"ondisk_log_size":8543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34943459,"num_objects":8532,"num_object_clones":0,"num_object_copies":17064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5362,"num_write_kb":21448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4529},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":547684835,"num_objects":133713,"num_object_clones":0,"num_object_copies":267426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3671,"num_objects_unfound":0,"num_objects_dirty":133713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95315,"num_write_kb":381260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4193,"num_bytes_recovered":17166336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1113038848,"data_stored":1112955749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133794,"ondisk_log_size":133794,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626461,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":429916,"kb_used_data":231692,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93941924,"statfs":{"total":96636764160,"available":96196530176,"internally_reserved":0,"allocated":237252608,"data_stored":236914295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":202960558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,8,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149744,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751608,"kb_used_data":463144,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93620232,"statfs":{"total":96636764160,"available":95867117568,"internally_reserved":0,"allocated":474259456,"data_stored":473993403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295366317},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,1,1,5,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640488,"kb_used_data":392280,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93731352,"statfs":{"total":96636764160,"available":95980904448,"internally_reserved":0,"allocated":401694720,"data_stored":401428667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254144170},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":474071040,"data_stored":474067427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":236904448,"data_stored":236828575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402063360,"data_stored":402059747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:33.914 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:34.507 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:34.509 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:34.529 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-10T15:55:34.478367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":554590710,"num_objects":135401,"num_object_clones":0,"num_object_copies":270802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3596,"num_objects_unfound":0,"num_objects_dirty":135401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97003,"num_write_kb":388006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4270,"num_bytes_recovered":17473555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135483,"ondisk_log_size":135483,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1863288,"kb_used_data":1102024,"kb_used_omap":12,"kb_used_meta":761203,"kb_avail":281252232,"statfs":{"total":289910292480,"available":288002285568,"internally_reserved":0,"allocated":1128472576,"data_stored":1127602157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":779471877},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,10,8,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26013696,"num_objects":6351,"num_object_clones":0,"num_object_copies":12702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3870,"num_objects_misplaced":3596,"num_objects_unfound":0,"num_objects_dirty":6351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6351,"num_write_kb":25404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002938"},"pg_stats":[{"pgid":"2.f","version":"46'8542","reported_seq":8618,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.207584+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:33.207584+0000","last_peered":"2024-09-10T15:55:33.207584+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:33.207584+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8542,"log_dups_size":0,"ondisk_log_size":8542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34988032,"num_objects":8542,"num_object_clones":0,"num_object_copies":17084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8542,"num_write_kb":34168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4569},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8558","reported_seq":8665,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:29.671326+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:29.671326+0000","last_peered":"2024-09-10T15:55:29.671326+0000","last_clean":"2024-09-10T15:55:29.671326+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:29.671326+0000","last_undegraded":"2024-09-10T15:55:29.671326+0000","last_fullsized":"2024-09-10T15:55:29.671326+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8558,"log_dups_size":0,"ondisk_log_size":8558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35028992,"num_objects":8552,"num_object_clones":0,"num_object_copies":17104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6894,"num_write_kb":27576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8316","reported_seq":8439,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:29.652063+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:29.652063+0000","last_peered":"2024-09-10T15:55:29.652063+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:29.652063+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8316,"log_dups_size":0,"ondisk_log_size":8316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6584,"num_write_kb":26336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8363","reported_seq":8442,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:29.417051+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:29.417051+0000","last_peered":"2024-09-10T15:55:29.417051+0000","last_clean":"2024-09-10T15:55:29.417051+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:29.417051+0000","last_undegraded":"2024-09-10T15:55:29.417051+0000","last_fullsized":"2024-09-10T15:55:29.417051+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8363,"log_dups_size":0,"ondisk_log_size":8363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34254848,"num_objects":8363,"num_object_clones":0,"num_object_copies":16726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8363,"num_write_kb":33452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8339","reported_seq":8465,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:33.199885+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:33.199885+0000","last_peered":"2024-09-10T15:55:33.199885+0000","last_clean":"2024-09-10T15:55:33.199885+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:33.199885+0000","last_undegraded":"2024-09-10T15:55:33.199885+0000","last_fullsized":"2024-09-10T15:55:33.199885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8339,"log_dups_size":0,"ondisk_log_size":8339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34127872,"num_objects":8332,"num_object_clones":0,"num_object_copies":16664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5410,"num_write_kb":21640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8585","reported_seq":9263,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:33.235974+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:33.235974+0000","last_peered":"2024-09-10T15:55:33.235974+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:33.235974+0000","last_undegraded":"2024-09-10T15:55:33.235974+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8585,"log_dups_size":0,"ondisk_log_size":8585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35143680,"num_objects":8580,"num_object_clones":0,"num_object_copies":17160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3596,"num_objects_unfound":0,"num_objects_dirty":8580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4984},{"shards":"0,2","objects":3596}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8315","reported_seq":8451,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.159245+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:33.159245+0000","last_peered":"2024-09-10T15:55:33.159245+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:33.159245+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8315,"log_dups_size":0,"ondisk_log_size":8315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34033664,"num_objects":8309,"num_object_clones":0,"num_object_copies":16618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4965,"num_write_kb":19860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4475},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8573","reported_seq":8704,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:33.189177+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:33.189177+0000","last_peered":"2024-09-10T15:55:33.189177+0000","last_clean":"2024-09-10T15:55:33.189177+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:33.189177+0000","last_undegraded":"2024-09-10T15:55:33.189177+0000","last_fullsized":"2024-09-10T15:55:33.189177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8573,"log_dups_size":0,"ondisk_log_size":8573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35106816,"num_objects":8571,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5332,"num_write_kb":21328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8553","reported_seq":16678,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:33.746114+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:33.746114+0000","last_peered":"2024-09-10T15:55:33.746114+0000","last_clean":"2024-09-10T15:55:33.746114+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:33.746114+0000","last_undegraded":"2024-09-10T15:55:33.746114+0000","last_fullsized":"2024-09-10T15:55:33.746114+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8553,"log_dups_size":0,"ondisk_log_size":8553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35004416,"num_objects":8546,"num_object_clones":0,"num_object_copies":17092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6530,"num_write_kb":26120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8436","reported_seq":8562,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.177334+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:33.177334+0000","last_peered":"2024-09-10T15:55:33.177334+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:33.177334+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8436,"log_dups_size":0,"ondisk_log_size":8436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34521088,"num_objects":8428,"num_object_clones":0,"num_object_copies":16856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5300,"num_write_kb":21200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4565},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8387","reported_seq":8510,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:29.530843+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:29.530843+0000","last_peered":"2024-09-10T15:55:29.530843+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:29.530843+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8387,"log_dups_size":0,"ondisk_log_size":8387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34332672,"num_objects":8382,"num_object_clones":0,"num_object_copies":16764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6361,"num_write_kb":25444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4550}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8272","reported_seq":8396,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:29.680137+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:29.680137+0000","last_peered":"2024-09-10T15:55:29.680137+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:29.680137+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8272,"log_dups_size":0,"ondisk_log_size":8272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33857536,"num_objects":8266,"num_object_clones":0,"num_object_copies":16532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6141,"num_write_kb":24564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8545","reported_seq":8672,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.221390+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:33.221390+0000","last_peered":"2024-09-10T15:55:33.221390+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:33.221390+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8545,"log_dups_size":0,"ondisk_log_size":8545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34992128,"num_objects":8543,"num_object_clones":0,"num_object_copies":17086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5737,"num_write_kb":22948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4710},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8503","reported_seq":8666,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.147745+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:33.147745+0000","last_peered":"2024-09-10T15:55:33.147745+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:33.147745+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8503,"log_dups_size":0,"ondisk_log_size":8503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34795520,"num_objects":8495,"num_object_clones":0,"num_object_copies":16990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4921,"num_write_kb":19684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4578},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8477","reported_seq":8616,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.219594+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:33.219594+0000","last_peered":"2024-09-10T15:55:33.219594+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:33.219594+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8477,"log_dups_size":0,"ondisk_log_size":8477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34705408,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5083,"num_write_kb":20332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4596},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8716","reported_seq":8854,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.232460+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:33.232460+0000","last_peered":"2024-09-10T15:55:33.232460+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:33.232460+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8716,"log_dups_size":0,"ondisk_log_size":8716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35652067,"num_objects":8705,"num_object_clones":0,"num_object_copies":17410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5535,"num_write_kb":22140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4702},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":554590691,"num_objects":135399,"num_object_clones":0,"num_object_copies":270798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3596,"num_objects_unfound":0,"num_objects_dirty":135399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97001,"num_write_kb":388004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":17473536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1128607744,"data_stored":1128524645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135480,"ondisk_log_size":135480,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626462,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441832,"kb_used_data":239448,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93930008,"statfs":{"total":96636764160,"available":96184328192,"internally_reserved":0,"allocated":245194752,"data_stored":244856439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207220398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149745,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780968,"kb_used_data":470296,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93590872,"statfs":{"total":96636764160,"available":95837052928,"internally_reserved":0,"allocated":481583104,"data_stored":481317051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318107309},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640488,"kb_used_data":392280,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93731352,"statfs":{"total":96636764160,"available":95980904448,"internally_reserved":0,"allocated":401694720,"data_stored":401428667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254144170},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481611776,"data_stored":481608163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244932608,"data_stored":244856735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402063360,"data_stored":402059747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:34.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:55:34.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'8542", 'reported_seq': 8618, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:33.207584+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:55:33.207584+0000', 'last_peered': '2024-09-10T15:55:33.207584+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:55:33.207584+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 8542, 'log_dups_size': 0, 'ondisk_log_size': 8542, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34988032, 'num_objects': 8542, 'num_object_clones': 0, 'num_object_copies': 17084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8542, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8542, 'num_write_kb': 34168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4569}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:34.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:55:34.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'8316", 'reported_seq': 8439, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:29.652063+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:55:29.652063+0000', 'last_peered': '2024-09-10T15:55:29.652063+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:55:29.652063+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 8316, 'log_dups_size': 0, 'ondisk_log_size': 8316, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34045952, 'num_objects': 8312, 'num_object_clones': 0, 'num_object_copies': 16624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8312, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6584, 'num_write_kb': 26336, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 4510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:34.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:55:34.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'8585", 'reported_seq': 9263, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:55:33.235974+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:55:33.235974+0000', 'last_peered': '2024-09-10T15:55:33.235974+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:55:33.235974+0000', 'last_undegraded': '2024-09-10T15:55:33.235974+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 8585, 'log_dups_size': 0, 'ondisk_log_size': 8585, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35143680, 'num_objects': 8580, 'num_object_clones': 0, 'num_object_copies': 17160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3596, 'num_objects_unfound': 0, 'num_objects_dirty': 8580, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5303, 'num_write_kb': 21212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 272, 'num_bytes_recovered': 1114112, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4984}, {'shards': '0,2', 'objects': 3596}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:34.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:55:34.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'8315", 'reported_seq': 8451, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:33.159245+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:55:33.159245+0000', 'last_peered': '2024-09-10T15:55:33.159245+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:55:33.159245+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 8315, 'log_dups_size': 0, 'ondisk_log_size': 8315, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34033664, 'num_objects': 8309, 'num_object_clones': 0, 'num_object_copies': 16618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8309, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4965, 'num_write_kb': 19860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4475}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:34.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:55:34.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'8436", 'reported_seq': 8562, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:33.177334+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:55:33.177334+0000', 'last_peered': '2024-09-10T15:55:33.177334+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:55:33.177334+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 8436, 'log_dups_size': 0, 'ondisk_log_size': 8436, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34521088, 'num_objects': 8428, 'num_object_clones': 0, 'num_object_copies': 16856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5300, 'num_write_kb': 21200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4565}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:34.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:55:34.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'8387", 'reported_seq': 8510, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:29.530843+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:55:29.530843+0000', 'last_peered': '2024-09-10T15:55:29.530843+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:55:29.530843+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 8387, 'log_dups_size': 0, 'ondisk_log_size': 8387, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34332672, 'num_objects': 8382, 'num_object_clones': 0, 'num_object_copies': 16764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8382, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6361, 'num_write_kb': 25444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 4550}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:34.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:55:34.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'8272", 'reported_seq': 8396, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:29.680137+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:55:29.680137+0000', 'last_peered': '2024-09-10T15:55:29.680137+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:55:29.680137+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 8272, 'log_dups_size': 0, 'ondisk_log_size': 8272, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33857536, 'num_objects': 8266, 'num_object_clones': 0, 'num_object_copies': 16532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8266, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6141, 'num_write_kb': 24564, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 4437}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:34.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:55:34.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'8545", 'reported_seq': 8672, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:33.221390+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:55:33.221390+0000', 'last_peered': '2024-09-10T15:55:33.221390+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:55:33.221390+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 8545, 'log_dups_size': 0, 'ondisk_log_size': 8545, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34992128, 'num_objects': 8543, 'num_object_clones': 0, 'num_object_copies': 17086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8543, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5737, 'num_write_kb': 22948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4710}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:34.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:55:34.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'8503", 'reported_seq': 8666, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:33.147745+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:55:33.147745+0000', 'last_peered': '2024-09-10T15:55:33.147745+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:55:33.147745+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 8503, 'log_dups_size': 0, 'ondisk_log_size': 8503, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34795520, 'num_objects': 8495, 'num_object_clones': 0, 'num_object_copies': 16990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8495, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4921, 'num_write_kb': 19684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4578}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:34.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:55:34.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'8477", 'reported_seq': 8616, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:33.219594+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:55:33.219594+0000', 'last_peered': '2024-09-10T15:55:33.219594+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:55:33.219594+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 8477, 'log_dups_size': 0, 'ondisk_log_size': 8477, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34705408, 'num_objects': 8473, 'num_object_clones': 0, 'num_object_copies': 16946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5083, 'num_write_kb': 20332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4596}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:34.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:55:34.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'8716", 'reported_seq': 8854, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:33.232460+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:55:33.232460+0000', 'last_peered': '2024-09-10T15:55:33.232460+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:55:33.232460+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 8716, 'log_dups_size': 0, 'ondisk_log_size': 8716, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35652067, 'num_objects': 8705, 'num_object_clones': 0, 'num_object_copies': 17410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8705, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5535, 'num_write_kb': 22140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4702}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:34.533 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:55:34.822 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 177 20 136722 136702 3.01647 1.77734 0.015096 0.0258911 2024-09-10T15:55:35.233 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:35.233 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/270802 objects degraded (14.321%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":261,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":135401,"data_bytes":554590710,"bytes_used":1908006912,"bytes_avail":288002285568,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":270802,"degraded_ratio":0.14320795267390934,"misplaced_objects":3596,"misplaced_total":270802,"misplaced_ratio":0.013279074748340116,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":92819,"recovering_keys_per_sec":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"write_bytes_sec":2167277,"read_op_per_sec":0,"write_op_per_sec":529},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:55:35.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:55:35.265 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:35.822 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 178 20 137181 137161 3.0096 1.79297 0.0258043 0.0259503 2024-09-10T15:55:35.826 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:35.826 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:35.853 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-10T15:55:34.478367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":554590710,"num_objects":135401,"num_object_clones":0,"num_object_copies":270802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3596,"num_objects_unfound":0,"num_objects_dirty":135401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97003,"num_write_kb":388006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4270,"num_bytes_recovered":17473555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135483,"ondisk_log_size":135483,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1863288,"kb_used_data":1102024,"kb_used_omap":12,"kb_used_meta":761203,"kb_avail":281252232,"statfs":{"total":289910292480,"available":288002285568,"internally_reserved":0,"allocated":1128472576,"data_stored":1127602157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":779471877},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,10,8,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26013696,"num_objects":6351,"num_object_clones":0,"num_object_copies":12702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3870,"num_objects_misplaced":3596,"num_objects_unfound":0,"num_objects_dirty":6351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6351,"num_write_kb":25404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002938"},"pg_stats":[{"pgid":"2.f","version":"46'8542","reported_seq":8618,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.207584+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:33.207584+0000","last_peered":"2024-09-10T15:55:33.207584+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:33.207584+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8542,"log_dups_size":0,"ondisk_log_size":8542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34988032,"num_objects":8542,"num_object_clones":0,"num_object_copies":17084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8542,"num_write_kb":34168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4569},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8558","reported_seq":8665,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:29.671326+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:29.671326+0000","last_peered":"2024-09-10T15:55:29.671326+0000","last_clean":"2024-09-10T15:55:29.671326+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:29.671326+0000","last_undegraded":"2024-09-10T15:55:29.671326+0000","last_fullsized":"2024-09-10T15:55:29.671326+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8558,"log_dups_size":0,"ondisk_log_size":8558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35028992,"num_objects":8552,"num_object_clones":0,"num_object_copies":17104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6894,"num_write_kb":27576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8316","reported_seq":8439,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:29.652063+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:29.652063+0000","last_peered":"2024-09-10T15:55:29.652063+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:29.652063+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8316,"log_dups_size":0,"ondisk_log_size":8316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6584,"num_write_kb":26336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8363","reported_seq":8442,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:29.417051+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:29.417051+0000","last_peered":"2024-09-10T15:55:29.417051+0000","last_clean":"2024-09-10T15:55:29.417051+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:29.417051+0000","last_undegraded":"2024-09-10T15:55:29.417051+0000","last_fullsized":"2024-09-10T15:55:29.417051+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8363,"log_dups_size":0,"ondisk_log_size":8363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34254848,"num_objects":8363,"num_object_clones":0,"num_object_copies":16726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8363,"num_write_kb":33452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8339","reported_seq":8465,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:33.199885+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:33.199885+0000","last_peered":"2024-09-10T15:55:33.199885+0000","last_clean":"2024-09-10T15:55:33.199885+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:33.199885+0000","last_undegraded":"2024-09-10T15:55:33.199885+0000","last_fullsized":"2024-09-10T15:55:33.199885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8339,"log_dups_size":0,"ondisk_log_size":8339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34127872,"num_objects":8332,"num_object_clones":0,"num_object_copies":16664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5410,"num_write_kb":21640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8585","reported_seq":9263,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:33.235974+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:33.235974+0000","last_peered":"2024-09-10T15:55:33.235974+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:33.235974+0000","last_undegraded":"2024-09-10T15:55:33.235974+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8585,"log_dups_size":0,"ondisk_log_size":8585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35143680,"num_objects":8580,"num_object_clones":0,"num_object_copies":17160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3596,"num_objects_unfound":0,"num_objects_dirty":8580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4984},{"shards":"0,2","objects":3596}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8315","reported_seq":8451,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.159245+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:33.159245+0000","last_peered":"2024-09-10T15:55:33.159245+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:33.159245+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8315,"log_dups_size":0,"ondisk_log_size":8315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34033664,"num_objects":8309,"num_object_clones":0,"num_object_copies":16618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4965,"num_write_kb":19860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4475},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8573","reported_seq":8704,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:33.189177+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:33.189177+0000","last_peered":"2024-09-10T15:55:33.189177+0000","last_clean":"2024-09-10T15:55:33.189177+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:33.189177+0000","last_undegraded":"2024-09-10T15:55:33.189177+0000","last_fullsized":"2024-09-10T15:55:33.189177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8573,"log_dups_size":0,"ondisk_log_size":8573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35106816,"num_objects":8571,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5332,"num_write_kb":21328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8553","reported_seq":16678,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:33.746114+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:33.746114+0000","last_peered":"2024-09-10T15:55:33.746114+0000","last_clean":"2024-09-10T15:55:33.746114+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:33.746114+0000","last_undegraded":"2024-09-10T15:55:33.746114+0000","last_fullsized":"2024-09-10T15:55:33.746114+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8553,"log_dups_size":0,"ondisk_log_size":8553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35004416,"num_objects":8546,"num_object_clones":0,"num_object_copies":17092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6530,"num_write_kb":26120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8436","reported_seq":8562,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.177334+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:33.177334+0000","last_peered":"2024-09-10T15:55:33.177334+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:33.177334+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8436,"log_dups_size":0,"ondisk_log_size":8436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34521088,"num_objects":8428,"num_object_clones":0,"num_object_copies":16856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5300,"num_write_kb":21200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4565},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8387","reported_seq":8510,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:29.530843+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:29.530843+0000","last_peered":"2024-09-10T15:55:29.530843+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:29.530843+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8387,"log_dups_size":0,"ondisk_log_size":8387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34332672,"num_objects":8382,"num_object_clones":0,"num_object_copies":16764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6361,"num_write_kb":25444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4550}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8272","reported_seq":8396,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:29.680137+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:29.680137+0000","last_peered":"2024-09-10T15:55:29.680137+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:29.680137+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8272,"log_dups_size":0,"ondisk_log_size":8272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33857536,"num_objects":8266,"num_object_clones":0,"num_object_copies":16532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6141,"num_write_kb":24564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8545","reported_seq":8672,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.221390+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:33.221390+0000","last_peered":"2024-09-10T15:55:33.221390+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:33.221390+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8545,"log_dups_size":0,"ondisk_log_size":8545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34992128,"num_objects":8543,"num_object_clones":0,"num_object_copies":17086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5737,"num_write_kb":22948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4710},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8503","reported_seq":8666,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.147745+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:33.147745+0000","last_peered":"2024-09-10T15:55:33.147745+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:33.147745+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8503,"log_dups_size":0,"ondisk_log_size":8503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34795520,"num_objects":8495,"num_object_clones":0,"num_object_copies":16990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4921,"num_write_kb":19684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4578},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8477","reported_seq":8616,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.219594+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:33.219594+0000","last_peered":"2024-09-10T15:55:33.219594+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:33.219594+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8477,"log_dups_size":0,"ondisk_log_size":8477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34705408,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5083,"num_write_kb":20332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4596},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8716","reported_seq":8854,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:33.232460+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:33.232460+0000","last_peered":"2024-09-10T15:55:33.232460+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:33.232460+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8716,"log_dups_size":0,"ondisk_log_size":8716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35652067,"num_objects":8705,"num_object_clones":0,"num_object_copies":17410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5535,"num_write_kb":22140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4702},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":554590691,"num_objects":135399,"num_object_clones":0,"num_object_copies":270798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3596,"num_objects_unfound":0,"num_objects_dirty":135399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97001,"num_write_kb":388004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":17473536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1128607744,"data_stored":1128524645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135480,"ondisk_log_size":135480,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626462,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441832,"kb_used_data":239448,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93930008,"statfs":{"total":96636764160,"available":96184328192,"internally_reserved":0,"allocated":245194752,"data_stored":244856439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207220398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149745,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780968,"kb_used_data":470296,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93590872,"statfs":{"total":96636764160,"available":95837052928,"internally_reserved":0,"allocated":481583104,"data_stored":481317051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318107309},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640488,"kb_used_data":392280,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93731352,"statfs":{"total":96636764160,"available":95980904448,"internally_reserved":0,"allocated":401694720,"data_stored":401428667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254144170},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481611776,"data_stored":481608163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244932608,"data_stored":244856735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402063360,"data_stored":402059747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:36.822 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 179 20 137711 137691 3.00435 2.07031 0.0454173 0.0259975 2024-09-10T15:55:37.822 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:2024-09-10T15:55:37.821694+0000 min lat: 0.00198913 max lat: 2.66872 avg lat: 0.0260461 2024-09-10T15:55:37.822 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:55:37.822 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 180 20 138216 138196 2.99862 1.97266 0.0685855 0.0260461 2024-09-10T15:55:38.822 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 181 20 138727 138707 2.99308 1.99609 0.0351552 0.0260954 2024-09-10T15:55:38.855 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:39.394 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:39.395 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:39.415 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-10T15:55:38.479974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":564183542,"num_objects":137743,"num_object_clones":0,"num_object_copies":275486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3527,"num_objects_unfound":0,"num_objects_dirty":137743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99345,"num_write_kb":397374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4339,"num_bytes_recovered":17756179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137825,"ondisk_log_size":137825,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1867348,"kb_used_data":1114400,"kb_used_omap":12,"kb_used_meta":752883,"kb_avail":281248172,"statfs":{"total":289910292480,"available":287998128128,"internally_reserved":0,"allocated":1141145600,"data_stored":1140275181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":770952197},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,10,8,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-231,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6524,"num_write_kb":26096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":946176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003587"},"pg_stats":[{"pgid":"2.f","version":"46'8702","reported_seq":8778,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.297967+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:38.297967+0000","last_peered":"2024-09-10T15:55:38.297967+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:38.297967+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8702,"log_dups_size":0,"ondisk_log_size":8702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35643392,"num_objects":8702,"num_object_clones":0,"num_object_copies":17404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8702,"num_write_kb":34808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4729},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8718","reported_seq":8825,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:34.651546+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:34.651546+0000","last_peered":"2024-09-10T15:55:34.651546+0000","last_clean":"2024-09-10T15:55:34.651546+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:34.651546+0000","last_undegraded":"2024-09-10T15:55:34.651546+0000","last_fullsized":"2024-09-10T15:55:34.651546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8718,"log_dups_size":0,"ondisk_log_size":8718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35684352,"num_objects":8712,"num_object_clones":0,"num_object_copies":17424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7054,"num_write_kb":28216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8483","reported_seq":8606,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:34.566894+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:34.566894+0000","last_peered":"2024-09-10T15:55:34.566894+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:34.566894+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8483,"log_dups_size":0,"ondisk_log_size":8483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34729984,"num_objects":8479,"num_object_clones":0,"num_object_copies":16958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6751,"num_write_kb":27004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4677}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8513","reported_seq":8592,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:34.660716+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:34.660716+0000","last_peered":"2024-09-10T15:55:34.660716+0000","last_clean":"2024-09-10T15:55:34.660716+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:34.660716+0000","last_undegraded":"2024-09-10T15:55:34.660716+0000","last_fullsized":"2024-09-10T15:55:34.660716+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8513,"log_dups_size":0,"ondisk_log_size":8513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34869248,"num_objects":8513,"num_object_clones":0,"num_object_copies":17026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8513,"num_write_kb":34052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8502","reported_seq":8628,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:38.270056+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:38.270056+0000","last_peered":"2024-09-10T15:55:38.270056+0000","last_clean":"2024-09-10T15:55:38.270056+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:38.270056+0000","last_undegraded":"2024-09-10T15:55:38.270056+0000","last_fullsized":"2024-09-10T15:55:38.270056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8502,"log_dups_size":0,"ondisk_log_size":8502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34795520,"num_objects":8495,"num_object_clones":0,"num_object_copies":16990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5573,"num_write_kb":22292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8747","reported_seq":9564,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:38.205993+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:38.205993+0000","last_peered":"2024-09-10T15:55:38.205993+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:38.205993+0000","last_undegraded":"2024-09-10T15:55:38.205993+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8747,"log_dups_size":0,"ondisk_log_size":8747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35807232,"num_objects":8742,"num_object_clones":0,"num_object_copies":17484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3527,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5465,"num_write_kb":21860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":1396736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5215},{"shards":"0,2","objects":3527}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8458","reported_seq":8598,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.224703+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:38.224703+0000","last_peered":"2024-09-10T15:55:38.224703+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:38.224703+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8458,"log_dups_size":0,"ondisk_log_size":8458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34619392,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5108,"num_write_kb":20432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4618},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8725","reported_seq":8856,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:38.146166+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:38.146166+0000","last_peered":"2024-09-10T15:55:38.146166+0000","last_clean":"2024-09-10T15:55:38.146166+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:38.146166+0000","last_undegraded":"2024-09-10T15:55:38.146166+0000","last_fullsized":"2024-09-10T15:55:38.146166+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8725,"log_dups_size":0,"ondisk_log_size":8725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35729408,"num_objects":8723,"num_object_clones":0,"num_object_copies":17446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5484,"num_write_kb":21936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8553","reported_seq":16678,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:33.746114+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:33.746114+0000","last_peered":"2024-09-10T15:55:33.746114+0000","last_clean":"2024-09-10T15:55:33.746114+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:33.746114+0000","last_undegraded":"2024-09-10T15:55:33.746114+0000","last_fullsized":"2024-09-10T15:55:33.746114+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8553,"log_dups_size":0,"ondisk_log_size":8553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35004416,"num_objects":8546,"num_object_clones":0,"num_object_copies":17092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6530,"num_write_kb":26120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8599","reported_seq":8724,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.288611+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:38.288611+0000","last_peered":"2024-09-10T15:55:38.288611+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:38.288611+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8599,"log_dups_size":0,"ondisk_log_size":8599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35188736,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5463,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4728},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8578","reported_seq":8702,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:34.618296+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:34.618296+0000","last_peered":"2024-09-10T15:55:34.618296+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:34.618296+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8578,"log_dups_size":0,"ondisk_log_size":8578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6552,"num_write_kb":26208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4741}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8429","reported_seq":8555,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:34.648248+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:34.648248+0000","last_peered":"2024-09-10T15:55:34.648248+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:34.648248+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8429,"log_dups_size":0,"ondisk_log_size":8429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34500608,"num_objects":8423,"num_object_clones":0,"num_object_copies":16846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6298,"num_write_kb":25192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4594}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8685","reported_seq":8812,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.270066+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:38.270066+0000","last_peered":"2024-09-10T15:55:38.270066+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:38.270066+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8685,"log_dups_size":0,"ondisk_log_size":8685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5877,"num_write_kb":23508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4850},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8657","reported_seq":8820,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.224998+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:38.224998+0000","last_peered":"2024-09-10T15:55:38.224998+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:38.224998+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8657,"log_dups_size":0,"ondisk_log_size":8657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426304,"num_objects":8649,"num_object_clones":0,"num_object_copies":17298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5075,"num_write_kb":20300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8627","reported_seq":8766,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.250498+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:38.250498+0000","last_peered":"2024-09-10T15:55:38.250498+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:38.250498+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8627,"log_dups_size":0,"ondisk_log_size":8627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35319808,"num_objects":8623,"num_object_clones":0,"num_object_copies":17246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5233,"num_write_kb":20932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4746},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8846","reported_seq":8984,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.248208+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:38.248208+0000","last_peered":"2024-09-10T15:55:38.248208+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:38.248208+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8846,"log_dups_size":0,"ondisk_log_size":8846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36184547,"num_objects":8835,"num_object_clones":0,"num_object_copies":17670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5665,"num_write_kb":22660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4832},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":564183523,"num_objects":137741,"num_object_clones":0,"num_object_copies":275482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3527,"num_objects_unfound":0,"num_objects_dirty":137741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99343,"num_write_kb":397372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4337,"num_bytes_recovered":17756160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1141219328,"data_stored":1141136229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137822,"ondisk_log_size":137822,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626462,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441832,"kb_used_data":239448,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93930008,"statfs":{"total":96636764160,"available":96184328192,"internally_reserved":0,"allocated":245194752,"data_stored":244856439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207220398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149746,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":775664,"kb_used_data":477472,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93596176,"statfs":{"total":96636764160,"available":95842484224,"internally_reserved":0,"allocated":488931328,"data_stored":488665275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305327789},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649852,"kb_used_data":397480,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93721988,"statfs":{"total":96636764160,"available":95971315712,"internally_reserved":0,"allocated":407019520,"data_stored":406753467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":258404010},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,2,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":489021440,"data_stored":489017827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244932608,"data_stored":244856735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":407265280,"data_stored":407261667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:39.417 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:39.823 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 182 20 139176 139156 2.98626 1.75391 0.0350446 0.0261513 2024-09-10T15:55:39.962 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:39.962 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:39.987 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-10T15:55:38.479974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":564183542,"num_objects":137743,"num_object_clones":0,"num_object_copies":275486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3527,"num_objects_unfound":0,"num_objects_dirty":137743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99345,"num_write_kb":397374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4339,"num_bytes_recovered":17756179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137825,"ondisk_log_size":137825,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1867348,"kb_used_data":1114400,"kb_used_omap":12,"kb_used_meta":752883,"kb_avail":281248172,"statfs":{"total":289910292480,"available":287998128128,"internally_reserved":0,"allocated":1141145600,"data_stored":1140275181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":770952197},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,10,8,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-231,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6524,"num_write_kb":26096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":946176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003587"},"pg_stats":[{"pgid":"2.f","version":"46'8702","reported_seq":8778,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.297967+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:38.297967+0000","last_peered":"2024-09-10T15:55:38.297967+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:38.297967+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8702,"log_dups_size":0,"ondisk_log_size":8702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35643392,"num_objects":8702,"num_object_clones":0,"num_object_copies":17404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8702,"num_write_kb":34808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4729},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8718","reported_seq":8825,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:34.651546+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:34.651546+0000","last_peered":"2024-09-10T15:55:34.651546+0000","last_clean":"2024-09-10T15:55:34.651546+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:34.651546+0000","last_undegraded":"2024-09-10T15:55:34.651546+0000","last_fullsized":"2024-09-10T15:55:34.651546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8718,"log_dups_size":0,"ondisk_log_size":8718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35684352,"num_objects":8712,"num_object_clones":0,"num_object_copies":17424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7054,"num_write_kb":28216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8483","reported_seq":8606,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:34.566894+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:34.566894+0000","last_peered":"2024-09-10T15:55:34.566894+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:34.566894+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8483,"log_dups_size":0,"ondisk_log_size":8483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34729984,"num_objects":8479,"num_object_clones":0,"num_object_copies":16958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6751,"num_write_kb":27004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4677}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8513","reported_seq":8592,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:34.660716+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:34.660716+0000","last_peered":"2024-09-10T15:55:34.660716+0000","last_clean":"2024-09-10T15:55:34.660716+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:34.660716+0000","last_undegraded":"2024-09-10T15:55:34.660716+0000","last_fullsized":"2024-09-10T15:55:34.660716+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8513,"log_dups_size":0,"ondisk_log_size":8513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34869248,"num_objects":8513,"num_object_clones":0,"num_object_copies":17026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8513,"num_write_kb":34052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8502","reported_seq":8628,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:38.270056+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:38.270056+0000","last_peered":"2024-09-10T15:55:38.270056+0000","last_clean":"2024-09-10T15:55:38.270056+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:38.270056+0000","last_undegraded":"2024-09-10T15:55:38.270056+0000","last_fullsized":"2024-09-10T15:55:38.270056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8502,"log_dups_size":0,"ondisk_log_size":8502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34795520,"num_objects":8495,"num_object_clones":0,"num_object_copies":16990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5573,"num_write_kb":22292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8747","reported_seq":9564,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:38.205993+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:38.205993+0000","last_peered":"2024-09-10T15:55:38.205993+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:38.205993+0000","last_undegraded":"2024-09-10T15:55:38.205993+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8747,"log_dups_size":0,"ondisk_log_size":8747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35807232,"num_objects":8742,"num_object_clones":0,"num_object_copies":17484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3527,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5465,"num_write_kb":21860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":1396736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5215},{"shards":"0,2","objects":3527}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8458","reported_seq":8598,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.224703+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:38.224703+0000","last_peered":"2024-09-10T15:55:38.224703+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:38.224703+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8458,"log_dups_size":0,"ondisk_log_size":8458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34619392,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5108,"num_write_kb":20432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4618},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8725","reported_seq":8856,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:38.146166+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:38.146166+0000","last_peered":"2024-09-10T15:55:38.146166+0000","last_clean":"2024-09-10T15:55:38.146166+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:38.146166+0000","last_undegraded":"2024-09-10T15:55:38.146166+0000","last_fullsized":"2024-09-10T15:55:38.146166+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8725,"log_dups_size":0,"ondisk_log_size":8725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35729408,"num_objects":8723,"num_object_clones":0,"num_object_copies":17446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5484,"num_write_kb":21936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8553","reported_seq":16678,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:33.746114+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:33.746114+0000","last_peered":"2024-09-10T15:55:33.746114+0000","last_clean":"2024-09-10T15:55:33.746114+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:33.746114+0000","last_undegraded":"2024-09-10T15:55:33.746114+0000","last_fullsized":"2024-09-10T15:55:33.746114+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8553,"log_dups_size":0,"ondisk_log_size":8553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35004416,"num_objects":8546,"num_object_clones":0,"num_object_copies":17092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6530,"num_write_kb":26120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8599","reported_seq":8724,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.288611+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:38.288611+0000","last_peered":"2024-09-10T15:55:38.288611+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:38.288611+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8599,"log_dups_size":0,"ondisk_log_size":8599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35188736,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5463,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4728},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8578","reported_seq":8702,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:34.618296+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:34.618296+0000","last_peered":"2024-09-10T15:55:34.618296+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:34.618296+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8578,"log_dups_size":0,"ondisk_log_size":8578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6552,"num_write_kb":26208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4741}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8429","reported_seq":8555,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:34.648248+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:34.648248+0000","last_peered":"2024-09-10T15:55:34.648248+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:34.648248+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8429,"log_dups_size":0,"ondisk_log_size":8429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34500608,"num_objects":8423,"num_object_clones":0,"num_object_copies":16846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6298,"num_write_kb":25192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4594}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8685","reported_seq":8812,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.270066+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:38.270066+0000","last_peered":"2024-09-10T15:55:38.270066+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:38.270066+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8685,"log_dups_size":0,"ondisk_log_size":8685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5877,"num_write_kb":23508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4850},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8657","reported_seq":8820,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.224998+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:38.224998+0000","last_peered":"2024-09-10T15:55:38.224998+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:38.224998+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8657,"log_dups_size":0,"ondisk_log_size":8657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426304,"num_objects":8649,"num_object_clones":0,"num_object_copies":17298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5075,"num_write_kb":20300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8627","reported_seq":8766,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.250498+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:38.250498+0000","last_peered":"2024-09-10T15:55:38.250498+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:38.250498+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8627,"log_dups_size":0,"ondisk_log_size":8627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35319808,"num_objects":8623,"num_object_clones":0,"num_object_copies":17246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5233,"num_write_kb":20932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4746},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8846","reported_seq":8984,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.248208+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:38.248208+0000","last_peered":"2024-09-10T15:55:38.248208+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:38.248208+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8846,"log_dups_size":0,"ondisk_log_size":8846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36184547,"num_objects":8835,"num_object_clones":0,"num_object_copies":17670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5665,"num_write_kb":22660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4832},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":564183523,"num_objects":137741,"num_object_clones":0,"num_object_copies":275482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3527,"num_objects_unfound":0,"num_objects_dirty":137741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99343,"num_write_kb":397372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4337,"num_bytes_recovered":17756160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1141219328,"data_stored":1141136229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137822,"ondisk_log_size":137822,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626462,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441832,"kb_used_data":239448,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93930008,"statfs":{"total":96636764160,"available":96184328192,"internally_reserved":0,"allocated":245194752,"data_stored":244856439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207220398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149746,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":775664,"kb_used_data":477472,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93596176,"statfs":{"total":96636764160,"available":95842484224,"internally_reserved":0,"allocated":488931328,"data_stored":488665275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305327789},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649852,"kb_used_data":397480,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93721988,"statfs":{"total":96636764160,"available":95971315712,"internally_reserved":0,"allocated":407019520,"data_stored":406753467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":258404010},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,2,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":489021440,"data_stored":489017827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244932608,"data_stored":244856735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":407265280,"data_stored":407261667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:39.988 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:55:39.988 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'8702", 'reported_seq': 8778, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:38.297967+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:55:38.297967+0000', 'last_peered': '2024-09-10T15:55:38.297967+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:55:38.297967+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 8702, 'log_dups_size': 0, 'ondisk_log_size': 8702, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35643392, 'num_objects': 8702, 'num_object_clones': 0, 'num_object_copies': 17404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8702, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8702, 'num_write_kb': 34808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4729}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:39.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:55:39.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'8483", 'reported_seq': 8606, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:34.566894+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:55:34.566894+0000', 'last_peered': '2024-09-10T15:55:34.566894+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:55:34.566894+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 8483, 'log_dups_size': 0, 'ondisk_log_size': 8483, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34729984, 'num_objects': 8479, 'num_object_clones': 0, 'num_object_copies': 16958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8479, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6751, 'num_write_kb': 27004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 4677}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:39.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:55:39.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'8747", 'reported_seq': 9564, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:55:38.205993+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:55:38.205993+0000', 'last_peered': '2024-09-10T15:55:38.205993+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:55:38.205993+0000', 'last_undegraded': '2024-09-10T15:55:38.205993+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 8747, 'log_dups_size': 0, 'ondisk_log_size': 8747, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35807232, 'num_objects': 8742, 'num_object_clones': 0, 'num_object_copies': 17484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3527, 'num_objects_unfound': 0, 'num_objects_dirty': 8742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5465, 'num_write_kb': 21860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 341, 'num_bytes_recovered': 1396736, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5215}, {'shards': '0,2', 'objects': 3527}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:39.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:55:39.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'8458", 'reported_seq': 8598, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:38.224703+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:55:38.224703+0000', 'last_peered': '2024-09-10T15:55:38.224703+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:55:38.224703+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 8458, 'log_dups_size': 0, 'ondisk_log_size': 8458, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34619392, 'num_objects': 8452, 'num_object_clones': 0, 'num_object_copies': 16904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8452, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5108, 'num_write_kb': 20432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4618}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:39.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:55:39.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'8599", 'reported_seq': 8724, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:38.288611+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:55:38.288611+0000', 'last_peered': '2024-09-10T15:55:38.288611+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:55:38.288611+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 8599, 'log_dups_size': 0, 'ondisk_log_size': 8599, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35188736, 'num_objects': 8591, 'num_object_clones': 0, 'num_object_copies': 17182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5463, 'num_write_kb': 21852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4728}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:39.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:55:39.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'8578", 'reported_seq': 8702, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:34.618296+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:55:34.618296+0000', 'last_peered': '2024-09-10T15:55:34.618296+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:55:34.618296+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 8578, 'log_dups_size': 0, 'ondisk_log_size': 8578, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35115008, 'num_objects': 8573, 'num_object_clones': 0, 'num_object_copies': 17146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8573, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6552, 'num_write_kb': 26208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 4741}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:39.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:55:39.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'8429", 'reported_seq': 8555, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:34.648248+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:55:34.648248+0000', 'last_peered': '2024-09-10T15:55:34.648248+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:55:34.648248+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 8429, 'log_dups_size': 0, 'ondisk_log_size': 8429, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34500608, 'num_objects': 8423, 'num_object_clones': 0, 'num_object_copies': 16846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8423, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6298, 'num_write_kb': 25192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 4594}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:39.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:55:39.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'8685", 'reported_seq': 8812, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:38.270066+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:55:38.270066+0000', 'last_peered': '2024-09-10T15:55:38.270066+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:55:38.270066+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 8685, 'log_dups_size': 0, 'ondisk_log_size': 8685, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35565568, 'num_objects': 8683, 'num_object_clones': 0, 'num_object_copies': 17366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8683, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5877, 'num_write_kb': 23508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4850}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:39.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:55:39.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'8657", 'reported_seq': 8820, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:38.224998+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:55:38.224998+0000', 'last_peered': '2024-09-10T15:55:38.224998+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:55:38.224998+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 8657, 'log_dups_size': 0, 'ondisk_log_size': 8657, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35426304, 'num_objects': 8649, 'num_object_clones': 0, 'num_object_copies': 17298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5075, 'num_write_kb': 20300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4732}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:39.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:55:39.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'8627", 'reported_seq': 8766, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:38.250498+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:55:38.250498+0000', 'last_peered': '2024-09-10T15:55:38.250498+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:55:38.250498+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 8627, 'log_dups_size': 0, 'ondisk_log_size': 8627, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35319808, 'num_objects': 8623, 'num_object_clones': 0, 'num_object_copies': 17246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8623, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5233, 'num_write_kb': 20932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4746}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:39.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:55:39.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'8846", 'reported_seq': 8984, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:38.248208+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:55:38.248208+0000', 'last_peered': '2024-09-10T15:55:38.248208+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:55:38.248208+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 8846, 'log_dups_size': 0, 'ondisk_log_size': 8846, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36184547, 'num_objects': 8835, 'num_object_clones': 0, 'num_object_copies': 17670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8835, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5665, 'num_write_kb': 22660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4832}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:39.991 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:55:40.681 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:40.681 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/275486 objects degraded (14.077%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":266,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":137743,"data_bytes":564183542,"bytes_used":1912164352,"bytes_avail":287998128128,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":275486,"degraded_ratio":0.14077303383838016,"misplaced_objects":3527,"misplaced_total":275486,"misplaced_ratio":0.012802828455892496,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":78824,"recovering_keys_per_sec":0,"num_objects_recovered":231,"num_bytes_recovered":946176,"num_keys_recovered":0,"write_bytes_sec":2226193,"read_op_per_sec":0,"write_op_per_sec":543},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:55:40.715 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:55:40.716 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:40.823 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 183 20 139626 139606 2.97955 1.75781 0.0638449 0.0262138 2024-09-10T15:55:41.306 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:41.306 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:41.327 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-10T15:55:40.480473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":567919094,"num_objects":138655,"num_object_clones":0,"num_object_copies":277310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3527,"num_objects_unfound":0,"num_objects_dirty":138655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100257,"num_write_kb":401022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4339,"num_bytes_recovered":17756179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138737,"ondisk_log_size":138737,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871652,"kb_used_data":1127028,"kb_used_omap":12,"kb_used_meta":744563,"kb_avail":281243868,"statfs":{"total":289910292480,"available":287993720832,"internally_reserved":0,"allocated":1154076672,"data_stored":1153206253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":762432517},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,7,10,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24186880,"num_objects":5905,"num_object_clones":0,"num_object_copies":11810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-144,"num_objects_unfound":0,"num_objects_dirty":5905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5905,"num_write_kb":23620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003610"},"pg_stats":[{"pgid":"2.f","version":"46'8702","reported_seq":8778,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.297967+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:38.297967+0000","last_peered":"2024-09-10T15:55:38.297967+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:38.297967+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8702,"log_dups_size":0,"ondisk_log_size":8702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35643392,"num_objects":8702,"num_object_clones":0,"num_object_copies":17404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8702,"num_write_kb":34808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4729},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8867","reported_seq":8973,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:39.686072+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:39.686072+0000","last_peered":"2024-09-10T15:55:39.686072+0000","last_clean":"2024-09-10T15:55:39.686072+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:39.686072+0000","last_undegraded":"2024-09-10T15:55:39.686072+0000","last_fullsized":"2024-09-10T15:55:39.686072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8867,"log_dups_size":0,"ondisk_log_size":8867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36294656,"num_objects":8861,"num_object_clones":0,"num_object_copies":17722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7203,"num_write_kb":28812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8642","reported_seq":8765,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:39.571478+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:39.571478+0000","last_peered":"2024-09-10T15:55:39.571478+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:39.571478+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8642,"log_dups_size":0,"ondisk_log_size":8642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35381248,"num_objects":8638,"num_object_clones":0,"num_object_copies":17276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6910,"num_write_kb":27640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4836}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8655","reported_seq":8733,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:39.653317+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:39.653317+0000","last_peered":"2024-09-10T15:55:39.653317+0000","last_clean":"2024-09-10T15:55:39.653317+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:39.653317+0000","last_undegraded":"2024-09-10T15:55:39.653317+0000","last_fullsized":"2024-09-10T15:55:39.653317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8655,"log_dups_size":0,"ondisk_log_size":8655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450880,"num_objects":8655,"num_object_clones":0,"num_object_copies":17310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8655,"num_write_kb":34620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8502","reported_seq":8628,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:38.270056+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:38.270056+0000","last_peered":"2024-09-10T15:55:38.270056+0000","last_clean":"2024-09-10T15:55:38.270056+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:38.270056+0000","last_undegraded":"2024-09-10T15:55:38.270056+0000","last_fullsized":"2024-09-10T15:55:38.270056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8502,"log_dups_size":0,"ondisk_log_size":8502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34795520,"num_objects":8495,"num_object_clones":0,"num_object_copies":16990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5573,"num_write_kb":22292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8747","reported_seq":9564,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:38.205993+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:38.205993+0000","last_peered":"2024-09-10T15:55:38.205993+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:38.205993+0000","last_undegraded":"2024-09-10T15:55:38.205993+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8747,"log_dups_size":0,"ondisk_log_size":8747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35807232,"num_objects":8742,"num_object_clones":0,"num_object_copies":17484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3527,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5465,"num_write_kb":21860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":1396736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5215},{"shards":"0,2","objects":3527}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8458","reported_seq":8598,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.224703+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:38.224703+0000","last_peered":"2024-09-10T15:55:38.224703+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:38.224703+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8458,"log_dups_size":0,"ondisk_log_size":8458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34619392,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5108,"num_write_kb":20432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4618},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8725","reported_seq":8856,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:38.146166+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:38.146166+0000","last_peered":"2024-09-10T15:55:38.146166+0000","last_clean":"2024-09-10T15:55:38.146166+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:38.146166+0000","last_undegraded":"2024-09-10T15:55:38.146166+0000","last_fullsized":"2024-09-10T15:55:38.146166+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8725,"log_dups_size":0,"ondisk_log_size":8725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35729408,"num_objects":8723,"num_object_clones":0,"num_object_copies":17446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5484,"num_write_kb":21936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8716","reported_seq":16843,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:38.733467+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:38.733467+0000","last_peered":"2024-09-10T15:55:38.733467+0000","last_clean":"2024-09-10T15:55:38.733467+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:38.733467+0000","last_undegraded":"2024-09-10T15:55:38.733467+0000","last_fullsized":"2024-09-10T15:55:38.733467+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8716,"log_dups_size":0,"ondisk_log_size":8716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35672064,"num_objects":8709,"num_object_clones":0,"num_object_copies":17418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6693,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8599","reported_seq":8724,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.288611+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:38.288611+0000","last_peered":"2024-09-10T15:55:38.288611+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:38.288611+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8599,"log_dups_size":0,"ondisk_log_size":8599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35188736,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5463,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4728},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8721","reported_seq":8845,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:39.671240+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:39.671240+0000","last_peered":"2024-09-10T15:55:39.671240+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:39.671240+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8721,"log_dups_size":0,"ondisk_log_size":8721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35700736,"num_objects":8716,"num_object_clones":0,"num_object_copies":17432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6695,"num_write_kb":26780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4884}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8585","reported_seq":8710,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:39.619484+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:39.619484+0000","last_peered":"2024-09-10T15:55:39.619484+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:39.619484+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8585,"log_dups_size":0,"ondisk_log_size":8585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35139584,"num_objects":8579,"num_object_clones":0,"num_object_copies":17158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6454,"num_write_kb":25816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4750}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8685","reported_seq":8812,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.270066+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:38.270066+0000","last_peered":"2024-09-10T15:55:38.270066+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:38.270066+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8685,"log_dups_size":0,"ondisk_log_size":8685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5877,"num_write_kb":23508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4850},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8657","reported_seq":8820,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.224998+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:38.224998+0000","last_peered":"2024-09-10T15:55:38.224998+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:38.224998+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8657,"log_dups_size":0,"ondisk_log_size":8657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426304,"num_objects":8649,"num_object_clones":0,"num_object_copies":17298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5075,"num_write_kb":20300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8627","reported_seq":8766,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.250498+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:38.250498+0000","last_peered":"2024-09-10T15:55:38.250498+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:38.250498+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8627,"log_dups_size":0,"ondisk_log_size":8627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35319808,"num_objects":8623,"num_object_clones":0,"num_object_copies":17246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5233,"num_write_kb":20932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4746},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8846","reported_seq":8984,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:38.248208+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:38.248208+0000","last_peered":"2024-09-10T15:55:38.248208+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:38.248208+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":8846,"log_dups_size":0,"ondisk_log_size":8846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36184547,"num_objects":8835,"num_object_clones":0,"num_object_copies":17670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5665,"num_write_kb":22660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4832},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":567919075,"num_objects":138653,"num_object_clones":0,"num_object_copies":277306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3527,"num_objects_unfound":0,"num_objects_dirty":138653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100255,"num_write_kb":401020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4337,"num_bytes_recovered":17756160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1154170880,"data_stored":1154087781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138734,"ondisk_log_size":138734,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626463,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437072,"kb_used_data":247168,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93934768,"statfs":{"total":96636764160,"available":96189202432,"internally_reserved":0,"allocated":253100032,"data_stored":252761719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":194440878},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149746,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":775664,"kb_used_data":477472,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93596176,"statfs":{"total":96636764160,"available":95842484224,"internally_reserved":0,"allocated":488931328,"data_stored":488665275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305327789},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658916,"kb_used_data":402388,"kb_used_omap":4,"kb_used_meta":256507,"kb_avail":93712924,"statfs":{"total":96636764160,"available":95962034176,"internally_reserved":0,"allocated":412045312,"data_stored":411779259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":262663850},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":489021440,"data_stored":489017827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252829696,"data_stored":252753823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":412319744,"data_stored":412316131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:41.823 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 184 20 140112 140092 2.97367 1.89844 0.0164766 0.0262661 2024-09-10T15:55:42.823 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 185 20 140627 140607 2.96847 2.01172 0.0344139 0.0263109 2024-09-10T15:55:43.823 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 186 20 141140 141120 2.96328 2.00391 0.0614463 0.0263578 2024-09-10T15:55:44.330 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:44.823 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 187 20 141623 141603 2.95753 1.88672 0.0286516 0.0264073 2024-09-10T15:55:44.883 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:44.884 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:44.909 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-10T15:55:44.481403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":574849526,"num_objects":140347,"num_object_clones":0,"num_object_copies":280694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3448,"num_objects_unfound":0,"num_objects_dirty":140347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101949,"num_write_kb":407790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4418,"num_bytes_recovered":18079763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140429,"ondisk_log_size":140429,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898788,"kb_used_data":1141684,"kb_used_omap":12,"kb_used_meta":757043,"kb_avail":281216732,"statfs":{"total":289910292480,"available":287965933568,"internally_reserved":0,"allocated":1169084416,"data_stored":1168213997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":775212037},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,6,9,12,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27164672,"num_objects":6632,"num_object_clones":0,"num_object_copies":13264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-223,"num_objects_unfound":0,"num_objects_dirty":6632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6632,"num_write_kb":26528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003523"},"pg_stats":[{"pgid":"2.f","version":"46'8842","reported_seq":8918,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.363421+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:43.363421+0000","last_peered":"2024-09-10T15:55:43.363421+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:43.363421+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8842,"log_dups_size":0,"ondisk_log_size":8842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36216832,"num_objects":8842,"num_object_clones":0,"num_object_copies":17684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8842,"num_write_kb":35368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4869},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8867","reported_seq":8973,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:39.686072+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:39.686072+0000","last_peered":"2024-09-10T15:55:39.686072+0000","last_clean":"2024-09-10T15:55:39.686072+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:39.686072+0000","last_undegraded":"2024-09-10T15:55:39.686072+0000","last_fullsized":"2024-09-10T15:55:39.686072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8867,"log_dups_size":0,"ondisk_log_size":8867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36294656,"num_objects":8861,"num_object_clones":0,"num_object_copies":17722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7203,"num_write_kb":28812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8642","reported_seq":8765,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:39.571478+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:39.571478+0000","last_peered":"2024-09-10T15:55:39.571478+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:39.571478+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8642,"log_dups_size":0,"ondisk_log_size":8642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35381248,"num_objects":8638,"num_object_clones":0,"num_object_copies":17276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6910,"num_write_kb":27640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4836}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8655","reported_seq":8733,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:39.653317+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:39.653317+0000","last_peered":"2024-09-10T15:55:39.653317+0000","last_clean":"2024-09-10T15:55:39.653317+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:39.653317+0000","last_undegraded":"2024-09-10T15:55:39.653317+0000","last_fullsized":"2024-09-10T15:55:39.653317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8655,"log_dups_size":0,"ondisk_log_size":8655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450880,"num_objects":8655,"num_object_clones":0,"num_object_copies":17310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8655,"num_write_kb":34620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8654","reported_seq":8780,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:43.337375+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:43.337375+0000","last_peered":"2024-09-10T15:55:43.337375+0000","last_clean":"2024-09-10T15:55:43.337375+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:43.337375+0000","last_undegraded":"2024-09-10T15:55:43.337375+0000","last_fullsized":"2024-09-10T15:55:43.337375+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8654,"log_dups_size":0,"ondisk_log_size":8654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35418112,"num_objects":8647,"num_object_clones":0,"num_object_copies":17294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5725,"num_write_kb":22900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8909","reported_seq":9882,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:43.379506+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:43.379506+0000","last_peered":"2024-09-10T15:55:43.379506+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:43.379506+0000","last_undegraded":"2024-09-10T15:55:43.379506+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8909,"log_dups_size":0,"ondisk_log_size":8909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36470784,"num_objects":8904,"num_object_clones":0,"num_object_copies":17808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3448,"num_objects_unfound":0,"num_objects_dirty":8904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5627,"num_write_kb":22508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":1720320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5456},{"shards":"0,2","objects":3448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8607","reported_seq":8747,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.318042+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:43.318042+0000","last_peered":"2024-09-10T15:55:43.318042+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:43.318042+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8607,"log_dups_size":0,"ondisk_log_size":8607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35229696,"num_objects":8601,"num_object_clones":0,"num_object_copies":17202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5257,"num_write_kb":21028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4767},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8887","reported_seq":9017,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:43.372732+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:43.372732+0000","last_peered":"2024-09-10T15:55:43.372732+0000","last_clean":"2024-09-10T15:55:43.372732+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:43.372732+0000","last_undegraded":"2024-09-10T15:55:43.372732+0000","last_fullsized":"2024-09-10T15:55:43.372732+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8887,"log_dups_size":0,"ondisk_log_size":8887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36392960,"num_objects":8885,"num_object_clones":0,"num_object_copies":17770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5646,"num_write_kb":22584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8869","reported_seq":16996,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:43.703097+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:43.703097+0000","last_peered":"2024-09-10T15:55:43.703097+0000","last_clean":"2024-09-10T15:55:43.703097+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:43.703097+0000","last_undegraded":"2024-09-10T15:55:43.703097+0000","last_fullsized":"2024-09-10T15:55:43.703097+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8869,"log_dups_size":0,"ondisk_log_size":8869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36298752,"num_objects":8862,"num_object_clones":0,"num_object_copies":17724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6846,"num_write_kb":27384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8727","reported_seq":8853,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.345614+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:43.345614+0000","last_peered":"2024-09-10T15:55:43.345614+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:43.345614+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8727,"log_dups_size":0,"ondisk_log_size":8727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35713024,"num_objects":8719,"num_object_clones":0,"num_object_copies":17438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5591,"num_write_kb":22364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4856},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8721","reported_seq":8845,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:39.671240+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:39.671240+0000","last_peered":"2024-09-10T15:55:39.671240+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:39.671240+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8721,"log_dups_size":0,"ondisk_log_size":8721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35700736,"num_objects":8716,"num_object_clones":0,"num_object_copies":17432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6695,"num_write_kb":26780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4884}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8585","reported_seq":8710,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:39.619484+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:39.619484+0000","last_peered":"2024-09-10T15:55:39.619484+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:39.619484+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8585,"log_dups_size":0,"ondisk_log_size":8585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35139584,"num_objects":8579,"num_object_clones":0,"num_object_copies":17158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6454,"num_write_kb":25816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4750}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8827","reported_seq":8954,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.365588+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:43.365588+0000","last_peered":"2024-09-10T15:55:43.365588+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:43.365588+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36147200,"num_objects":8825,"num_object_clones":0,"num_object_copies":17650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6019,"num_write_kb":24076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4992},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8809","reported_seq":8972,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.433236+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:43.433236+0000","last_peered":"2024-09-10T15:55:43.433236+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:43.433236+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8809,"log_dups_size":0,"ondisk_log_size":8809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36048896,"num_objects":8801,"num_object_clones":0,"num_object_copies":17602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5227,"num_write_kb":20908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4884},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8806","reported_seq":8945,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.384569+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:43.384569+0000","last_peered":"2024-09-10T15:55:43.384569+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:43.384569+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8806,"log_dups_size":0,"ondisk_log_size":8806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36052992,"num_objects":8802,"num_object_clones":0,"num_object_copies":17604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5412,"num_write_kb":21648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4925},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9019","reported_seq":9156,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.405395+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:43.405395+0000","last_peered":"2024-09-10T15:55:43.405395+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:43.405395+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9019,"log_dups_size":0,"ondisk_log_size":9019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36893155,"num_objects":9008,"num_object_clones":0,"num_object_copies":18016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5838,"num_write_kb":23352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5005},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":574849507,"num_objects":140345,"num_object_clones":0,"num_object_copies":280690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3448,"num_objects_unfound":0,"num_objects_dirty":140345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101947,"num_write_kb":407788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4416,"num_bytes_recovered":18079744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1169518592,"data_stored":1169435493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140426,"ondisk_log_size":140426,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626464,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452860,"kb_used_data":254636,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93918980,"statfs":{"total":96636764160,"available":96173035520,"internally_reserved":0,"allocated":260747264,"data_stored":260408951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":202960558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149747,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":787012,"kb_used_data":484660,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93584828,"statfs":{"total":96636764160,"available":95830863872,"internally_reserved":0,"allocated":496291840,"data_stored":496025787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309587629},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,3,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658916,"kb_used_data":402388,"kb_used_omap":4,"kb_used_meta":256507,"kb_avail":93712924,"statfs":{"total":96636764160,"available":95962034176,"internally_reserved":0,"allocated":412045312,"data_stored":411779259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":262663850},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":496652288,"data_stored":496648675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260546560,"data_stored":260470687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":412319744,"data_stored":412316131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:44.911 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:45.460 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:45.462 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:45.486 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-10T15:55:44.481403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":574849526,"num_objects":140347,"num_object_clones":0,"num_object_copies":280694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3448,"num_objects_unfound":0,"num_objects_dirty":140347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101949,"num_write_kb":407790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4418,"num_bytes_recovered":18079763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140429,"ondisk_log_size":140429,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898788,"kb_used_data":1141684,"kb_used_omap":12,"kb_used_meta":757043,"kb_avail":281216732,"statfs":{"total":289910292480,"available":287965933568,"internally_reserved":0,"allocated":1169084416,"data_stored":1168213997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":775212037},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,6,9,12,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27164672,"num_objects":6632,"num_object_clones":0,"num_object_copies":13264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-223,"num_objects_unfound":0,"num_objects_dirty":6632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6632,"num_write_kb":26528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003523"},"pg_stats":[{"pgid":"2.f","version":"46'8842","reported_seq":8918,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.363421+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:43.363421+0000","last_peered":"2024-09-10T15:55:43.363421+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:43.363421+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8842,"log_dups_size":0,"ondisk_log_size":8842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36216832,"num_objects":8842,"num_object_clones":0,"num_object_copies":17684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8842,"num_write_kb":35368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4869},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8867","reported_seq":8973,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:39.686072+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:39.686072+0000","last_peered":"2024-09-10T15:55:39.686072+0000","last_clean":"2024-09-10T15:55:39.686072+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:39.686072+0000","last_undegraded":"2024-09-10T15:55:39.686072+0000","last_fullsized":"2024-09-10T15:55:39.686072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":8867,"log_dups_size":0,"ondisk_log_size":8867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36294656,"num_objects":8861,"num_object_clones":0,"num_object_copies":17722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7203,"num_write_kb":28812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8642","reported_seq":8765,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:39.571478+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:39.571478+0000","last_peered":"2024-09-10T15:55:39.571478+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:39.571478+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8642,"log_dups_size":0,"ondisk_log_size":8642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35381248,"num_objects":8638,"num_object_clones":0,"num_object_copies":17276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6910,"num_write_kb":27640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4836}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8655","reported_seq":8733,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:39.653317+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:39.653317+0000","last_peered":"2024-09-10T15:55:39.653317+0000","last_clean":"2024-09-10T15:55:39.653317+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:39.653317+0000","last_undegraded":"2024-09-10T15:55:39.653317+0000","last_fullsized":"2024-09-10T15:55:39.653317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8655,"log_dups_size":0,"ondisk_log_size":8655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450880,"num_objects":8655,"num_object_clones":0,"num_object_copies":17310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8655,"num_write_kb":34620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8654","reported_seq":8780,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:43.337375+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:43.337375+0000","last_peered":"2024-09-10T15:55:43.337375+0000","last_clean":"2024-09-10T15:55:43.337375+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:43.337375+0000","last_undegraded":"2024-09-10T15:55:43.337375+0000","last_fullsized":"2024-09-10T15:55:43.337375+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8654,"log_dups_size":0,"ondisk_log_size":8654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35418112,"num_objects":8647,"num_object_clones":0,"num_object_copies":17294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5725,"num_write_kb":22900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8909","reported_seq":9882,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:43.379506+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:43.379506+0000","last_peered":"2024-09-10T15:55:43.379506+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:43.379506+0000","last_undegraded":"2024-09-10T15:55:43.379506+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8909,"log_dups_size":0,"ondisk_log_size":8909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36470784,"num_objects":8904,"num_object_clones":0,"num_object_copies":17808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3448,"num_objects_unfound":0,"num_objects_dirty":8904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5627,"num_write_kb":22508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":1720320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5456},{"shards":"0,2","objects":3448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8607","reported_seq":8747,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.318042+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:43.318042+0000","last_peered":"2024-09-10T15:55:43.318042+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:43.318042+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8607,"log_dups_size":0,"ondisk_log_size":8607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35229696,"num_objects":8601,"num_object_clones":0,"num_object_copies":17202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5257,"num_write_kb":21028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4767},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8887","reported_seq":9017,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:43.372732+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:43.372732+0000","last_peered":"2024-09-10T15:55:43.372732+0000","last_clean":"2024-09-10T15:55:43.372732+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:43.372732+0000","last_undegraded":"2024-09-10T15:55:43.372732+0000","last_fullsized":"2024-09-10T15:55:43.372732+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8887,"log_dups_size":0,"ondisk_log_size":8887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36392960,"num_objects":8885,"num_object_clones":0,"num_object_copies":17770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5646,"num_write_kb":22584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8869","reported_seq":16996,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:43.703097+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:43.703097+0000","last_peered":"2024-09-10T15:55:43.703097+0000","last_clean":"2024-09-10T15:55:43.703097+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:43.703097+0000","last_undegraded":"2024-09-10T15:55:43.703097+0000","last_fullsized":"2024-09-10T15:55:43.703097+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8869,"log_dups_size":0,"ondisk_log_size":8869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36298752,"num_objects":8862,"num_object_clones":0,"num_object_copies":17724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6846,"num_write_kb":27384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8727","reported_seq":8853,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.345614+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:43.345614+0000","last_peered":"2024-09-10T15:55:43.345614+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:43.345614+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8727,"log_dups_size":0,"ondisk_log_size":8727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35713024,"num_objects":8719,"num_object_clones":0,"num_object_copies":17438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5591,"num_write_kb":22364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4856},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8721","reported_seq":8845,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:39.671240+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:39.671240+0000","last_peered":"2024-09-10T15:55:39.671240+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:39.671240+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8721,"log_dups_size":0,"ondisk_log_size":8721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35700736,"num_objects":8716,"num_object_clones":0,"num_object_copies":17432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6695,"num_write_kb":26780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":4884}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8585","reported_seq":8710,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:39.619484+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:39.619484+0000","last_peered":"2024-09-10T15:55:39.619484+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:39.619484+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8585,"log_dups_size":0,"ondisk_log_size":8585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35139584,"num_objects":8579,"num_object_clones":0,"num_object_copies":17158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6454,"num_write_kb":25816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4750}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8827","reported_seq":8954,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.365588+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:43.365588+0000","last_peered":"2024-09-10T15:55:43.365588+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:43.365588+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36147200,"num_objects":8825,"num_object_clones":0,"num_object_copies":17650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6019,"num_write_kb":24076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4992},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8809","reported_seq":8972,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.433236+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:43.433236+0000","last_peered":"2024-09-10T15:55:43.433236+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:43.433236+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8809,"log_dups_size":0,"ondisk_log_size":8809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36048896,"num_objects":8801,"num_object_clones":0,"num_object_copies":17602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5227,"num_write_kb":20908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4884},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8806","reported_seq":8945,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.384569+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:43.384569+0000","last_peered":"2024-09-10T15:55:43.384569+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:43.384569+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8806,"log_dups_size":0,"ondisk_log_size":8806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36052992,"num_objects":8802,"num_object_clones":0,"num_object_copies":17604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5412,"num_write_kb":21648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4925},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9019","reported_seq":9156,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.405395+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:43.405395+0000","last_peered":"2024-09-10T15:55:43.405395+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:43.405395+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9019,"log_dups_size":0,"ondisk_log_size":9019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36893155,"num_objects":9008,"num_object_clones":0,"num_object_copies":18016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5838,"num_write_kb":23352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5005},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":574849507,"num_objects":140345,"num_object_clones":0,"num_object_copies":280690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3448,"num_objects_unfound":0,"num_objects_dirty":140345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101947,"num_write_kb":407788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4416,"num_bytes_recovered":18079744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1169518592,"data_stored":1169435493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140426,"ondisk_log_size":140426,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626464,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452860,"kb_used_data":254636,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93918980,"statfs":{"total":96636764160,"available":96173035520,"internally_reserved":0,"allocated":260747264,"data_stored":260408951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":202960558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149747,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":787012,"kb_used_data":484660,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93584828,"statfs":{"total":96636764160,"available":95830863872,"internally_reserved":0,"allocated":496291840,"data_stored":496025787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309587629},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,3,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658916,"kb_used_data":402388,"kb_used_omap":4,"kb_used_meta":256507,"kb_avail":93712924,"statfs":{"total":96636764160,"available":95962034176,"internally_reserved":0,"allocated":412045312,"data_stored":411779259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":262663850},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":496652288,"data_stored":496648675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260546560,"data_stored":260470687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":412319744,"data_stored":412316131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:45.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:55:45.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'8842", 'reported_seq': 8918, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:43.363421+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:55:43.363421+0000', 'last_peered': '2024-09-10T15:55:43.363421+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:55:43.363421+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 8842, 'log_dups_size': 0, 'ondisk_log_size': 8842, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36216832, 'num_objects': 8842, 'num_object_clones': 0, 'num_object_copies': 17684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8842, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8842, 'num_write_kb': 35368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4869}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:45.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:55:45.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'8642", 'reported_seq': 8765, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:39.571478+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:55:39.571478+0000', 'last_peered': '2024-09-10T15:55:39.571478+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:55:39.571478+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 8642, 'log_dups_size': 0, 'ondisk_log_size': 8642, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35381248, 'num_objects': 8638, 'num_object_clones': 0, 'num_object_copies': 17276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8638, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6910, 'num_write_kb': 27640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 4836}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:45.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:55:45.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'8909", 'reported_seq': 9882, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:55:43.379506+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:55:43.379506+0000', 'last_peered': '2024-09-10T15:55:43.379506+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:55:43.379506+0000', 'last_undegraded': '2024-09-10T15:55:43.379506+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 8909, 'log_dups_size': 0, 'ondisk_log_size': 8909, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36470784, 'num_objects': 8904, 'num_object_clones': 0, 'num_object_copies': 17808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3448, 'num_objects_unfound': 0, 'num_objects_dirty': 8904, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5627, 'num_write_kb': 22508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 420, 'num_bytes_recovered': 1720320, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5456}, {'shards': '0,2', 'objects': 3448}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:45.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:55:45.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'8607", 'reported_seq': 8747, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:43.318042+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:55:43.318042+0000', 'last_peered': '2024-09-10T15:55:43.318042+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:55:43.318042+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 8607, 'log_dups_size': 0, 'ondisk_log_size': 8607, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35229696, 'num_objects': 8601, 'num_object_clones': 0, 'num_object_copies': 17202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8601, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5257, 'num_write_kb': 21028, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4767}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:45.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:55:45.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'8727", 'reported_seq': 8853, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:43.345614+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:55:43.345614+0000', 'last_peered': '2024-09-10T15:55:43.345614+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:55:43.345614+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 8727, 'log_dups_size': 0, 'ondisk_log_size': 8727, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35713024, 'num_objects': 8719, 'num_object_clones': 0, 'num_object_copies': 17438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8719, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5591, 'num_write_kb': 22364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4856}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:45.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:55:45.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'8721", 'reported_seq': 8845, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:39.671240+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:55:39.671240+0000', 'last_peered': '2024-09-10T15:55:39.671240+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:55:39.671240+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 8721, 'log_dups_size': 0, 'ondisk_log_size': 8721, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35700736, 'num_objects': 8716, 'num_object_clones': 0, 'num_object_copies': 17432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8716, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6695, 'num_write_kb': 26780, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 4884}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:45.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:55:45.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'8585", 'reported_seq': 8710, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:39.619484+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:55:39.619484+0000', 'last_peered': '2024-09-10T15:55:39.619484+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:55:39.619484+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 8585, 'log_dups_size': 0, 'ondisk_log_size': 8585, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35139584, 'num_objects': 8579, 'num_object_clones': 0, 'num_object_copies': 17158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8579, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6454, 'num_write_kb': 25816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 4750}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:45.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:55:45.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'8827", 'reported_seq': 8954, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:43.365588+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:55:43.365588+0000', 'last_peered': '2024-09-10T15:55:43.365588+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:55:43.365588+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 8827, 'log_dups_size': 0, 'ondisk_log_size': 8827, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36147200, 'num_objects': 8825, 'num_object_clones': 0, 'num_object_copies': 17650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8825, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6019, 'num_write_kb': 24076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4992}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:45.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:55:45.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'8809", 'reported_seq': 8972, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:43.433236+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:55:43.433236+0000', 'last_peered': '2024-09-10T15:55:43.433236+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:55:43.433236+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 8809, 'log_dups_size': 0, 'ondisk_log_size': 8809, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36048896, 'num_objects': 8801, 'num_object_clones': 0, 'num_object_copies': 17602, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8801, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5227, 'num_write_kb': 20908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4884}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:45.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:55:45.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'8806", 'reported_seq': 8945, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:43.384569+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:55:43.384569+0000', 'last_peered': '2024-09-10T15:55:43.384569+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:55:43.384569+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 8806, 'log_dups_size': 0, 'ondisk_log_size': 8806, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36052992, 'num_objects': 8802, 'num_object_clones': 0, 'num_object_copies': 17604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8802, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5412, 'num_write_kb': 21648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4925}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:45.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:55:45.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'9019", 'reported_seq': 9156, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:43.405395+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:55:43.405395+0000', 'last_peered': '2024-09-10T15:55:43.405395+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:55:43.405395+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 9019, 'log_dups_size': 0, 'ondisk_log_size': 9019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36893155, 'num_objects': 9008, 'num_object_clones': 0, 'num_object_copies': 18016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9008, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5838, 'num_write_kb': 23352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5005}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:45.490 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:55:45.823 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 188 20 142083 142063 2.95135 1.79688 0.0197103 0.0264635 2024-09-10T15:55:46.200 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:46.200 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/280694 objects degraded (13.816%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":272,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140347,"data_bytes":574849526,"bytes_used":1944358912,"bytes_avail":287965933568,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":280694,"degraded_ratio":0.13816112920119419,"misplaced_objects":3448,"misplaced_total":280694,"misplaced_ratio":0.012283839341061798,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":76094,"recovering_keys_per_sec":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"write_bytes_sec":2263058,"read_op_per_sec":0,"write_op_per_sec":552},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:55:46.225 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:55:46.225 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:46.823 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 189 20 142520 142500 2.94477 1.70703 0.0585783 0.0265225 2024-09-10T15:55:46.827 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:46.829 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:46.849 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-10T15:55:46.482079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":578052598,"num_objects":141129,"num_object_clones":0,"num_object_copies":282258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3448,"num_objects_unfound":0,"num_objects_dirty":141129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102731,"num_write_kb":410918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4418,"num_bytes_recovered":18079763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141211,"ondisk_log_size":141211,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1925956,"kb_used_data":1146644,"kb_used_omap":12,"kb_used_meta":779251,"kb_avail":281189564,"statfs":{"total":289910292480,"available":287938113536,"internally_reserved":0,"allocated":1174163456,"data_stored":1173293037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":797953029},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,5,7,15,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23461888,"num_objects":5728,"num_object_clones":0,"num_object_copies":11456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-148,"num_objects_unfound":0,"num_objects_dirty":5728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5728,"num_write_kb":22912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":606208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003712"},"pg_stats":[{"pgid":"2.f","version":"46'8842","reported_seq":8918,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.363421+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:43.363421+0000","last_peered":"2024-09-10T15:55:43.363421+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:43.363421+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8842,"log_dups_size":0,"ondisk_log_size":8842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36216832,"num_objects":8842,"num_object_clones":0,"num_object_copies":17684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8842,"num_write_kb":35368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4869},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9038","reported_seq":9145,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:44.642908+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:44.642908+0000","last_peered":"2024-09-10T15:55:44.642908+0000","last_clean":"2024-09-10T15:55:44.642908+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:44.642908+0000","last_undegraded":"2024-09-10T15:55:44.642908+0000","last_fullsized":"2024-09-10T15:55:44.642908+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9038,"log_dups_size":0,"ondisk_log_size":9038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36995072,"num_objects":9032,"num_object_clones":0,"num_object_copies":18064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7374,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8784","reported_seq":8907,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:44.541933+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:44.541933+0000","last_peered":"2024-09-10T15:55:44.541933+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:44.541933+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8784,"log_dups_size":0,"ondisk_log_size":8784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962880,"num_objects":8780,"num_object_clones":0,"num_object_copies":17560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7052,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4978}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8808","reported_seq":8887,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:44.674592+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:44.674592+0000","last_peered":"2024-09-10T15:55:44.674592+0000","last_clean":"2024-09-10T15:55:44.674592+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:44.674592+0000","last_undegraded":"2024-09-10T15:55:44.674592+0000","last_fullsized":"2024-09-10T15:55:44.674592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8808,"log_dups_size":0,"ondisk_log_size":8808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36077568,"num_objects":8808,"num_object_clones":0,"num_object_copies":17616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8808,"num_write_kb":35232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8654","reported_seq":8780,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:43.337375+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:43.337375+0000","last_peered":"2024-09-10T15:55:43.337375+0000","last_clean":"2024-09-10T15:55:43.337375+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:43.337375+0000","last_undegraded":"2024-09-10T15:55:43.337375+0000","last_fullsized":"2024-09-10T15:55:43.337375+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8654,"log_dups_size":0,"ondisk_log_size":8654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35418112,"num_objects":8647,"num_object_clones":0,"num_object_copies":17294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5725,"num_write_kb":22900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8909","reported_seq":9882,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:43.379506+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:43.379506+0000","last_peered":"2024-09-10T15:55:43.379506+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:43.379506+0000","last_undegraded":"2024-09-10T15:55:43.379506+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8909,"log_dups_size":0,"ondisk_log_size":8909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36470784,"num_objects":8904,"num_object_clones":0,"num_object_copies":17808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3448,"num_objects_unfound":0,"num_objects_dirty":8904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5627,"num_write_kb":22508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":1720320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5456},{"shards":"0,2","objects":3448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8607","reported_seq":8747,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.318042+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:43.318042+0000","last_peered":"2024-09-10T15:55:43.318042+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:43.318042+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8607,"log_dups_size":0,"ondisk_log_size":8607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35229696,"num_objects":8601,"num_object_clones":0,"num_object_copies":17202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5257,"num_write_kb":21028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4767},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8887","reported_seq":9017,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:43.372732+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:43.372732+0000","last_peered":"2024-09-10T15:55:43.372732+0000","last_clean":"2024-09-10T15:55:43.372732+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:43.372732+0000","last_undegraded":"2024-09-10T15:55:43.372732+0000","last_fullsized":"2024-09-10T15:55:43.372732+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8887,"log_dups_size":0,"ondisk_log_size":8887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36392960,"num_objects":8885,"num_object_clones":0,"num_object_copies":17770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5646,"num_write_kb":22584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8869","reported_seq":16996,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:43.703097+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:43.703097+0000","last_peered":"2024-09-10T15:55:43.703097+0000","last_clean":"2024-09-10T15:55:43.703097+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:43.703097+0000","last_undegraded":"2024-09-10T15:55:43.703097+0000","last_fullsized":"2024-09-10T15:55:43.703097+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8869,"log_dups_size":0,"ondisk_log_size":8869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36298752,"num_objects":8862,"num_object_clones":0,"num_object_copies":17724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6846,"num_write_kb":27384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8727","reported_seq":8853,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.345614+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:43.345614+0000","last_peered":"2024-09-10T15:55:43.345614+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:43.345614+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8727,"log_dups_size":0,"ondisk_log_size":8727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35713024,"num_objects":8719,"num_object_clones":0,"num_object_copies":17438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5591,"num_write_kb":22364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4856},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8871","reported_seq":8995,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:44.678092+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:44.678092+0000","last_peered":"2024-09-10T15:55:44.678092+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:44.678092+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8871,"log_dups_size":0,"ondisk_log_size":8871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36315136,"num_objects":8866,"num_object_clones":0,"num_object_copies":17732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6845,"num_write_kb":27380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8751","reported_seq":8877,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:44.701556+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:44.701556+0000","last_peered":"2024-09-10T15:55:44.701556+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:44.701556+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8751,"log_dups_size":0,"ondisk_log_size":8751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35819520,"num_objects":8745,"num_object_clones":0,"num_object_copies":17490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6620,"num_write_kb":26480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4916}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8827","reported_seq":8954,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.365588+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:43.365588+0000","last_peered":"2024-09-10T15:55:43.365588+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:43.365588+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36147200,"num_objects":8825,"num_object_clones":0,"num_object_copies":17650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6019,"num_write_kb":24076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4992},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8809","reported_seq":8972,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.433236+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:43.433236+0000","last_peered":"2024-09-10T15:55:43.433236+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:43.433236+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8809,"log_dups_size":0,"ondisk_log_size":8809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36048896,"num_objects":8801,"num_object_clones":0,"num_object_copies":17602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5227,"num_write_kb":20908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4884},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8806","reported_seq":8945,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.384569+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:43.384569+0000","last_peered":"2024-09-10T15:55:43.384569+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:43.384569+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8806,"log_dups_size":0,"ondisk_log_size":8806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36052992,"num_objects":8802,"num_object_clones":0,"num_object_copies":17604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5412,"num_write_kb":21648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4925},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9019","reported_seq":9156,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.405395+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:43.405395+0000","last_peered":"2024-09-10T15:55:43.405395+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:43.405395+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9019,"log_dups_size":0,"ondisk_log_size":9019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36893155,"num_objects":9008,"num_object_clones":0,"num_object_copies":18016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5838,"num_write_kb":23352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5005},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":578052579,"num_objects":141127,"num_object_clones":0,"num_object_copies":282254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3448,"num_objects_unfound":0,"num_objects_dirty":141127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102729,"num_write_kb":410916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4416,"num_bytes_recovered":18079744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1174622208,"data_stored":1174539109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141208,"ondisk_log_size":141208,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626464,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452860,"kb_used_data":254636,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93918980,"statfs":{"total":96636764160,"available":96173035520,"internally_reserved":0,"allocated":260747264,"data_stored":260408951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":202960558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149747,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":787012,"kb_used_data":484660,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93584828,"statfs":{"total":96636764160,"available":95830863872,"internally_reserved":0,"allocated":496291840,"data_stored":496025787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309587629},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,3,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686084,"kb_used_data":407348,"kb_used_omap":4,"kb_used_meta":278715,"kb_avail":93685756,"statfs":{"total":96636764160,"available":95934214144,"internally_reserved":0,"allocated":417124352,"data_stored":416858299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":285404842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":496652288,"data_stored":496648675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260546560,"data_stored":260470687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417423360,"data_stored":417419747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:47.823 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 190 20 143030 143010 2.93975 1.99219 0.0300407 0.0265676 2024-09-10T15:55:48.823 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 191 20 143537 143517 2.93473 1.98047 0.055775 0.0266138 2024-09-10T15:55:49.823 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 192 20 144027 144007 2.92942 1.91406 0.023878 0.0266619 2024-09-10T15:55:49.851 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:50.408 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:50.411 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:50.438 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-10T15:55:48.482743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":578052598,"num_objects":141129,"num_object_clones":0,"num_object_copies":282258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3448,"num_objects_unfound":0,"num_objects_dirty":141129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102731,"num_write_kb":410918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4418,"num_bytes_recovered":18079763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141211,"ondisk_log_size":141211,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1925956,"kb_used_data":1146644,"kb_used_omap":12,"kb_used_meta":779251,"kb_avail":281189564,"statfs":{"total":289910292480,"available":287938113536,"internally_reserved":0,"allocated":1174163456,"data_stored":1173293037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":797953029},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,5,7,15,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20082688,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-148,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4903,"num_write_kb":19612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":606208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003464"},"pg_stats":[{"pgid":"2.f","version":"46'8842","reported_seq":8918,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.363421+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:43.363421+0000","last_peered":"2024-09-10T15:55:43.363421+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:43.363421+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8842,"log_dups_size":0,"ondisk_log_size":8842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36216832,"num_objects":8842,"num_object_clones":0,"num_object_copies":17684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8842,"num_write_kb":35368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4869},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9038","reported_seq":9145,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:44.642908+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:44.642908+0000","last_peered":"2024-09-10T15:55:44.642908+0000","last_clean":"2024-09-10T15:55:44.642908+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:44.642908+0000","last_undegraded":"2024-09-10T15:55:44.642908+0000","last_fullsized":"2024-09-10T15:55:44.642908+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9038,"log_dups_size":0,"ondisk_log_size":9038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36995072,"num_objects":9032,"num_object_clones":0,"num_object_copies":18064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7374,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8784","reported_seq":8907,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:44.541933+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:44.541933+0000","last_peered":"2024-09-10T15:55:44.541933+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:44.541933+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8784,"log_dups_size":0,"ondisk_log_size":8784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962880,"num_objects":8780,"num_object_clones":0,"num_object_copies":17560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7052,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":4978}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8808","reported_seq":8887,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:44.674592+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:44.674592+0000","last_peered":"2024-09-10T15:55:44.674592+0000","last_clean":"2024-09-10T15:55:44.674592+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:44.674592+0000","last_undegraded":"2024-09-10T15:55:44.674592+0000","last_fullsized":"2024-09-10T15:55:44.674592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8808,"log_dups_size":0,"ondisk_log_size":8808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36077568,"num_objects":8808,"num_object_clones":0,"num_object_copies":17616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8808,"num_write_kb":35232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8654","reported_seq":8780,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:43.337375+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:43.337375+0000","last_peered":"2024-09-10T15:55:43.337375+0000","last_clean":"2024-09-10T15:55:43.337375+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:43.337375+0000","last_undegraded":"2024-09-10T15:55:43.337375+0000","last_fullsized":"2024-09-10T15:55:43.337375+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8654,"log_dups_size":0,"ondisk_log_size":8654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35418112,"num_objects":8647,"num_object_clones":0,"num_object_copies":17294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5725,"num_write_kb":22900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8909","reported_seq":9882,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:43.379506+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:43.379506+0000","last_peered":"2024-09-10T15:55:43.379506+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:43.379506+0000","last_undegraded":"2024-09-10T15:55:43.379506+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":8909,"log_dups_size":0,"ondisk_log_size":8909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36470784,"num_objects":8904,"num_object_clones":0,"num_object_copies":17808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3448,"num_objects_unfound":0,"num_objects_dirty":8904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5627,"num_write_kb":22508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":1720320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5456},{"shards":"0,2","objects":3448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8607","reported_seq":8747,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.318042+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:43.318042+0000","last_peered":"2024-09-10T15:55:43.318042+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:43.318042+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8607,"log_dups_size":0,"ondisk_log_size":8607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35229696,"num_objects":8601,"num_object_clones":0,"num_object_copies":17202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5257,"num_write_kb":21028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4767},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8887","reported_seq":9017,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:43.372732+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:43.372732+0000","last_peered":"2024-09-10T15:55:43.372732+0000","last_clean":"2024-09-10T15:55:43.372732+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:43.372732+0000","last_undegraded":"2024-09-10T15:55:43.372732+0000","last_fullsized":"2024-09-10T15:55:43.372732+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":8887,"log_dups_size":0,"ondisk_log_size":8887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36392960,"num_objects":8885,"num_object_clones":0,"num_object_copies":17770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5646,"num_write_kb":22584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8869","reported_seq":16996,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:43.703097+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:43.703097+0000","last_peered":"2024-09-10T15:55:43.703097+0000","last_clean":"2024-09-10T15:55:43.703097+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:43.703097+0000","last_undegraded":"2024-09-10T15:55:43.703097+0000","last_fullsized":"2024-09-10T15:55:43.703097+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":8869,"log_dups_size":0,"ondisk_log_size":8869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36298752,"num_objects":8862,"num_object_clones":0,"num_object_copies":17724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6846,"num_write_kb":27384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8727","reported_seq":8853,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.345614+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:43.345614+0000","last_peered":"2024-09-10T15:55:43.345614+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:43.345614+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8727,"log_dups_size":0,"ondisk_log_size":8727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35713024,"num_objects":8719,"num_object_clones":0,"num_object_copies":17438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5591,"num_write_kb":22364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4856},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8871","reported_seq":8995,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:44.678092+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:44.678092+0000","last_peered":"2024-09-10T15:55:44.678092+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:44.678092+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":8871,"log_dups_size":0,"ondisk_log_size":8871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36315136,"num_objects":8866,"num_object_clones":0,"num_object_copies":17732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6845,"num_write_kb":27380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8751","reported_seq":8877,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:44.701556+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:44.701556+0000","last_peered":"2024-09-10T15:55:44.701556+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:44.701556+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8751,"log_dups_size":0,"ondisk_log_size":8751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35819520,"num_objects":8745,"num_object_clones":0,"num_object_copies":17490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6620,"num_write_kb":26480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":4916}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8827","reported_seq":8954,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.365588+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:43.365588+0000","last_peered":"2024-09-10T15:55:43.365588+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:43.365588+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36147200,"num_objects":8825,"num_object_clones":0,"num_object_copies":17650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6019,"num_write_kb":24076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4992},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8809","reported_seq":8972,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.433236+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:43.433236+0000","last_peered":"2024-09-10T15:55:43.433236+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:43.433236+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8809,"log_dups_size":0,"ondisk_log_size":8809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36048896,"num_objects":8801,"num_object_clones":0,"num_object_copies":17602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5227,"num_write_kb":20908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4884},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8806","reported_seq":8945,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.384569+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:43.384569+0000","last_peered":"2024-09-10T15:55:43.384569+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:43.384569+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8806,"log_dups_size":0,"ondisk_log_size":8806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36052992,"num_objects":8802,"num_object_clones":0,"num_object_copies":17604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5412,"num_write_kb":21648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4925},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9019","reported_seq":9156,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:43.405395+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:43.405395+0000","last_peered":"2024-09-10T15:55:43.405395+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:43.405395+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9019,"log_dups_size":0,"ondisk_log_size":9019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36893155,"num_objects":9008,"num_object_clones":0,"num_object_copies":18016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5838,"num_write_kb":23352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5005},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":578052579,"num_objects":141127,"num_object_clones":0,"num_object_copies":282254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3448,"num_objects_unfound":0,"num_objects_dirty":141127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102729,"num_write_kb":410916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4416,"num_bytes_recovered":18079744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1174622208,"data_stored":1174539109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141208,"ondisk_log_size":141208,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626464,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452860,"kb_used_data":254636,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93918980,"statfs":{"total":96636764160,"available":96173035520,"internally_reserved":0,"allocated":260747264,"data_stored":260408951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":202960558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149747,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":787012,"kb_used_data":484660,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93584828,"statfs":{"total":96636764160,"available":95830863872,"internally_reserved":0,"allocated":496291840,"data_stored":496025787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309587629},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,3,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686084,"kb_used_data":407348,"kb_used_omap":4,"kb_used_meta":278715,"kb_avail":93685756,"statfs":{"total":96636764160,"available":95934214144,"internally_reserved":0,"allocated":417124352,"data_stored":416858299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":285404842},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":496652288,"data_stored":496648675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260546560,"data_stored":260470687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417423360,"data_stored":417419747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:50.439 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:50.826 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 193 20 144456 144436 2.92292 1.67578 0.0381738 0.0267197 2024-09-10T15:55:50.975 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:50.976 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:51.013 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-10T15:55:50.483361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":588022262,"num_objects":143563,"num_object_clones":0,"num_object_copies":287126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3374,"num_objects_unfound":0,"num_objects_dirty":143563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105165,"num_write_kb":420654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4492,"num_bytes_recovered":18382867,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143645,"ondisk_log_size":143645,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1941360,"kb_used_data":1166208,"kb_used_omap":12,"kb_used_meta":775091,"kb_avail":281174160,"statfs":{"total":289910292480,"available":287922339840,"internally_reserved":0,"allocated":1194196992,"data_stored":1193326573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":793693189},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,1,3,6,13,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23838720,"num_objects":5820,"num_object_clones":0,"num_object_copies":11640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-153,"num_objects_unfound":0,"num_objects_dirty":5820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5820,"num_write_kb":23280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":626688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003387"},"pg_stats":[{"pgid":"2.f","version":"46'9002","reported_seq":9078,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.509149+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:48.509149+0000","last_peered":"2024-09-10T15:55:48.509149+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:48.509149+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9002,"log_dups_size":0,"ondisk_log_size":9002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36872192,"num_objects":9002,"num_object_clones":0,"num_object_copies":18004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9002,"num_write_kb":36008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5029},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9186","reported_seq":9293,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:49.633538+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:49.633538+0000","last_peered":"2024-09-10T15:55:49.633538+0000","last_clean":"2024-09-10T15:55:49.633538+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:49.633538+0000","last_undegraded":"2024-09-10T15:55:49.633538+0000","last_fullsized":"2024-09-10T15:55:49.633538+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9186,"log_dups_size":0,"ondisk_log_size":9186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37601280,"num_objects":9180,"num_object_clones":0,"num_object_copies":18360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":30088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8921","reported_seq":9044,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:49.625840+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:49.625840+0000","last_peered":"2024-09-10T15:55:49.625840+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:49.625840+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8921,"log_dups_size":0,"ondisk_log_size":8921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524032,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7189,"num_write_kb":28756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5115}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8943","reported_seq":9022,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:49.681452+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:49.681452+0000","last_peered":"2024-09-10T15:55:49.681452+0000","last_clean":"2024-09-10T15:55:49.681452+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:49.681452+0000","last_undegraded":"2024-09-10T15:55:49.681452+0000","last_fullsized":"2024-09-10T15:55:49.681452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8943,"log_dups_size":0,"ondisk_log_size":8943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36630528,"num_objects":8943,"num_object_clones":0,"num_object_copies":17886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8943,"num_write_kb":35772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8802","reported_seq":8928,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:48.440650+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:48.440650+0000","last_peered":"2024-09-10T15:55:48.440650+0000","last_clean":"2024-09-10T15:55:48.440650+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:48.440650+0000","last_undegraded":"2024-09-10T15:55:48.440650+0000","last_fullsized":"2024-09-10T15:55:48.440650+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8802,"log_dups_size":0,"ondisk_log_size":8802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36024320,"num_objects":8795,"num_object_clones":0,"num_object_copies":17590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5873,"num_write_kb":23492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9061","reported_seq":10181,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:48.458655+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:48.458655+0000","last_peered":"2024-09-10T15:55:48.458655+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:48.458655+0000","last_undegraded":"2024-09-10T15:55:48.458655+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9061,"log_dups_size":0,"ondisk_log_size":9061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37093376,"num_objects":9056,"num_object_clones":0,"num_object_copies":18112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3374,"num_objects_unfound":0,"num_objects_dirty":9056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5779,"num_write_kb":23116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":2023424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5682},{"shards":"0,2","objects":3374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8759","reported_seq":8899,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.412767+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:48.412767+0000","last_peered":"2024-09-10T15:55:48.412767+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:48.412767+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8759,"log_dups_size":0,"ondisk_log_size":8759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5409,"num_write_kb":21636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4919},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9049","reported_seq":9180,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:48.480595+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:48.480595+0000","last_peered":"2024-09-10T15:55:48.480595+0000","last_clean":"2024-09-10T15:55:48.480595+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:48.480595+0000","last_undegraded":"2024-09-10T15:55:48.480595+0000","last_fullsized":"2024-09-10T15:55:48.480595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9049,"log_dups_size":0,"ondisk_log_size":9049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37056512,"num_objects":9047,"num_object_clones":0,"num_object_copies":18094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5808,"num_write_kb":23232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9031","reported_seq":17157,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:48.816335+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:48.816335+0000","last_peered":"2024-09-10T15:55:48.816335+0000","last_clean":"2024-09-10T15:55:48.816335+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:48.816335+0000","last_undegraded":"2024-09-10T15:55:48.816335+0000","last_fullsized":"2024-09-10T15:55:48.816335+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9031,"log_dups_size":0,"ondisk_log_size":9031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36962304,"num_objects":9024,"num_object_clones":0,"num_object_copies":18048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7008,"num_write_kb":28032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8875","reported_seq":8998,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.388862+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:48.388862+0000","last_peered":"2024-09-10T15:55:48.388862+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:48.388862+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8875,"log_dups_size":0,"ondisk_log_size":8875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36319232,"num_objects":8867,"num_object_clones":0,"num_object_copies":17734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5739,"num_write_kb":22956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5004},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9035","reported_seq":9159,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:49.672401+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:49.672401+0000","last_peered":"2024-09-10T15:55:49.672401+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:49.672401+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9035,"log_dups_size":0,"ondisk_log_size":9035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36986880,"num_objects":9030,"num_object_clones":0,"num_object_copies":18060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7009,"num_write_kb":28036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5198}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8884","reported_seq":9010,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:49.673728+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:49.673728+0000","last_peered":"2024-09-10T15:55:49.673728+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:49.673728+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8884,"log_dups_size":0,"ondisk_log_size":8884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36364288,"num_objects":8878,"num_object_clones":0,"num_object_copies":17756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6753,"num_write_kb":27012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5049}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8993","reported_seq":9120,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.459718+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:48.459718+0000","last_peered":"2024-09-10T15:55:48.459718+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:48.459718+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8993,"log_dups_size":0,"ondisk_log_size":8993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36827136,"num_objects":8991,"num_object_clones":0,"num_object_copies":17982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6185,"num_write_kb":24740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5158},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8951","reported_seq":9114,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.421690+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:48.421690+0000","last_peered":"2024-09-10T15:55:48.421690+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:48.421690+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8951,"log_dups_size":0,"ondisk_log_size":8951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36630528,"num_objects":8943,"num_object_clones":0,"num_object_copies":17886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5369,"num_write_kb":21476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5026},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8969","reported_seq":9106,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.497084+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:48.497084+0000","last_peered":"2024-09-10T15:55:48.497084+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:48.497084+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8969,"log_dups_size":0,"ondisk_log_size":8969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36720640,"num_objects":8965,"num_object_clones":0,"num_object_copies":17930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5575,"num_write_kb":22300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5088},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9181","reported_seq":9319,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.459215+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:48.459215+0000","last_peered":"2024-09-10T15:55:48.459215+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:48.459215+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9181,"log_dups_size":0,"ondisk_log_size":9181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37556707,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6000,"num_write_kb":24000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5167},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":588022243,"num_objects":143561,"num_object_clones":0,"num_object_copies":287122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3374,"num_objects_unfound":0,"num_objects_dirty":143561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105163,"num_write_kb":420652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4490,"num_bytes_recovered":18382848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1194717184,"data_stored":1194634085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143642,"ondisk_log_size":143642,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626465,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464620,"kb_used_data":262236,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93907220,"statfs":{"total":96636764160,"available":96160993280,"internally_reserved":0,"allocated":268529664,"data_stored":268191351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207220398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149748,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798440,"kb_used_data":491928,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93573400,"statfs":{"total":96636764160,"available":95819161600,"internally_reserved":0,"allocated":503734272,"data_stored":503468219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313847469},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,1,2,8,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678300,"kb_used_data":412044,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93693540,"statfs":{"total":96636764160,"available":95942184960,"internally_reserved":0,"allocated":421933056,"data_stored":421667003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":504242176,"data_stored":504238563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":268288000,"data_stored":268212127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":422187008,"data_stored":422183395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:51.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:55:51.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'9002", 'reported_seq': 9078, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:48.509149+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:55:48.509149+0000', 'last_peered': '2024-09-10T15:55:48.509149+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:55:48.509149+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 9002, 'log_dups_size': 0, 'ondisk_log_size': 9002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36872192, 'num_objects': 9002, 'num_object_clones': 0, 'num_object_copies': 18004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9002, 'num_write_kb': 36008, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5029}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:51.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:55:51.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'8921", 'reported_seq': 9044, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:49.625840+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:55:49.625840+0000', 'last_peered': '2024-09-10T15:55:49.625840+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:55:49.625840+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 8921, 'log_dups_size': 0, 'ondisk_log_size': 8921, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36524032, 'num_objects': 8917, 'num_object_clones': 0, 'num_object_copies': 17834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8917, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7189, 'num_write_kb': 28756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 5115}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:51.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:55:51.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'9061", 'reported_seq': 10181, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:55:48.458655+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:55:48.458655+0000', 'last_peered': '2024-09-10T15:55:48.458655+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:55:48.458655+0000', 'last_undegraded': '2024-09-10T15:55:48.458655+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 9061, 'log_dups_size': 0, 'ondisk_log_size': 9061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37093376, 'num_objects': 9056, 'num_object_clones': 0, 'num_object_copies': 18112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3374, 'num_objects_unfound': 0, 'num_objects_dirty': 9056, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5779, 'num_write_kb': 23116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 494, 'num_bytes_recovered': 2023424, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5682}, {'shards': '0,2', 'objects': 3374}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:51.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:55:51.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'8759", 'reported_seq': 8899, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:48.412767+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:55:48.412767+0000', 'last_peered': '2024-09-10T15:55:48.412767+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:55:48.412767+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 8759, 'log_dups_size': 0, 'ondisk_log_size': 8759, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35852288, 'num_objects': 8753, 'num_object_clones': 0, 'num_object_copies': 17506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5409, 'num_write_kb': 21636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4919}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:51.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:55:51.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'8875", 'reported_seq': 8998, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:48.388862+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:55:48.388862+0000', 'last_peered': '2024-09-10T15:55:48.388862+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:55:48.388862+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 8875, 'log_dups_size': 0, 'ondisk_log_size': 8875, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36319232, 'num_objects': 8867, 'num_object_clones': 0, 'num_object_copies': 17734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8867, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5739, 'num_write_kb': 22956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5004}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:51.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:55:51.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'9035", 'reported_seq': 9159, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:49.672401+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:55:49.672401+0000', 'last_peered': '2024-09-10T15:55:49.672401+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:55:49.672401+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 9035, 'log_dups_size': 0, 'ondisk_log_size': 9035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36986880, 'num_objects': 9030, 'num_object_clones': 0, 'num_object_copies': 18060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9030, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7009, 'num_write_kb': 28036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 5198}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:51.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:55:51.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'8884", 'reported_seq': 9010, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:49.673728+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:55:49.673728+0000', 'last_peered': '2024-09-10T15:55:49.673728+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:55:49.673728+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 8884, 'log_dups_size': 0, 'ondisk_log_size': 8884, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36364288, 'num_objects': 8878, 'num_object_clones': 0, 'num_object_copies': 17756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8878, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6753, 'num_write_kb': 27012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 5049}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:51.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:55:51.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'8993", 'reported_seq': 9120, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:48.459718+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:55:48.459718+0000', 'last_peered': '2024-09-10T15:55:48.459718+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:55:48.459718+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 8993, 'log_dups_size': 0, 'ondisk_log_size': 8993, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36827136, 'num_objects': 8991, 'num_object_clones': 0, 'num_object_copies': 17982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8991, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6185, 'num_write_kb': 24740, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5158}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:51.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:55:51.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'8951", 'reported_seq': 9114, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:48.421690+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:55:48.421690+0000', 'last_peered': '2024-09-10T15:55:48.421690+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:55:48.421690+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 8951, 'log_dups_size': 0, 'ondisk_log_size': 8951, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36630528, 'num_objects': 8943, 'num_object_clones': 0, 'num_object_copies': 17886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8943, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5369, 'num_write_kb': 21476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5026}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:51.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:55:51.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'8969", 'reported_seq': 9106, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:48.497084+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:55:48.497084+0000', 'last_peered': '2024-09-10T15:55:48.497084+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:55:48.497084+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 8969, 'log_dups_size': 0, 'ondisk_log_size': 8969, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36720640, 'num_objects': 8965, 'num_object_clones': 0, 'num_object_copies': 17930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8965, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5575, 'num_write_kb': 22300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5088}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:51.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:55:51.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'9181", 'reported_seq': 9319, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:48.459215+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:55:48.459215+0000', 'last_peered': '2024-09-10T15:55:48.459215+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:55:48.459215+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 9181, 'log_dups_size': 0, 'ondisk_log_size': 9181, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37556707, 'num_objects': 9170, 'num_object_clones': 0, 'num_object_copies': 18340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9170, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6000, 'num_write_kb': 24000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5167}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:51.017 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:55:51.726 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:51.726 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/287126 objects degraded (13.507%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":277,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":143563,"data_bytes":588022262,"bytes_used":1987952640,"bytes_avail":287922339840,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":287126,"degraded_ratio":0.13506613821109897,"misplaced_objects":3374,"misplaced_total":287126,"misplaced_ratio":0.011750938612316544,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":52209,"recovering_keys_per_sec":0,"num_objects_recovered":153,"num_bytes_recovered":626688,"num_keys_recovered":0,"write_bytes_sec":1985999,"read_op_per_sec":0,"write_op_per_sec":484},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:55:51.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:55:51.753 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:51.824 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 194 20 144892 144872 2.91663 1.70312 0.0604573 0.0267783 2024-09-10T15:55:52.283 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:52.285 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:52.304 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-10T15:55:50.483361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":588022262,"num_objects":143563,"num_object_clones":0,"num_object_copies":287126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3374,"num_objects_unfound":0,"num_objects_dirty":143563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105165,"num_write_kb":420654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4492,"num_bytes_recovered":18382867,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143645,"ondisk_log_size":143645,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1941360,"kb_used_data":1166208,"kb_used_omap":12,"kb_used_meta":775091,"kb_avail":281174160,"statfs":{"total":289910292480,"available":287922339840,"internally_reserved":0,"allocated":1194196992,"data_stored":1193326573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":793693189},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,1,3,6,13,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23838720,"num_objects":5820,"num_object_clones":0,"num_object_copies":11640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-153,"num_objects_unfound":0,"num_objects_dirty":5820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5820,"num_write_kb":23280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":626688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003387"},"pg_stats":[{"pgid":"2.f","version":"46'9002","reported_seq":9078,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.509149+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:48.509149+0000","last_peered":"2024-09-10T15:55:48.509149+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:48.509149+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9002,"log_dups_size":0,"ondisk_log_size":9002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36872192,"num_objects":9002,"num_object_clones":0,"num_object_copies":18004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9002,"num_write_kb":36008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5029},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9186","reported_seq":9293,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:49.633538+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:49.633538+0000","last_peered":"2024-09-10T15:55:49.633538+0000","last_clean":"2024-09-10T15:55:49.633538+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:49.633538+0000","last_undegraded":"2024-09-10T15:55:49.633538+0000","last_fullsized":"2024-09-10T15:55:49.633538+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9186,"log_dups_size":0,"ondisk_log_size":9186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37601280,"num_objects":9180,"num_object_clones":0,"num_object_copies":18360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":30088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8921","reported_seq":9044,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:49.625840+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:49.625840+0000","last_peered":"2024-09-10T15:55:49.625840+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:49.625840+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8921,"log_dups_size":0,"ondisk_log_size":8921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524032,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7189,"num_write_kb":28756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5115}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8943","reported_seq":9022,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:49.681452+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:49.681452+0000","last_peered":"2024-09-10T15:55:49.681452+0000","last_clean":"2024-09-10T15:55:49.681452+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:49.681452+0000","last_undegraded":"2024-09-10T15:55:49.681452+0000","last_fullsized":"2024-09-10T15:55:49.681452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8943,"log_dups_size":0,"ondisk_log_size":8943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36630528,"num_objects":8943,"num_object_clones":0,"num_object_copies":17886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8943,"num_write_kb":35772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8802","reported_seq":8928,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:48.440650+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:48.440650+0000","last_peered":"2024-09-10T15:55:48.440650+0000","last_clean":"2024-09-10T15:55:48.440650+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:48.440650+0000","last_undegraded":"2024-09-10T15:55:48.440650+0000","last_fullsized":"2024-09-10T15:55:48.440650+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8802,"log_dups_size":0,"ondisk_log_size":8802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36024320,"num_objects":8795,"num_object_clones":0,"num_object_copies":17590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5873,"num_write_kb":23492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9061","reported_seq":10181,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:48.458655+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:48.458655+0000","last_peered":"2024-09-10T15:55:48.458655+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:48.458655+0000","last_undegraded":"2024-09-10T15:55:48.458655+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9061,"log_dups_size":0,"ondisk_log_size":9061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37093376,"num_objects":9056,"num_object_clones":0,"num_object_copies":18112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3374,"num_objects_unfound":0,"num_objects_dirty":9056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5779,"num_write_kb":23116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":2023424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5682},{"shards":"0,2","objects":3374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8759","reported_seq":8899,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.412767+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:48.412767+0000","last_peered":"2024-09-10T15:55:48.412767+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:48.412767+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8759,"log_dups_size":0,"ondisk_log_size":8759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5409,"num_write_kb":21636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4919},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9049","reported_seq":9180,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:48.480595+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:48.480595+0000","last_peered":"2024-09-10T15:55:48.480595+0000","last_clean":"2024-09-10T15:55:48.480595+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:48.480595+0000","last_undegraded":"2024-09-10T15:55:48.480595+0000","last_fullsized":"2024-09-10T15:55:48.480595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9049,"log_dups_size":0,"ondisk_log_size":9049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37056512,"num_objects":9047,"num_object_clones":0,"num_object_copies":18094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5808,"num_write_kb":23232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9031","reported_seq":17157,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:48.816335+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:48.816335+0000","last_peered":"2024-09-10T15:55:48.816335+0000","last_clean":"2024-09-10T15:55:48.816335+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:48.816335+0000","last_undegraded":"2024-09-10T15:55:48.816335+0000","last_fullsized":"2024-09-10T15:55:48.816335+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9031,"log_dups_size":0,"ondisk_log_size":9031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36962304,"num_objects":9024,"num_object_clones":0,"num_object_copies":18048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7008,"num_write_kb":28032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8875","reported_seq":8998,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.388862+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:48.388862+0000","last_peered":"2024-09-10T15:55:48.388862+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:48.388862+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":8875,"log_dups_size":0,"ondisk_log_size":8875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36319232,"num_objects":8867,"num_object_clones":0,"num_object_copies":17734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5739,"num_write_kb":22956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5004},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9035","reported_seq":9159,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:49.672401+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:49.672401+0000","last_peered":"2024-09-10T15:55:49.672401+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:49.672401+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9035,"log_dups_size":0,"ondisk_log_size":9035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36986880,"num_objects":9030,"num_object_clones":0,"num_object_copies":18060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7009,"num_write_kb":28036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5198}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8884","reported_seq":9010,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:49.673728+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:49.673728+0000","last_peered":"2024-09-10T15:55:49.673728+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:49.673728+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8884,"log_dups_size":0,"ondisk_log_size":8884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36364288,"num_objects":8878,"num_object_clones":0,"num_object_copies":17756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6753,"num_write_kb":27012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5049}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8993","reported_seq":9120,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.459718+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:48.459718+0000","last_peered":"2024-09-10T15:55:48.459718+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:48.459718+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":8993,"log_dups_size":0,"ondisk_log_size":8993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36827136,"num_objects":8991,"num_object_clones":0,"num_object_copies":17982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6185,"num_write_kb":24740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5158},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8951","reported_seq":9114,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.421690+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:48.421690+0000","last_peered":"2024-09-10T15:55:48.421690+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:48.421690+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":8951,"log_dups_size":0,"ondisk_log_size":8951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36630528,"num_objects":8943,"num_object_clones":0,"num_object_copies":17886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5369,"num_write_kb":21476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5026},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8969","reported_seq":9106,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.497084+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:48.497084+0000","last_peered":"2024-09-10T15:55:48.497084+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:48.497084+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":8969,"log_dups_size":0,"ondisk_log_size":8969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36720640,"num_objects":8965,"num_object_clones":0,"num_object_copies":17930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5575,"num_write_kb":22300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5088},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9181","reported_seq":9319,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:48.459215+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:48.459215+0000","last_peered":"2024-09-10T15:55:48.459215+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:48.459215+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9181,"log_dups_size":0,"ondisk_log_size":9181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37556707,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6000,"num_write_kb":24000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5167},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":588022243,"num_objects":143561,"num_object_clones":0,"num_object_copies":287122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3374,"num_objects_unfound":0,"num_objects_dirty":143561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105163,"num_write_kb":420652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4490,"num_bytes_recovered":18382848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1194717184,"data_stored":1194634085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143642,"ondisk_log_size":143642,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626465,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464620,"kb_used_data":262236,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93907220,"statfs":{"total":96636764160,"available":96160993280,"internally_reserved":0,"allocated":268529664,"data_stored":268191351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207220398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149748,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798440,"kb_used_data":491928,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93573400,"statfs":{"total":96636764160,"available":95819161600,"internally_reserved":0,"allocated":503734272,"data_stored":503468219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313847469},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,1,2,8,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678300,"kb_used_data":412044,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93693540,"statfs":{"total":96636764160,"available":95942184960,"internally_reserved":0,"allocated":421933056,"data_stored":421667003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":504242176,"data_stored":504238563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":268288000,"data_stored":268212127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":422187008,"data_stored":422183395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:52.824 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 195 20 145351 145331 2.91087 1.79297 0.0666794 0.0268312 2024-09-10T15:55:53.824 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 196 20 145852 145832 2.906 1.95703 0.0235164 0.0268762 2024-09-10T15:55:54.824 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 197 20 146340 146320 2.90093 1.90625 0.0423653 0.0269234 2024-09-10T15:55:55.306 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:55.824 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 198 20 146815 146795 2.89565 1.85547 0.0402212 0.0269731 2024-09-10T15:55:55.856 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:55.857 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:55.881 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-10T15:55:54.485006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594514422,"num_objects":145148,"num_object_clones":0,"num_object_copies":290296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":145148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106750,"num_write_kb":426994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4572,"num_bytes_recovered":18710547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145230,"ondisk_log_size":145230,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1945916,"kb_used_data":1179084,"kb_used_omap":12,"kb_used_meta":766771,"kb_avail":281169604,"statfs":{"total":289910292480,"available":287917674496,"internally_reserved":0,"allocated":1207382016,"data_stored":1206511597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":785173509},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,8,5,8,12,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26595328,"num_objects":6493,"num_object_clones":0,"num_object_copies":12986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-233,"num_objects_unfound":0,"num_objects_dirty":6493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6493,"num_write_kb":25972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":954368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004071"},"pg_stats":[{"pgid":"2.f","version":"46'9133","reported_seq":9209,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.478527+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:53.478527+0000","last_peered":"2024-09-10T15:55:53.478527+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:53.478527+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9133,"log_dups_size":0,"ondisk_log_size":9133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37408768,"num_objects":9133,"num_object_clones":0,"num_object_copies":18266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9133,"num_write_kb":36532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5160},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9186","reported_seq":9293,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:49.633538+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:49.633538+0000","last_peered":"2024-09-10T15:55:49.633538+0000","last_clean":"2024-09-10T15:55:49.633538+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:49.633538+0000","last_undegraded":"2024-09-10T15:55:49.633538+0000","last_fullsized":"2024-09-10T15:55:49.633538+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9186,"log_dups_size":0,"ondisk_log_size":9186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37601280,"num_objects":9180,"num_object_clones":0,"num_object_copies":18360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":30088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8921","reported_seq":9044,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:49.625840+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:49.625840+0000","last_peered":"2024-09-10T15:55:49.625840+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:49.625840+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8921,"log_dups_size":0,"ondisk_log_size":8921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524032,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7189,"num_write_kb":28756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5115}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8943","reported_seq":9022,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:49.681452+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:49.681452+0000","last_peered":"2024-09-10T15:55:49.681452+0000","last_clean":"2024-09-10T15:55:49.681452+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:49.681452+0000","last_undegraded":"2024-09-10T15:55:49.681452+0000","last_fullsized":"2024-09-10T15:55:49.681452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8943,"log_dups_size":0,"ondisk_log_size":8943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36630528,"num_objects":8943,"num_object_clones":0,"num_object_copies":17886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8943,"num_write_kb":35772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8949","reported_seq":9075,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:53.528183+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:53.528183+0000","last_peered":"2024-09-10T15:55:53.528183+0000","last_clean":"2024-09-10T15:55:53.528183+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:53.528183+0000","last_undegraded":"2024-09-10T15:55:53.528183+0000","last_fullsized":"2024-09-10T15:55:53.528183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8949,"log_dups_size":0,"ondisk_log_size":8949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36626432,"num_objects":8942,"num_object_clones":0,"num_object_copies":17884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6020,"num_write_kb":24080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9192","reported_seq":10475,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:53.557387+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:53.557387+0000","last_peered":"2024-09-10T15:55:53.557387+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:53.557387+0000","last_undegraded":"2024-09-10T15:55:53.557387+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9192,"log_dups_size":0,"ondisk_log_size":9192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629952,"num_objects":9187,"num_object_clones":0,"num_object_copies":18374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":9187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5910,"num_write_kb":23640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":574,"num_bytes_recovered":2351104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5893},{"shards":"0,2","objects":3294}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8901","reported_seq":9041,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.514035+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:53.514035+0000","last_peered":"2024-09-10T15:55:53.514035+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:53.514035+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8901,"log_dups_size":0,"ondisk_log_size":8901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5551,"num_write_kb":22204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5061},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9204","reported_seq":9335,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:53.495011+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:53.495011+0000","last_peered":"2024-09-10T15:55:53.495011+0000","last_clean":"2024-09-10T15:55:53.495011+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:53.495011+0000","last_undegraded":"2024-09-10T15:55:53.495011+0000","last_fullsized":"2024-09-10T15:55:53.495011+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9204,"log_dups_size":0,"ondisk_log_size":9204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37691392,"num_objects":9202,"num_object_clones":0,"num_object_copies":18404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5963,"num_write_kb":23852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9156","reported_seq":17283,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:53.789352+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:53.789352+0000","last_peered":"2024-09-10T15:55:53.789352+0000","last_clean":"2024-09-10T15:55:53.789352+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:53.789352+0000","last_undegraded":"2024-09-10T15:55:53.789352+0000","last_fullsized":"2024-09-10T15:55:53.789352+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9156,"log_dups_size":0,"ondisk_log_size":9156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37474304,"num_objects":9149,"num_object_clones":0,"num_object_copies":18298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7133,"num_write_kb":28532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9018","reported_seq":9144,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.524782+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:53.524782+0000","last_peered":"2024-09-10T15:55:53.524782+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:53.524782+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9018,"log_dups_size":0,"ondisk_log_size":9018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36904960,"num_objects":9010,"num_object_clones":0,"num_object_copies":18020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5882,"num_write_kb":23528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5147},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9035","reported_seq":9159,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:49.672401+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:49.672401+0000","last_peered":"2024-09-10T15:55:49.672401+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:49.672401+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9035,"log_dups_size":0,"ondisk_log_size":9035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36986880,"num_objects":9030,"num_object_clones":0,"num_object_copies":18060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7009,"num_write_kb":28036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5198}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8884","reported_seq":9010,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:49.673728+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:49.673728+0000","last_peered":"2024-09-10T15:55:49.673728+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:49.673728+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8884,"log_dups_size":0,"ondisk_log_size":8884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36364288,"num_objects":8878,"num_object_clones":0,"num_object_copies":17756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6753,"num_write_kb":27012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5049}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9131","reported_seq":9258,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.524722+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:53.524722+0000","last_peered":"2024-09-10T15:55:53.524722+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:53.524722+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9131,"log_dups_size":0,"ondisk_log_size":9131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392384,"num_objects":9129,"num_object_clones":0,"num_object_copies":18258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6323,"num_write_kb":25292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5296},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9126","reported_seq":9283,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.497410+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:53.497410+0000","last_peered":"2024-09-10T15:55:53.497410+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:53.497410+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9126,"log_dups_size":0,"ondisk_log_size":9126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37347328,"num_objects":9118,"num_object_clones":0,"num_object_copies":18236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5544,"num_write_kb":22176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5201},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9125","reported_seq":9264,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.337122+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:53.337122+0000","last_peered":"2024-09-10T15:55:53.337122+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:53.337122+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9125,"log_dups_size":0,"ondisk_log_size":9125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37359616,"num_objects":9121,"num_object_clones":0,"num_object_copies":18242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5731,"num_write_kb":22924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5244},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9323","reported_seq":9461,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.544770+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:53.544770+0000","last_peered":"2024-09-10T15:55:53.544770+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:53.544770+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9323,"log_dups_size":0,"ondisk_log_size":9323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38138339,"num_objects":9312,"num_object_clones":0,"num_object_copies":18624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6142,"num_write_kb":24568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5309},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":594514403,"num_objects":145146,"num_object_clones":0,"num_object_copies":290292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":145146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106748,"num_write_kb":426992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4570,"num_bytes_recovered":18710528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1209327616,"data_stored":1209244517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145227,"ondisk_log_size":145227,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626466,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458036,"kb_used_data":268132,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93913804,"statfs":{"total":96636764160,"available":96167735296,"internally_reserved":0,"allocated":274567168,"data_stored":274228855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":194440878},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149749,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809580,"kb_used_data":498908,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93562260,"statfs":{"total":96636764160,"available":95807754240,"internally_reserved":0,"allocated":510881792,"data_stored":510615739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318107309},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2,6,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678300,"kb_used_data":412044,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93693540,"statfs":{"total":96636764160,"available":95942184960,"internally_reserved":0,"allocated":421933056,"data_stored":421667003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":511344640,"data_stored":511341027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275795968,"data_stored":275720095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":422187008,"data_stored":422183395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:55.882 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:56.415 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:56.416 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:56.441 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-10T15:55:54.485006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594514422,"num_objects":145148,"num_object_clones":0,"num_object_copies":290296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":145148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106750,"num_write_kb":426994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4572,"num_bytes_recovered":18710547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145230,"ondisk_log_size":145230,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1945916,"kb_used_data":1179084,"kb_used_omap":12,"kb_used_meta":766771,"kb_avail":281169604,"statfs":{"total":289910292480,"available":287917674496,"internally_reserved":0,"allocated":1207382016,"data_stored":1206511597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":785173509},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,8,5,8,12,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26595328,"num_objects":6493,"num_object_clones":0,"num_object_copies":12986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-233,"num_objects_unfound":0,"num_objects_dirty":6493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6493,"num_write_kb":25972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":954368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004071"},"pg_stats":[{"pgid":"2.f","version":"46'9133","reported_seq":9209,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.478527+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:53.478527+0000","last_peered":"2024-09-10T15:55:53.478527+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:53.478527+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9133,"log_dups_size":0,"ondisk_log_size":9133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37408768,"num_objects":9133,"num_object_clones":0,"num_object_copies":18266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9133,"num_write_kb":36532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5160},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9186","reported_seq":9293,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:49.633538+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:49.633538+0000","last_peered":"2024-09-10T15:55:49.633538+0000","last_clean":"2024-09-10T15:55:49.633538+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:49.633538+0000","last_undegraded":"2024-09-10T15:55:49.633538+0000","last_fullsized":"2024-09-10T15:55:49.633538+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9186,"log_dups_size":0,"ondisk_log_size":9186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37601280,"num_objects":9180,"num_object_clones":0,"num_object_copies":18360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7522,"num_write_kb":30088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8921","reported_seq":9044,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:49.625840+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:49.625840+0000","last_peered":"2024-09-10T15:55:49.625840+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:49.625840+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":8921,"log_dups_size":0,"ondisk_log_size":8921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524032,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7189,"num_write_kb":28756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5115}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'8943","reported_seq":9022,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:49.681452+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:49.681452+0000","last_peered":"2024-09-10T15:55:49.681452+0000","last_clean":"2024-09-10T15:55:49.681452+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:49.681452+0000","last_undegraded":"2024-09-10T15:55:49.681452+0000","last_fullsized":"2024-09-10T15:55:49.681452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":8943,"log_dups_size":0,"ondisk_log_size":8943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36630528,"num_objects":8943,"num_object_clones":0,"num_object_copies":17886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8943,"num_write_kb":35772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8949","reported_seq":9075,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:53.528183+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:53.528183+0000","last_peered":"2024-09-10T15:55:53.528183+0000","last_clean":"2024-09-10T15:55:53.528183+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:53.528183+0000","last_undegraded":"2024-09-10T15:55:53.528183+0000","last_fullsized":"2024-09-10T15:55:53.528183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8949,"log_dups_size":0,"ondisk_log_size":8949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36626432,"num_objects":8942,"num_object_clones":0,"num_object_copies":17884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6020,"num_write_kb":24080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9192","reported_seq":10475,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:53.557387+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:53.557387+0000","last_peered":"2024-09-10T15:55:53.557387+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:53.557387+0000","last_undegraded":"2024-09-10T15:55:53.557387+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9192,"log_dups_size":0,"ondisk_log_size":9192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629952,"num_objects":9187,"num_object_clones":0,"num_object_copies":18374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":9187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5910,"num_write_kb":23640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":574,"num_bytes_recovered":2351104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5893},{"shards":"0,2","objects":3294}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8901","reported_seq":9041,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.514035+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:53.514035+0000","last_peered":"2024-09-10T15:55:53.514035+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:53.514035+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8901,"log_dups_size":0,"ondisk_log_size":8901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5551,"num_write_kb":22204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5061},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9204","reported_seq":9335,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:53.495011+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:53.495011+0000","last_peered":"2024-09-10T15:55:53.495011+0000","last_clean":"2024-09-10T15:55:53.495011+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:53.495011+0000","last_undegraded":"2024-09-10T15:55:53.495011+0000","last_fullsized":"2024-09-10T15:55:53.495011+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9204,"log_dups_size":0,"ondisk_log_size":9204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37691392,"num_objects":9202,"num_object_clones":0,"num_object_copies":18404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5963,"num_write_kb":23852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9156","reported_seq":17283,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:53.789352+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:53.789352+0000","last_peered":"2024-09-10T15:55:53.789352+0000","last_clean":"2024-09-10T15:55:53.789352+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:53.789352+0000","last_undegraded":"2024-09-10T15:55:53.789352+0000","last_fullsized":"2024-09-10T15:55:53.789352+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9156,"log_dups_size":0,"ondisk_log_size":9156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37474304,"num_objects":9149,"num_object_clones":0,"num_object_copies":18298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7133,"num_write_kb":28532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9018","reported_seq":9144,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.524782+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:53.524782+0000","last_peered":"2024-09-10T15:55:53.524782+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:53.524782+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9018,"log_dups_size":0,"ondisk_log_size":9018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36904960,"num_objects":9010,"num_object_clones":0,"num_object_copies":18020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5882,"num_write_kb":23528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5147},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9035","reported_seq":9159,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:49.672401+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:49.672401+0000","last_peered":"2024-09-10T15:55:49.672401+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:49.672401+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9035,"log_dups_size":0,"ondisk_log_size":9035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36986880,"num_objects":9030,"num_object_clones":0,"num_object_copies":18060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7009,"num_write_kb":28036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5198}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8884","reported_seq":9010,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:49.673728+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:49.673728+0000","last_peered":"2024-09-10T15:55:49.673728+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:49.673728+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":8884,"log_dups_size":0,"ondisk_log_size":8884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36364288,"num_objects":8878,"num_object_clones":0,"num_object_copies":17756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6753,"num_write_kb":27012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5049}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9131","reported_seq":9258,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.524722+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:53.524722+0000","last_peered":"2024-09-10T15:55:53.524722+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:53.524722+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9131,"log_dups_size":0,"ondisk_log_size":9131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392384,"num_objects":9129,"num_object_clones":0,"num_object_copies":18258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6323,"num_write_kb":25292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5296},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9126","reported_seq":9283,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.497410+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:53.497410+0000","last_peered":"2024-09-10T15:55:53.497410+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:53.497410+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9126,"log_dups_size":0,"ondisk_log_size":9126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37347328,"num_objects":9118,"num_object_clones":0,"num_object_copies":18236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5544,"num_write_kb":22176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5201},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9125","reported_seq":9264,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.337122+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:53.337122+0000","last_peered":"2024-09-10T15:55:53.337122+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:53.337122+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9125,"log_dups_size":0,"ondisk_log_size":9125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37359616,"num_objects":9121,"num_object_clones":0,"num_object_copies":18242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5731,"num_write_kb":22924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5244},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9323","reported_seq":9461,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.544770+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:53.544770+0000","last_peered":"2024-09-10T15:55:53.544770+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:53.544770+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9323,"log_dups_size":0,"ondisk_log_size":9323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38138339,"num_objects":9312,"num_object_clones":0,"num_object_copies":18624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6142,"num_write_kb":24568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5309},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":594514403,"num_objects":145146,"num_object_clones":0,"num_object_copies":290292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":145146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106748,"num_write_kb":426992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4570,"num_bytes_recovered":18710528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1209327616,"data_stored":1209244517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145227,"ondisk_log_size":145227,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626466,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458036,"kb_used_data":268132,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93913804,"statfs":{"total":96636764160,"available":96167735296,"internally_reserved":0,"allocated":274567168,"data_stored":274228855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":194440878},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149749,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809580,"kb_used_data":498908,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93562260,"statfs":{"total":96636764160,"available":95807754240,"internally_reserved":0,"allocated":510881792,"data_stored":510615739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318107309},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2,6,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678300,"kb_used_data":412044,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93693540,"statfs":{"total":96636764160,"available":95942184960,"internally_reserved":0,"allocated":421933056,"data_stored":421667003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":511344640,"data_stored":511341027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275795968,"data_stored":275720095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":422187008,"data_stored":422183395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:56.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:55:56.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'9133", 'reported_seq': 9209, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:53.478527+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:55:53.478527+0000', 'last_peered': '2024-09-10T15:55:53.478527+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:55:53.478527+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 9133, 'log_dups_size': 0, 'ondisk_log_size': 9133, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37408768, 'num_objects': 9133, 'num_object_clones': 0, 'num_object_copies': 18266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9133, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9133, 'num_write_kb': 36532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5160}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:56.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:55:56.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'8921", 'reported_seq': 9044, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:49.625840+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:55:49.625840+0000', 'last_peered': '2024-09-10T15:55:49.625840+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:55:49.625840+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 8921, 'log_dups_size': 0, 'ondisk_log_size': 8921, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36524032, 'num_objects': 8917, 'num_object_clones': 0, 'num_object_copies': 17834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8917, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7189, 'num_write_kb': 28756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 5115}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:56.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:55:56.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'9192", 'reported_seq': 10475, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:55:53.557387+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:55:53.557387+0000', 'last_peered': '2024-09-10T15:55:53.557387+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:55:53.557387+0000', 'last_undegraded': '2024-09-10T15:55:53.557387+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 9192, 'log_dups_size': 0, 'ondisk_log_size': 9192, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37629952, 'num_objects': 9187, 'num_object_clones': 0, 'num_object_copies': 18374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3294, 'num_objects_unfound': 0, 'num_objects_dirty': 9187, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5910, 'num_write_kb': 23640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 574, 'num_bytes_recovered': 2351104, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5893}, {'shards': '0,2', 'objects': 3294}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:56.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:55:56.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'8901", 'reported_seq': 9041, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:53.514035+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:55:53.514035+0000', 'last_peered': '2024-09-10T15:55:53.514035+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:55:53.514035+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 8901, 'log_dups_size': 0, 'ondisk_log_size': 8901, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36433920, 'num_objects': 8895, 'num_object_clones': 0, 'num_object_copies': 17790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8895, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5551, 'num_write_kb': 22204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5061}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:56.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:55:56.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'9018", 'reported_seq': 9144, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:53.524782+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:55:53.524782+0000', 'last_peered': '2024-09-10T15:55:53.524782+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:55:53.524782+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 9018, 'log_dups_size': 0, 'ondisk_log_size': 9018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36904960, 'num_objects': 9010, 'num_object_clones': 0, 'num_object_copies': 18020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9010, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5882, 'num_write_kb': 23528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5147}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:56.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:55:56.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'9035", 'reported_seq': 9159, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:49.672401+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:55:49.672401+0000', 'last_peered': '2024-09-10T15:55:49.672401+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:55:49.672401+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 9035, 'log_dups_size': 0, 'ondisk_log_size': 9035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36986880, 'num_objects': 9030, 'num_object_clones': 0, 'num_object_copies': 18060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9030, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7009, 'num_write_kb': 28036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 5198}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:56.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:55:56.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'8884", 'reported_seq': 9010, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:49.673728+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:55:49.673728+0000', 'last_peered': '2024-09-10T15:55:49.673728+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:55:49.673728+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 8884, 'log_dups_size': 0, 'ondisk_log_size': 8884, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36364288, 'num_objects': 8878, 'num_object_clones': 0, 'num_object_copies': 17756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8878, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6753, 'num_write_kb': 27012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 5049}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:55:56.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:55:56.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'9131", 'reported_seq': 9258, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:53.524722+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:55:53.524722+0000', 'last_peered': '2024-09-10T15:55:53.524722+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:55:53.524722+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 9131, 'log_dups_size': 0, 'ondisk_log_size': 9131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37392384, 'num_objects': 9129, 'num_object_clones': 0, 'num_object_copies': 18258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9129, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6323, 'num_write_kb': 25292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5296}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:56.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:55:56.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'9126", 'reported_seq': 9283, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:53.497410+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:55:53.497410+0000', 'last_peered': '2024-09-10T15:55:53.497410+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:55:53.497410+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 9126, 'log_dups_size': 0, 'ondisk_log_size': 9126, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37347328, 'num_objects': 9118, 'num_object_clones': 0, 'num_object_copies': 18236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9118, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5544, 'num_write_kb': 22176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5201}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:56.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:55:56.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'9125", 'reported_seq': 9264, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:53.337122+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:55:53.337122+0000', 'last_peered': '2024-09-10T15:55:53.337122+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:55:53.337122+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 9125, 'log_dups_size': 0, 'ondisk_log_size': 9125, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37359616, 'num_objects': 9121, 'num_object_clones': 0, 'num_object_copies': 18242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9121, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5731, 'num_write_kb': 22924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5244}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:56.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:55:56.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'9323", 'reported_seq': 9461, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:53.544770+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:55:53.544770+0000', 'last_peered': '2024-09-10T15:55:53.544770+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:55:53.544770+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 9323, 'log_dups_size': 0, 'ondisk_log_size': 9323, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38138339, 'num_objects': 9312, 'num_object_clones': 0, 'num_object_copies': 18624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9312, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6142, 'num_write_kb': 24568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5309}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:55:56.445 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:55:56.824 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 199 20 147236 147216 2.88936 1.64453 0.0209295 0.027031 2024-09-10T15:55:57.173 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:57.192 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/291842 objects degraded (13.288%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":283,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":145921,"data_bytes":597680630,"bytes_used":2001600512,"bytes_avail":287908691968,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":291842,"degraded_ratio":0.13288354657657225,"misplaced_objects":3294,"misplaced_total":291842,"misplaced_ratio":0.011286929228829299,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":52545,"recovering_keys_per_sec":0,"num_objects_recovered":154,"num_bytes_recovered":630784,"num_keys_recovered":0,"write_bytes_sec":1901885,"read_op_per_sec":0,"write_op_per_sec":464},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:55:57.198 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:55:57.198 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:55:57.755 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:55:57.756 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:55:57.786 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-10T15:55:56.485861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597680630,"num_objects":145921,"num_object_clones":0,"num_object_copies":291842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":145921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107523,"num_write_kb":430086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4572,"num_bytes_recovered":18710547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146003,"ondisk_log_size":146003,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1954688,"kb_used_data":1183696,"kb_used_omap":12,"kb_used_meta":770931,"kb_avail":281160832,"statfs":{"total":289910292480,"available":287908691968,"internally_reserved":0,"allocated":1212104704,"data_stored":1211234285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":789433349},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,10,5,8,11,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22831104,"num_objects":5574,"num_object_clones":0,"num_object_copies":11148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-154,"num_objects_unfound":0,"num_objects_dirty":5574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5574,"num_write_kb":22296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":630784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004457"},"pg_stats":[{"pgid":"2.f","version":"46'9133","reported_seq":9209,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.478527+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:53.478527+0000","last_peered":"2024-09-10T15:55:53.478527+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:53.478527+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9133,"log_dups_size":0,"ondisk_log_size":9133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37408768,"num_objects":9133,"num_object_clones":0,"num_object_copies":18266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9133,"num_write_kb":36532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5160},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9330","reported_seq":9437,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:54.632857+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:54.632857+0000","last_peered":"2024-09-10T15:55:54.632857+0000","last_clean":"2024-09-10T15:55:54.632857+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:54.632857+0000","last_undegraded":"2024-09-10T15:55:54.632857+0000","last_fullsized":"2024-09-10T15:55:54.632857+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9330,"log_dups_size":0,"ondisk_log_size":9330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38191104,"num_objects":9324,"num_object_clones":0,"num_object_copies":18648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7666,"num_write_kb":30664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9099","reported_seq":9222,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:54.701417+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:54.701417+0000","last_peered":"2024-09-10T15:55:54.701417+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:54.701417+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9099,"log_dups_size":0,"ondisk_log_size":9099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37253120,"num_objects":9095,"num_object_clones":0,"num_object_copies":18190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7367,"num_write_kb":29468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5293}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9084","reported_seq":9163,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:54.637088+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:54.637088+0000","last_peered":"2024-09-10T15:55:54.637088+0000","last_clean":"2024-09-10T15:55:54.637088+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:54.637088+0000","last_undegraded":"2024-09-10T15:55:54.637088+0000","last_fullsized":"2024-09-10T15:55:54.637088+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9084,"log_dups_size":0,"ondisk_log_size":9084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37208064,"num_objects":9084,"num_object_clones":0,"num_object_copies":18168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9084,"num_write_kb":36336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8949","reported_seq":9075,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:53.528183+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:53.528183+0000","last_peered":"2024-09-10T15:55:53.528183+0000","last_clean":"2024-09-10T15:55:53.528183+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:53.528183+0000","last_undegraded":"2024-09-10T15:55:53.528183+0000","last_fullsized":"2024-09-10T15:55:53.528183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":8949,"log_dups_size":0,"ondisk_log_size":8949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36626432,"num_objects":8942,"num_object_clones":0,"num_object_copies":17884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6020,"num_write_kb":24080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9192","reported_seq":10475,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:53.557387+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:53.557387+0000","last_peered":"2024-09-10T15:55:53.557387+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:53.557387+0000","last_undegraded":"2024-09-10T15:55:53.557387+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9192,"log_dups_size":0,"ondisk_log_size":9192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629952,"num_objects":9187,"num_object_clones":0,"num_object_copies":18374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":9187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5910,"num_write_kb":23640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":574,"num_bytes_recovered":2351104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5893},{"shards":"0,2","objects":3294}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8901","reported_seq":9041,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.514035+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:53.514035+0000","last_peered":"2024-09-10T15:55:53.514035+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:53.514035+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":8901,"log_dups_size":0,"ondisk_log_size":8901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5551,"num_write_kb":22204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5061},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9204","reported_seq":9335,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:53.495011+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:53.495011+0000","last_peered":"2024-09-10T15:55:53.495011+0000","last_clean":"2024-09-10T15:55:53.495011+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:53.495011+0000","last_undegraded":"2024-09-10T15:55:53.495011+0000","last_fullsized":"2024-09-10T15:55:53.495011+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9204,"log_dups_size":0,"ondisk_log_size":9204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37691392,"num_objects":9202,"num_object_clones":0,"num_object_copies":18404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5963,"num_write_kb":23852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9156","reported_seq":17283,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:53.789352+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:53.789352+0000","last_peered":"2024-09-10T15:55:53.789352+0000","last_clean":"2024-09-10T15:55:53.789352+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:53.789352+0000","last_undegraded":"2024-09-10T15:55:53.789352+0000","last_fullsized":"2024-09-10T15:55:53.789352+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9156,"log_dups_size":0,"ondisk_log_size":9156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37474304,"num_objects":9149,"num_object_clones":0,"num_object_copies":18298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7133,"num_write_kb":28532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9018","reported_seq":9144,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.524782+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:53.524782+0000","last_peered":"2024-09-10T15:55:53.524782+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:53.524782+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9018,"log_dups_size":0,"ondisk_log_size":9018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36904960,"num_objects":9010,"num_object_clones":0,"num_object_copies":18020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5882,"num_write_kb":23528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5147},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9189","reported_seq":9313,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:54.606534+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:54.606534+0000","last_peered":"2024-09-10T15:55:54.606534+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:54.606534+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9189,"log_dups_size":0,"ondisk_log_size":9189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37617664,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7163,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5352}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9040","reported_seq":9166,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:54.683701+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:54.683701+0000","last_peered":"2024-09-10T15:55:54.683701+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:54.683701+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9040,"log_dups_size":0,"ondisk_log_size":9040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37003264,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6909,"num_write_kb":27636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5205}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9131","reported_seq":9258,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.524722+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:53.524722+0000","last_peered":"2024-09-10T15:55:53.524722+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:53.524722+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9131,"log_dups_size":0,"ondisk_log_size":9131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392384,"num_objects":9129,"num_object_clones":0,"num_object_copies":18258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6323,"num_write_kb":25292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5296},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9126","reported_seq":9283,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.497410+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:53.497410+0000","last_peered":"2024-09-10T15:55:53.497410+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:53.497410+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9126,"log_dups_size":0,"ondisk_log_size":9126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37347328,"num_objects":9118,"num_object_clones":0,"num_object_copies":18236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5544,"num_write_kb":22176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5201},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9125","reported_seq":9264,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.337122+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:53.337122+0000","last_peered":"2024-09-10T15:55:53.337122+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:53.337122+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9125,"log_dups_size":0,"ondisk_log_size":9125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37359616,"num_objects":9121,"num_object_clones":0,"num_object_copies":18242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5731,"num_write_kb":22924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5244},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9323","reported_seq":9461,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:53.544770+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:53.544770+0000","last_peered":"2024-09-10T15:55:53.544770+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:53.544770+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9323,"log_dups_size":0,"ondisk_log_size":9323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38138339,"num_objects":9312,"num_object_clones":0,"num_object_copies":18624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6142,"num_write_kb":24568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5309},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":597680611,"num_objects":145919,"num_object_clones":0,"num_object_copies":291838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":145919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107521,"num_write_kb":430084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4570,"num_bytes_recovered":18710528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1214205952,"data_stored":1214122853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146000,"ondisk_log_size":146000,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626466,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458036,"kb_used_data":268132,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93913804,"statfs":{"total":96636764160,"available":96167735296,"internally_reserved":0,"allocated":274567168,"data_stored":274228855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":194440878},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149749,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809580,"kb_used_data":498908,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93562260,"statfs":{"total":96636764160,"available":95807754240,"internally_reserved":0,"allocated":510881792,"data_stored":510615739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318107309},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2,6,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687072,"kb_used_data":416656,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93684768,"statfs":{"total":96636764160,"available":95933202432,"internally_reserved":0,"allocated":426655744,"data_stored":426389691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":511344640,"data_stored":511341027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275795968,"data_stored":275720095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":427065344,"data_stored":427061731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:55:57.824 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:2024-09-10T15:55:57.823926+0000 min lat: 0.00198913 max lat: 2.66872 avg lat: 0.0270868 2024-09-10T15:55:57.824 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:55:57.824 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 200 20 147678 147658 2.88354 1.72656 0.144215 0.0270868 2024-09-10T15:55:58.824 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 201 20 148164 148144 2.87864 1.89844 0.00645975 0.0271275 2024-09-10T15:55:59.827 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 202 19 148571 148552 2.87224 1.59375 0.0297772 0.0271938 2024-09-10T15:56:00.788 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:00.827 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 203 20 149060 149040 2.86748 1.90625 0.0567134 0.0272366 2024-09-10T15:56:01.345 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:01.351 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:01.374 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-10T15:56:00.487324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":607089142,"num_objects":148218,"num_object_clones":0,"num_object_copies":296436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3213,"num_objects_unfound":0,"num_objects_dirty":148218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109820,"num_write_kb":439274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4653,"num_bytes_recovered":19042323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148300,"ondisk_log_size":148300,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1987476,"kb_used_data":1202596,"kb_used_omap":12,"kb_used_meta":784819,"kb_avail":281128044,"statfs":{"total":289910292480,"available":287875117056,"internally_reserved":0,"allocated":1231458304,"data_stored":1230587885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":803654661},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,10,16,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29036544,"num_objects":7089,"num_object_clones":0,"num_object_copies":14178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-235,"num_objects_unfound":0,"num_objects_dirty":7089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7089,"num_write_kb":28356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":962560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004581"},"pg_stats":[{"pgid":"2.f","version":"46'9268","reported_seq":9344,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.596755+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:58.596755+0000","last_peered":"2024-09-10T15:55:58.596755+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:58.596755+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9268,"log_dups_size":0,"ondisk_log_size":9268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37961728,"num_objects":9268,"num_object_clones":0,"num_object_copies":18536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9268,"num_write_kb":37072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5295},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9457","reported_seq":9564,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:59.721536+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:59.721536+0000","last_peered":"2024-09-10T15:55:59.721536+0000","last_clean":"2024-09-10T15:55:59.721536+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:59.721536+0000","last_undegraded":"2024-09-10T15:55:59.721536+0000","last_fullsized":"2024-09-10T15:55:59.721536+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9457,"log_dups_size":0,"ondisk_log_size":9457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38711296,"num_objects":9451,"num_object_clones":0,"num_object_copies":18902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7793,"num_write_kb":31172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9236","reported_seq":9357,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:59.682383+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:59.682383+0000","last_peered":"2024-09-10T15:55:59.682383+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:59.682383+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9236,"log_dups_size":0,"ondisk_log_size":9236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37814272,"num_objects":9232,"num_object_clones":0,"num_object_copies":18464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7504,"num_write_kb":30016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5430}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9227","reported_seq":9306,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:59.721158+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:59.721158+0000","last_peered":"2024-09-10T15:55:59.721158+0000","last_clean":"2024-09-10T15:55:59.721158+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:59.721158+0000","last_undegraded":"2024-09-10T15:55:59.721158+0000","last_fullsized":"2024-09-10T15:55:59.721158+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9227,"log_dups_size":0,"ondisk_log_size":9227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37793792,"num_objects":9227,"num_object_clones":0,"num_object_copies":18454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9227,"num_write_kb":36908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9097","reported_seq":9223,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:58.574045+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:58.574045+0000","last_peered":"2024-09-10T15:55:58.574045+0000","last_clean":"2024-09-10T15:55:58.574045+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:58.574045+0000","last_undegraded":"2024-09-10T15:55:58.574045+0000","last_fullsized":"2024-09-10T15:55:58.574045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9097,"log_dups_size":0,"ondisk_log_size":9097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37232640,"num_objects":9090,"num_object_clones":0,"num_object_copies":18180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6168,"num_write_kb":24672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9346","reported_seq":10789,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:58.593064+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:58.593064+0000","last_peered":"2024-09-10T15:55:58.593064+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:58.593064+0000","last_undegraded":"2024-09-10T15:55:58.593064+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9346,"log_dups_size":0,"ondisk_log_size":9346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38260736,"num_objects":9341,"num_object_clones":0,"num_object_copies":18682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3213,"num_objects_unfound":0,"num_objects_dirty":9341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6064,"num_write_kb":24256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":655,"num_bytes_recovered":2682880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6128},{"shards":"0,2","objects":3213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9039","reported_seq":9179,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.569383+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:58.569383+0000","last_peered":"2024-09-10T15:55:58.569383+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:58.569383+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9039,"log_dups_size":0,"ondisk_log_size":9039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36999168,"num_objects":9033,"num_object_clones":0,"num_object_copies":18066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5689,"num_write_kb":22756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5199},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9331","reported_seq":9462,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:58.581554+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:58.581554+0000","last_peered":"2024-09-10T15:55:58.581554+0000","last_clean":"2024-09-10T15:55:58.581554+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:58.581554+0000","last_undegraded":"2024-09-10T15:55:58.581554+0000","last_fullsized":"2024-09-10T15:55:58.581554+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9331,"log_dups_size":0,"ondisk_log_size":9331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38211584,"num_objects":9329,"num_object_clones":0,"num_object_copies":18658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6090,"num_write_kb":24360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9293","reported_seq":17420,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:58.741537+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:58.741537+0000","last_peered":"2024-09-10T15:55:58.741537+0000","last_clean":"2024-09-10T15:55:58.741537+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:58.741537+0000","last_undegraded":"2024-09-10T15:55:58.741537+0000","last_fullsized":"2024-09-10T15:55:58.741537+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38035456,"num_objects":9286,"num_object_clones":0,"num_object_copies":18572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7270,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9164","reported_seq":9290,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.573990+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:58.573990+0000","last_peered":"2024-09-10T15:55:58.573990+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:58.573990+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9164,"log_dups_size":0,"ondisk_log_size":9164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37502976,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6028,"num_write_kb":24112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5293},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9341","reported_seq":9465,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:59.704628+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:59.704628+0000","last_peered":"2024-09-10T15:55:59.704628+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:59.704628+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9341,"log_dups_size":0,"ondisk_log_size":9341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38240256,"num_objects":9336,"num_object_clones":0,"num_object_copies":18672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7315,"num_write_kb":29260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5504}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9173","reported_seq":9299,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:59.676954+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:59.676954+0000","last_peered":"2024-09-10T15:55:59.676954+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:59.676954+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9173,"log_dups_size":0,"ondisk_log_size":9173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37548032,"num_objects":9167,"num_object_clones":0,"num_object_copies":18334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7042,"num_write_kb":28168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5338}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9297","reported_seq":9424,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.602731+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:58.602731+0000","last_peered":"2024-09-10T15:55:58.602731+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:58.602731+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9297,"log_dups_size":0,"ondisk_log_size":9297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072320,"num_objects":9295,"num_object_clones":0,"num_object_copies":18590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5462},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9279","reported_seq":9438,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.574118+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:58.574118+0000","last_peered":"2024-09-10T15:55:58.574118+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:58.574118+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9279,"log_dups_size":0,"ondisk_log_size":9279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37974016,"num_objects":9271,"num_object_clones":0,"num_object_copies":18542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5697,"num_write_kb":22788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9285","reported_seq":9421,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.608334+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:58.608334+0000","last_peered":"2024-09-10T15:55:58.608334+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:58.608334+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9285,"log_dups_size":0,"ondisk_log_size":9285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38014976,"num_objects":9281,"num_object_clones":0,"num_object_copies":18562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5891,"num_write_kb":23564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5404},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9464","reported_seq":9602,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.561525+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:58.561525+0000","last_peered":"2024-09-10T15:55:58.561525+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:58.561525+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9464,"log_dups_size":0,"ondisk_log_size":9464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38715875,"num_objects":9453,"num_object_clones":0,"num_object_copies":18906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6283,"num_write_kb":25132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5450},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":607089123,"num_objects":148216,"num_object_clones":0,"num_object_copies":296432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3213,"num_objects_unfound":0,"num_objects_dirty":148216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109818,"num_write_kb":439272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4651,"num_bytes_recovered":19042304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1233248256,"data_stored":1233165157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148297,"ondisk_log_size":148297,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626467,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487692,"kb_used_data":275580,"kb_used_omap":4,"kb_used_meta":212091,"kb_avail":93884148,"statfs":{"total":96636764160,"available":96137367552,"internally_reserved":0,"allocated":282193920,"data_stored":281855607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217181870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149750,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":808256,"kb_used_data":505904,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93563584,"statfs":{"total":96636764160,"available":95809110016,"internally_reserved":0,"allocated":518045696,"data_stored":517779643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309587629},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,8,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691528,"kb_used_data":421112,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93680312,"statfs":{"total":96636764160,"available":95928639488,"internally_reserved":0,"allocated":431218688,"data_stored":430952635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":518537216,"data_stored":518533603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283205632,"data_stored":283129759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431505408,"data_stored":431501795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:01.376 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:01.827 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 204 20 149504 149484 2.86193 1.73438 0.0259342 0.0272902 2024-09-10T15:56:01.990 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:01.990 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:02.023 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-10T15:56:00.487324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":607089142,"num_objects":148218,"num_object_clones":0,"num_object_copies":296436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3213,"num_objects_unfound":0,"num_objects_dirty":148218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109820,"num_write_kb":439274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4653,"num_bytes_recovered":19042323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148300,"ondisk_log_size":148300,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1987476,"kb_used_data":1202596,"kb_used_omap":12,"kb_used_meta":784819,"kb_avail":281128044,"statfs":{"total":289910292480,"available":287875117056,"internally_reserved":0,"allocated":1231458304,"data_stored":1230587885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":803654661},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,10,16,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29036544,"num_objects":7089,"num_object_clones":0,"num_object_copies":14178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-235,"num_objects_unfound":0,"num_objects_dirty":7089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7089,"num_write_kb":28356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":962560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004581"},"pg_stats":[{"pgid":"2.f","version":"46'9268","reported_seq":9344,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.596755+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:58.596755+0000","last_peered":"2024-09-10T15:55:58.596755+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:58.596755+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9268,"log_dups_size":0,"ondisk_log_size":9268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37961728,"num_objects":9268,"num_object_clones":0,"num_object_copies":18536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9268,"num_write_kb":37072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5295},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9457","reported_seq":9564,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:59.721536+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:59.721536+0000","last_peered":"2024-09-10T15:55:59.721536+0000","last_clean":"2024-09-10T15:55:59.721536+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:59.721536+0000","last_undegraded":"2024-09-10T15:55:59.721536+0000","last_fullsized":"2024-09-10T15:55:59.721536+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9457,"log_dups_size":0,"ondisk_log_size":9457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38711296,"num_objects":9451,"num_object_clones":0,"num_object_copies":18902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7793,"num_write_kb":31172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9236","reported_seq":9357,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:59.682383+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:59.682383+0000","last_peered":"2024-09-10T15:55:59.682383+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:59.682383+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9236,"log_dups_size":0,"ondisk_log_size":9236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37814272,"num_objects":9232,"num_object_clones":0,"num_object_copies":18464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7504,"num_write_kb":30016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5430}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9227","reported_seq":9306,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:59.721158+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:59.721158+0000","last_peered":"2024-09-10T15:55:59.721158+0000","last_clean":"2024-09-10T15:55:59.721158+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:59.721158+0000","last_undegraded":"2024-09-10T15:55:59.721158+0000","last_fullsized":"2024-09-10T15:55:59.721158+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9227,"log_dups_size":0,"ondisk_log_size":9227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37793792,"num_objects":9227,"num_object_clones":0,"num_object_copies":18454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9227,"num_write_kb":36908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9097","reported_seq":9223,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:58.574045+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:58.574045+0000","last_peered":"2024-09-10T15:55:58.574045+0000","last_clean":"2024-09-10T15:55:58.574045+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:58.574045+0000","last_undegraded":"2024-09-10T15:55:58.574045+0000","last_fullsized":"2024-09-10T15:55:58.574045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9097,"log_dups_size":0,"ondisk_log_size":9097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37232640,"num_objects":9090,"num_object_clones":0,"num_object_copies":18180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6168,"num_write_kb":24672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9346","reported_seq":10789,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:58.593064+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:58.593064+0000","last_peered":"2024-09-10T15:55:58.593064+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:58.593064+0000","last_undegraded":"2024-09-10T15:55:58.593064+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9346,"log_dups_size":0,"ondisk_log_size":9346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38260736,"num_objects":9341,"num_object_clones":0,"num_object_copies":18682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3213,"num_objects_unfound":0,"num_objects_dirty":9341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6064,"num_write_kb":24256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":655,"num_bytes_recovered":2682880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6128},{"shards":"0,2","objects":3213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9039","reported_seq":9179,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.569383+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:58.569383+0000","last_peered":"2024-09-10T15:55:58.569383+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:58.569383+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9039,"log_dups_size":0,"ondisk_log_size":9039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36999168,"num_objects":9033,"num_object_clones":0,"num_object_copies":18066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5689,"num_write_kb":22756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5199},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9331","reported_seq":9462,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:58.581554+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:58.581554+0000","last_peered":"2024-09-10T15:55:58.581554+0000","last_clean":"2024-09-10T15:55:58.581554+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:58.581554+0000","last_undegraded":"2024-09-10T15:55:58.581554+0000","last_fullsized":"2024-09-10T15:55:58.581554+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9331,"log_dups_size":0,"ondisk_log_size":9331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38211584,"num_objects":9329,"num_object_clones":0,"num_object_copies":18658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6090,"num_write_kb":24360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9293","reported_seq":17420,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:58.741537+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:58.741537+0000","last_peered":"2024-09-10T15:55:58.741537+0000","last_clean":"2024-09-10T15:55:58.741537+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:58.741537+0000","last_undegraded":"2024-09-10T15:55:58.741537+0000","last_fullsized":"2024-09-10T15:55:58.741537+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38035456,"num_objects":9286,"num_object_clones":0,"num_object_copies":18572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7270,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9164","reported_seq":9290,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.573990+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:58.573990+0000","last_peered":"2024-09-10T15:55:58.573990+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:58.573990+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9164,"log_dups_size":0,"ondisk_log_size":9164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37502976,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6028,"num_write_kb":24112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5293},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9341","reported_seq":9465,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:59.704628+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:59.704628+0000","last_peered":"2024-09-10T15:55:59.704628+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:59.704628+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9341,"log_dups_size":0,"ondisk_log_size":9341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38240256,"num_objects":9336,"num_object_clones":0,"num_object_copies":18672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7315,"num_write_kb":29260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5504}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9173","reported_seq":9299,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:59.676954+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:59.676954+0000","last_peered":"2024-09-10T15:55:59.676954+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:59.676954+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9173,"log_dups_size":0,"ondisk_log_size":9173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37548032,"num_objects":9167,"num_object_clones":0,"num_object_copies":18334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7042,"num_write_kb":28168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5338}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9297","reported_seq":9424,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.602731+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:58.602731+0000","last_peered":"2024-09-10T15:55:58.602731+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:58.602731+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9297,"log_dups_size":0,"ondisk_log_size":9297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072320,"num_objects":9295,"num_object_clones":0,"num_object_copies":18590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5462},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9279","reported_seq":9438,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.574118+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:58.574118+0000","last_peered":"2024-09-10T15:55:58.574118+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:58.574118+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9279,"log_dups_size":0,"ondisk_log_size":9279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37974016,"num_objects":9271,"num_object_clones":0,"num_object_copies":18542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5697,"num_write_kb":22788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9285","reported_seq":9421,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.608334+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:58.608334+0000","last_peered":"2024-09-10T15:55:58.608334+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:58.608334+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9285,"log_dups_size":0,"ondisk_log_size":9285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38014976,"num_objects":9281,"num_object_clones":0,"num_object_copies":18562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5891,"num_write_kb":23564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5404},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9464","reported_seq":9602,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.561525+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:58.561525+0000","last_peered":"2024-09-10T15:55:58.561525+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:58.561525+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9464,"log_dups_size":0,"ondisk_log_size":9464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38715875,"num_objects":9453,"num_object_clones":0,"num_object_copies":18906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6283,"num_write_kb":25132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5450},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":607089123,"num_objects":148216,"num_object_clones":0,"num_object_copies":296432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3213,"num_objects_unfound":0,"num_objects_dirty":148216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109818,"num_write_kb":439272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4651,"num_bytes_recovered":19042304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1233248256,"data_stored":1233165157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148297,"ondisk_log_size":148297,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626467,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487692,"kb_used_data":275580,"kb_used_omap":4,"kb_used_meta":212091,"kb_avail":93884148,"statfs":{"total":96636764160,"available":96137367552,"internally_reserved":0,"allocated":282193920,"data_stored":281855607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217181870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149750,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":808256,"kb_used_data":505904,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93563584,"statfs":{"total":96636764160,"available":95809110016,"internally_reserved":0,"allocated":518045696,"data_stored":517779643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309587629},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,8,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691528,"kb_used_data":421112,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93680312,"statfs":{"total":96636764160,"available":95928639488,"internally_reserved":0,"allocated":431218688,"data_stored":430952635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":518537216,"data_stored":518533603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283205632,"data_stored":283129759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431505408,"data_stored":431501795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:02.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:56:02.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'9268", 'reported_seq': 9344, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:58.596755+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:55:58.596755+0000', 'last_peered': '2024-09-10T15:55:58.596755+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:55:58.596755+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 9268, 'log_dups_size': 0, 'ondisk_log_size': 9268, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37961728, 'num_objects': 9268, 'num_object_clones': 0, 'num_object_copies': 18536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9268, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9268, 'num_write_kb': 37072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5295}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:02.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:56:02.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'9236", 'reported_seq': 9357, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:59.682383+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:55:59.682383+0000', 'last_peered': '2024-09-10T15:55:59.682383+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:55:59.682383+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 9236, 'log_dups_size': 0, 'ondisk_log_size': 9236, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37814272, 'num_objects': 9232, 'num_object_clones': 0, 'num_object_copies': 18464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9232, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7504, 'num_write_kb': 30016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 5430}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:02.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:56:02.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'9346", 'reported_seq': 10789, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:55:58.593064+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:55:58.593064+0000', 'last_peered': '2024-09-10T15:55:58.593064+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:55:58.593064+0000', 'last_undegraded': '2024-09-10T15:55:58.593064+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 9346, 'log_dups_size': 0, 'ondisk_log_size': 9346, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38260736, 'num_objects': 9341, 'num_object_clones': 0, 'num_object_copies': 18682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3213, 'num_objects_unfound': 0, 'num_objects_dirty': 9341, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6064, 'num_write_kb': 24256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 655, 'num_bytes_recovered': 2682880, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6128}, {'shards': '0,2', 'objects': 3213}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:02.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:56:02.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'9039", 'reported_seq': 9179, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:58.569383+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:55:58.569383+0000', 'last_peered': '2024-09-10T15:55:58.569383+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:55:58.569383+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 9039, 'log_dups_size': 0, 'ondisk_log_size': 9039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36999168, 'num_objects': 9033, 'num_object_clones': 0, 'num_object_copies': 18066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5689, 'num_write_kb': 22756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5199}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:02.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:56:02.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'9164", 'reported_seq': 9290, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:58.573990+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:55:58.573990+0000', 'last_peered': '2024-09-10T15:55:58.573990+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:55:58.573990+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 9164, 'log_dups_size': 0, 'ondisk_log_size': 9164, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37502976, 'num_objects': 9156, 'num_object_clones': 0, 'num_object_copies': 18312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6028, 'num_write_kb': 24112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5293}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:02.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:56:02.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'9341", 'reported_seq': 9465, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:59.704628+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:55:59.704628+0000', 'last_peered': '2024-09-10T15:55:59.704628+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:55:59.704628+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 9341, 'log_dups_size': 0, 'ondisk_log_size': 9341, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38240256, 'num_objects': 9336, 'num_object_clones': 0, 'num_object_copies': 18672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9336, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7315, 'num_write_kb': 29260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 5504}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:02.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:56:02.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'9173", 'reported_seq': 9299, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:59.676954+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:55:59.676954+0000', 'last_peered': '2024-09-10T15:55:59.676954+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:55:59.676954+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 9173, 'log_dups_size': 0, 'ondisk_log_size': 9173, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37548032, 'num_objects': 9167, 'num_object_clones': 0, 'num_object_copies': 18334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9167, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7042, 'num_write_kb': 28168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 5338}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:02.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:56:02.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'9297", 'reported_seq': 9424, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:58.602731+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:55:58.602731+0000', 'last_peered': '2024-09-10T15:55:58.602731+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:55:58.602731+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 9297, 'log_dups_size': 0, 'ondisk_log_size': 9297, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38072320, 'num_objects': 9295, 'num_object_clones': 0, 'num_object_copies': 18590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6489, 'num_write_kb': 25956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5462}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:02.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:56:02.027 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'9279", 'reported_seq': 9438, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:58.574118+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:55:58.574118+0000', 'last_peered': '2024-09-10T15:55:58.574118+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:55:58.574118+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 9279, 'log_dups_size': 0, 'ondisk_log_size': 9279, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37974016, 'num_objects': 9271, 'num_object_clones': 0, 'num_object_copies': 18542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5697, 'num_write_kb': 22788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5354}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:02.027 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:56:02.027 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'9285", 'reported_seq': 9421, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:58.608334+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:55:58.608334+0000', 'last_peered': '2024-09-10T15:55:58.608334+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:55:58.608334+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 9285, 'log_dups_size': 0, 'ondisk_log_size': 9285, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38014976, 'num_objects': 9281, 'num_object_clones': 0, 'num_object_copies': 18562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9281, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5891, 'num_write_kb': 23564, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5404}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:02.027 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:56:02.027 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'9464", 'reported_seq': 9602, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:55:58.561525+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:55:58.561525+0000', 'last_peered': '2024-09-10T15:55:58.561525+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:55:58.561525+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 9464, 'log_dups_size': 0, 'ondisk_log_size': 9464, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38715875, 'num_objects': 9453, 'num_object_clones': 0, 'num_object_copies': 18906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9453, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6283, 'num_write_kb': 25132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5450}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:02.027 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:56:02.735 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:02.735 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/296436 objects degraded (13.082%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":288,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":148218,"data_bytes":607089142,"bytes_used":2035175424,"bytes_avail":287875117056,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":296436,"degraded_ratio":0.13082419139375784,"misplaced_objects":3213,"misplaced_total":296436,"misplaced_ratio":0.010838764522527628,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":80182,"recovering_keys_per_sec":0,"num_objects_recovered":235,"num_bytes_recovered":962560,"num_keys_recovered":0,"write_bytes_sec":2418788,"read_op_per_sec":0,"write_op_per_sec":590},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:56:02.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:56:02.758 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:02.827 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 205 20 149931 149911 2.8561 1.66797 0.0767866 0.0273458 2024-09-10T15:56:03.300 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:03.301 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:03.324 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-10T15:56:02.488010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":607089142,"num_objects":148218,"num_object_clones":0,"num_object_copies":296436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3213,"num_objects_unfound":0,"num_objects_dirty":148218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109820,"num_write_kb":439274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4653,"num_bytes_recovered":19042323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148300,"ondisk_log_size":148300,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1987476,"kb_used_data":1202596,"kb_used_omap":12,"kb_used_meta":784819,"kb_avail":281128044,"statfs":{"total":289910292480,"available":287875117056,"internally_reserved":0,"allocated":1231458304,"data_stored":1230587885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":803654661},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,10,16,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19066880,"num_objects":4655,"num_object_clones":0,"num_object_copies":9310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-161,"num_objects_unfound":0,"num_objects_dirty":4655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004648"},"pg_stats":[{"pgid":"2.f","version":"46'9268","reported_seq":9344,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.596755+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:55:58.596755+0000","last_peered":"2024-09-10T15:55:58.596755+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:55:58.596755+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9268,"log_dups_size":0,"ondisk_log_size":9268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37961728,"num_objects":9268,"num_object_clones":0,"num_object_copies":18536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9268,"num_write_kb":37072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5295},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9457","reported_seq":9564,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:59.721536+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:55:59.721536+0000","last_peered":"2024-09-10T15:55:59.721536+0000","last_clean":"2024-09-10T15:55:59.721536+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:55:59.721536+0000","last_undegraded":"2024-09-10T15:55:59.721536+0000","last_fullsized":"2024-09-10T15:55:59.721536+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9457,"log_dups_size":0,"ondisk_log_size":9457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38711296,"num_objects":9451,"num_object_clones":0,"num_object_copies":18902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7793,"num_write_kb":31172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9236","reported_seq":9357,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:59.682383+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:55:59.682383+0000","last_peered":"2024-09-10T15:55:59.682383+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:55:59.682383+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9236,"log_dups_size":0,"ondisk_log_size":9236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37814272,"num_objects":9232,"num_object_clones":0,"num_object_copies":18464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7504,"num_write_kb":30016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5430}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9227","reported_seq":9306,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:59.721158+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:55:59.721158+0000","last_peered":"2024-09-10T15:55:59.721158+0000","last_clean":"2024-09-10T15:55:59.721158+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:55:59.721158+0000","last_undegraded":"2024-09-10T15:55:59.721158+0000","last_fullsized":"2024-09-10T15:55:59.721158+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9227,"log_dups_size":0,"ondisk_log_size":9227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37793792,"num_objects":9227,"num_object_clones":0,"num_object_copies":18454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9227,"num_write_kb":36908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9097","reported_seq":9223,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:58.574045+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:55:58.574045+0000","last_peered":"2024-09-10T15:55:58.574045+0000","last_clean":"2024-09-10T15:55:58.574045+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:55:58.574045+0000","last_undegraded":"2024-09-10T15:55:58.574045+0000","last_fullsized":"2024-09-10T15:55:58.574045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9097,"log_dups_size":0,"ondisk_log_size":9097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37232640,"num_objects":9090,"num_object_clones":0,"num_object_copies":18180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6168,"num_write_kb":24672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9346","reported_seq":10789,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:55:58.593064+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:55:58.593064+0000","last_peered":"2024-09-10T15:55:58.593064+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:55:58.593064+0000","last_undegraded":"2024-09-10T15:55:58.593064+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9346,"log_dups_size":0,"ondisk_log_size":9346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38260736,"num_objects":9341,"num_object_clones":0,"num_object_copies":18682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3213,"num_objects_unfound":0,"num_objects_dirty":9341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6064,"num_write_kb":24256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":655,"num_bytes_recovered":2682880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6128},{"shards":"0,2","objects":3213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9039","reported_seq":9179,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.569383+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:55:58.569383+0000","last_peered":"2024-09-10T15:55:58.569383+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:55:58.569383+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9039,"log_dups_size":0,"ondisk_log_size":9039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36999168,"num_objects":9033,"num_object_clones":0,"num_object_copies":18066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5689,"num_write_kb":22756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5199},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9331","reported_seq":9462,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:58.581554+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:55:58.581554+0000","last_peered":"2024-09-10T15:55:58.581554+0000","last_clean":"2024-09-10T15:55:58.581554+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:55:58.581554+0000","last_undegraded":"2024-09-10T15:55:58.581554+0000","last_fullsized":"2024-09-10T15:55:58.581554+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9331,"log_dups_size":0,"ondisk_log_size":9331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38211584,"num_objects":9329,"num_object_clones":0,"num_object_copies":18658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6090,"num_write_kb":24360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9293","reported_seq":17420,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:58.741537+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:55:58.741537+0000","last_peered":"2024-09-10T15:55:58.741537+0000","last_clean":"2024-09-10T15:55:58.741537+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:55:58.741537+0000","last_undegraded":"2024-09-10T15:55:58.741537+0000","last_fullsized":"2024-09-10T15:55:58.741537+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38035456,"num_objects":9286,"num_object_clones":0,"num_object_copies":18572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7270,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9164","reported_seq":9290,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.573990+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:55:58.573990+0000","last_peered":"2024-09-10T15:55:58.573990+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:55:58.573990+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9164,"log_dups_size":0,"ondisk_log_size":9164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37502976,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6028,"num_write_kb":24112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5293},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9341","reported_seq":9465,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:59.704628+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:55:59.704628+0000","last_peered":"2024-09-10T15:55:59.704628+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:55:59.704628+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9341,"log_dups_size":0,"ondisk_log_size":9341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38240256,"num_objects":9336,"num_object_clones":0,"num_object_copies":18672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7315,"num_write_kb":29260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5504}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9173","reported_seq":9299,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:59.676954+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:55:59.676954+0000","last_peered":"2024-09-10T15:55:59.676954+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:55:59.676954+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9173,"log_dups_size":0,"ondisk_log_size":9173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37548032,"num_objects":9167,"num_object_clones":0,"num_object_copies":18334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7042,"num_write_kb":28168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5338}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9297","reported_seq":9424,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.602731+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:55:58.602731+0000","last_peered":"2024-09-10T15:55:58.602731+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:55:58.602731+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9297,"log_dups_size":0,"ondisk_log_size":9297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072320,"num_objects":9295,"num_object_clones":0,"num_object_copies":18590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5462},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9279","reported_seq":9438,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.574118+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:55:58.574118+0000","last_peered":"2024-09-10T15:55:58.574118+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:55:58.574118+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9279,"log_dups_size":0,"ondisk_log_size":9279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37974016,"num_objects":9271,"num_object_clones":0,"num_object_copies":18542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5697,"num_write_kb":22788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9285","reported_seq":9421,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.608334+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:55:58.608334+0000","last_peered":"2024-09-10T15:55:58.608334+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:55:58.608334+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9285,"log_dups_size":0,"ondisk_log_size":9285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38014976,"num_objects":9281,"num_object_clones":0,"num_object_copies":18562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5891,"num_write_kb":23564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5404},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9464","reported_seq":9602,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:55:58.561525+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:55:58.561525+0000","last_peered":"2024-09-10T15:55:58.561525+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:55:58.561525+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9464,"log_dups_size":0,"ondisk_log_size":9464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38715875,"num_objects":9453,"num_object_clones":0,"num_object_copies":18906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6283,"num_write_kb":25132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5450},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":607089123,"num_objects":148216,"num_object_clones":0,"num_object_copies":296432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3213,"num_objects_unfound":0,"num_objects_dirty":148216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109818,"num_write_kb":439272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4651,"num_bytes_recovered":19042304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1233248256,"data_stored":1233165157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148297,"ondisk_log_size":148297,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626467,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487692,"kb_used_data":275580,"kb_used_omap":4,"kb_used_meta":212091,"kb_avail":93884148,"statfs":{"total":96636764160,"available":96137367552,"internally_reserved":0,"allocated":282193920,"data_stored":281855607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217181870},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149750,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":808256,"kb_used_data":505904,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93563584,"statfs":{"total":96636764160,"available":95809110016,"internally_reserved":0,"allocated":518045696,"data_stored":517779643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309587629},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,8,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691528,"kb_used_data":421112,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93680312,"statfs":{"total":96636764160,"available":95928639488,"internally_reserved":0,"allocated":431218688,"data_stored":430952635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":518537216,"data_stored":518533603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283205632,"data_stored":283129759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431505408,"data_stored":431501795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:03.827 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 206 20 150395 150375 2.85104 1.8125 0.0901881 0.0273945 2024-09-10T15:56:04.827 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 207 20 150884 150864 2.84649 1.91016 0.0335506 0.027438 2024-09-10T15:56:05.828 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 208 20 151369 151349 2.84191 1.89453 0.0326664 0.0274832 2024-09-10T15:56:06.327 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:06.828 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 209 20 151831 151811 2.83695 1.80469 0.0311302 0.0275307 2024-09-10T15:56:06.906 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:06.907 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:06.932 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-10T15:56:06.489347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616395254,"num_objects":150490,"num_object_clones":0,"num_object_copies":300980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":150490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112092,"num_write_kb":448362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4733,"num_bytes_recovered":19370003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150572,"ondisk_log_size":150572,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2023904,"kb_used_data":1222384,"kb_used_omap":12,"kb_used_meta":801459,"kb_avail":281091616,"statfs":{"total":289910292480,"available":287837814784,"internally_reserved":0,"allocated":1251721216,"data_stored":1250850797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":820694021},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,7,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21880832,"num_objects":5342,"num_object_clones":0,"num_object_copies":10684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-161,"num_objects_unfound":0,"num_objects_dirty":5342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5342,"num_write_kb":21368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004340"},"pg_stats":[{"pgid":"2.f","version":"46'9420","reported_seq":9496,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.672546+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:03.672546+0000","last_peered":"2024-09-10T15:56:03.672546+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:03.672546+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9420,"log_dups_size":0,"ondisk_log_size":9420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38584320,"num_objects":9420,"num_object_clones":0,"num_object_copies":18840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9420,"num_write_kb":37680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5447},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9609","reported_seq":9716,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:04.702833+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:04.702833+0000","last_peered":"2024-09-10T15:56:04.702833+0000","last_clean":"2024-09-10T15:56:04.702833+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:04.702833+0000","last_undegraded":"2024-09-10T15:56:04.702833+0000","last_fullsized":"2024-09-10T15:56:04.702833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9609,"log_dups_size":0,"ondisk_log_size":9609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39333888,"num_objects":9603,"num_object_clones":0,"num_object_copies":19206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7945,"num_write_kb":31780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9390","reported_seq":9513,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:04.592825+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:04.592825+0000","last_peered":"2024-09-10T15:56:04.592825+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:04.592825+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9390,"log_dups_size":0,"ondisk_log_size":9390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38445056,"num_objects":9386,"num_object_clones":0,"num_object_copies":18772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7658,"num_write_kb":30632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5584}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9373","reported_seq":9452,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:04.721788+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:04.721788+0000","last_peered":"2024-09-10T15:56:04.721788+0000","last_clean":"2024-09-10T15:56:04.721788+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:04.721788+0000","last_undegraded":"2024-09-10T15:56:04.721788+0000","last_fullsized":"2024-09-10T15:56:04.721788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9373,"log_dups_size":0,"ondisk_log_size":9373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38391808,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9373,"num_write_kb":37492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9213","reported_seq":9338,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:03.648566+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:03.648566+0000","last_peered":"2024-09-10T15:56:03.648566+0000","last_clean":"2024-09-10T15:56:03.648566+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:03.648566+0000","last_undegraded":"2024-09-10T15:56:03.648566+0000","last_fullsized":"2024-09-10T15:56:03.648566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9213,"log_dups_size":0,"ondisk_log_size":9213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37707776,"num_objects":9206,"num_object_clones":0,"num_object_copies":18412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6284,"num_write_kb":25136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9483","reported_seq":11088,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:03.694436+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:03.694436+0000","last_peered":"2024-09-10T15:56:03.694436+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:03.694436+0000","last_undegraded":"2024-09-10T15:56:03.694436+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9483,"log_dups_size":0,"ondisk_log_size":9483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38821888,"num_objects":9478,"num_object_clones":0,"num_object_copies":18956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":9478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6201,"num_write_kb":24804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":735,"num_bytes_recovered":3010560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6345},{"shards":"0,2","objects":3133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9156","reported_seq":9296,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.596031+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:03.596031+0000","last_peered":"2024-09-10T15:56:03.596031+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:03.596031+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9156,"log_dups_size":0,"ondisk_log_size":9156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37478400,"num_objects":9150,"num_object_clones":0,"num_object_copies":18300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5806,"num_write_kb":23224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5316},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9472","reported_seq":9603,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:03.621269+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:03.621269+0000","last_peered":"2024-09-10T15:56:03.621269+0000","last_clean":"2024-09-10T15:56:03.621269+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:03.621269+0000","last_undegraded":"2024-09-10T15:56:03.621269+0000","last_fullsized":"2024-09-10T15:56:03.621269+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38789120,"num_objects":9470,"num_object_clones":0,"num_object_copies":18940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6231,"num_write_kb":24924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9450","reported_seq":17577,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:03.882528+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:03.882528+0000","last_peered":"2024-09-10T15:56:03.882528+0000","last_clean":"2024-09-10T15:56:03.882528+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:03.882528+0000","last_undegraded":"2024-09-10T15:56:03.882528+0000","last_fullsized":"2024-09-10T15:56:03.882528+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9450,"log_dups_size":0,"ondisk_log_size":9450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38678528,"num_objects":9443,"num_object_clones":0,"num_object_copies":18886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7427,"num_write_kb":29708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9282","reported_seq":9408,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.561588+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:03.561588+0000","last_peered":"2024-09-10T15:56:03.561588+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:03.561588+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9282,"log_dups_size":0,"ondisk_log_size":9282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37986304,"num_objects":9274,"num_object_clones":0,"num_object_copies":18548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6146,"num_write_kb":24584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5411},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9472","reported_seq":9596,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:04.679806+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:04.679806+0000","last_peered":"2024-09-10T15:56:04.679806+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:04.679806+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38776832,"num_objects":9467,"num_object_clones":0,"num_object_copies":18934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7446,"num_write_kb":29784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5635}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9324","reported_seq":9450,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:04.688009+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:04.688009+0000","last_peered":"2024-09-10T15:56:04.688009+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:04.688009+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9324,"log_dups_size":0,"ondisk_log_size":9324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38166528,"num_objects":9318,"num_object_clones":0,"num_object_copies":18636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7193,"num_write_kb":28772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5489}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9462","reported_seq":9589,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.708943+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:03.708943+0000","last_peered":"2024-09-10T15:56:03.708943+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:03.708943+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9462,"log_dups_size":0,"ondisk_log_size":9462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38748160,"num_objects":9460,"num_object_clones":0,"num_object_copies":18920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6654,"num_write_kb":26616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5627},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9414","reported_seq":9577,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.685001+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:03.685001+0000","last_peered":"2024-09-10T15:56:03.685001+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:03.685001+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9414,"log_dups_size":0,"ondisk_log_size":9414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5832,"num_write_kb":23328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5489},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9437","reported_seq":9572,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.719248+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:03.719248+0000","last_peered":"2024-09-10T15:56:03.719248+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:03.719248+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9437,"log_dups_size":0,"ondisk_log_size":9437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38637568,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6043,"num_write_kb":24172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5556},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9612","reported_seq":9750,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.663237+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:03.663237+0000","last_peered":"2024-09-10T15:56:03.663237+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:03.663237+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9612,"log_dups_size":0,"ondisk_log_size":9612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39322083,"num_objects":9601,"num_object_clones":0,"num_object_copies":19202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6431,"num_write_kb":25724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5598},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":616395235,"num_objects":150488,"num_object_clones":0,"num_object_copies":300976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":150488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112090,"num_write_kb":448360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4731,"num_bytes_recovered":19369984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1252339712,"data_stored":1252256613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150569,"ondisk_log_size":150569,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626468,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504508,"kb_used_data":284076,"kb_used_omap":4,"kb_used_meta":220411,"kb_avail":93867332,"statfs":{"total":96636764160,"available":96120147968,"internally_reserved":0,"allocated":290893824,"data_stored":290555511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":225701550},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149751,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819060,"kb_used_data":512548,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93552780,"statfs":{"total":96636764160,"available":95798046720,"internally_reserved":0,"allocated":524849152,"data_stored":524583099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313847469},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,1,6,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700336,"kb_used_data":425760,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93671504,"statfs":{"total":96636764160,"available":95919620096,"internally_reserved":0,"allocated":435978240,"data_stored":435712187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":525393920,"data_stored":525390307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":290639872,"data_stored":290563999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":436305920,"data_stored":436302307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:06.934 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:07.528 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:07.529 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:07.552 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-10T15:56:06.489347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616395254,"num_objects":150490,"num_object_clones":0,"num_object_copies":300980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":150490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112092,"num_write_kb":448362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4733,"num_bytes_recovered":19370003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150572,"ondisk_log_size":150572,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2023904,"kb_used_data":1222384,"kb_used_omap":12,"kb_used_meta":801459,"kb_avail":281091616,"statfs":{"total":289910292480,"available":287837814784,"internally_reserved":0,"allocated":1251721216,"data_stored":1250850797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":820694021},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,7,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21880832,"num_objects":5342,"num_object_clones":0,"num_object_copies":10684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-161,"num_objects_unfound":0,"num_objects_dirty":5342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5342,"num_write_kb":21368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004340"},"pg_stats":[{"pgid":"2.f","version":"46'9420","reported_seq":9496,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.672546+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:03.672546+0000","last_peered":"2024-09-10T15:56:03.672546+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:03.672546+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9420,"log_dups_size":0,"ondisk_log_size":9420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38584320,"num_objects":9420,"num_object_clones":0,"num_object_copies":18840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9420,"num_write_kb":37680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5447},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9609","reported_seq":9716,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:04.702833+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:04.702833+0000","last_peered":"2024-09-10T15:56:04.702833+0000","last_clean":"2024-09-10T15:56:04.702833+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:04.702833+0000","last_undegraded":"2024-09-10T15:56:04.702833+0000","last_fullsized":"2024-09-10T15:56:04.702833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9609,"log_dups_size":0,"ondisk_log_size":9609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39333888,"num_objects":9603,"num_object_clones":0,"num_object_copies":19206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7945,"num_write_kb":31780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9390","reported_seq":9513,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:04.592825+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:04.592825+0000","last_peered":"2024-09-10T15:56:04.592825+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:04.592825+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9390,"log_dups_size":0,"ondisk_log_size":9390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38445056,"num_objects":9386,"num_object_clones":0,"num_object_copies":18772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7658,"num_write_kb":30632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5584}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9373","reported_seq":9452,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:04.721788+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:04.721788+0000","last_peered":"2024-09-10T15:56:04.721788+0000","last_clean":"2024-09-10T15:56:04.721788+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:04.721788+0000","last_undegraded":"2024-09-10T15:56:04.721788+0000","last_fullsized":"2024-09-10T15:56:04.721788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9373,"log_dups_size":0,"ondisk_log_size":9373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38391808,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9373,"num_write_kb":37492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9213","reported_seq":9338,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:03.648566+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:03.648566+0000","last_peered":"2024-09-10T15:56:03.648566+0000","last_clean":"2024-09-10T15:56:03.648566+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:03.648566+0000","last_undegraded":"2024-09-10T15:56:03.648566+0000","last_fullsized":"2024-09-10T15:56:03.648566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9213,"log_dups_size":0,"ondisk_log_size":9213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37707776,"num_objects":9206,"num_object_clones":0,"num_object_copies":18412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6284,"num_write_kb":25136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9483","reported_seq":11088,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:03.694436+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:03.694436+0000","last_peered":"2024-09-10T15:56:03.694436+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:03.694436+0000","last_undegraded":"2024-09-10T15:56:03.694436+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9483,"log_dups_size":0,"ondisk_log_size":9483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38821888,"num_objects":9478,"num_object_clones":0,"num_object_copies":18956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":9478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6201,"num_write_kb":24804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":735,"num_bytes_recovered":3010560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6345},{"shards":"0,2","objects":3133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9156","reported_seq":9296,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.596031+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:03.596031+0000","last_peered":"2024-09-10T15:56:03.596031+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:03.596031+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9156,"log_dups_size":0,"ondisk_log_size":9156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37478400,"num_objects":9150,"num_object_clones":0,"num_object_copies":18300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5806,"num_write_kb":23224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5316},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9472","reported_seq":9603,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:03.621269+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:03.621269+0000","last_peered":"2024-09-10T15:56:03.621269+0000","last_clean":"2024-09-10T15:56:03.621269+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:03.621269+0000","last_undegraded":"2024-09-10T15:56:03.621269+0000","last_fullsized":"2024-09-10T15:56:03.621269+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38789120,"num_objects":9470,"num_object_clones":0,"num_object_copies":18940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6231,"num_write_kb":24924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9450","reported_seq":17577,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:03.882528+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:03.882528+0000","last_peered":"2024-09-10T15:56:03.882528+0000","last_clean":"2024-09-10T15:56:03.882528+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:03.882528+0000","last_undegraded":"2024-09-10T15:56:03.882528+0000","last_fullsized":"2024-09-10T15:56:03.882528+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9450,"log_dups_size":0,"ondisk_log_size":9450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38678528,"num_objects":9443,"num_object_clones":0,"num_object_copies":18886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7427,"num_write_kb":29708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9282","reported_seq":9408,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.561588+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:03.561588+0000","last_peered":"2024-09-10T15:56:03.561588+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:03.561588+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9282,"log_dups_size":0,"ondisk_log_size":9282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37986304,"num_objects":9274,"num_object_clones":0,"num_object_copies":18548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6146,"num_write_kb":24584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5411},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9472","reported_seq":9596,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:04.679806+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:04.679806+0000","last_peered":"2024-09-10T15:56:04.679806+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:04.679806+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38776832,"num_objects":9467,"num_object_clones":0,"num_object_copies":18934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7446,"num_write_kb":29784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5635}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9324","reported_seq":9450,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:04.688009+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:04.688009+0000","last_peered":"2024-09-10T15:56:04.688009+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:04.688009+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9324,"log_dups_size":0,"ondisk_log_size":9324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38166528,"num_objects":9318,"num_object_clones":0,"num_object_copies":18636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7193,"num_write_kb":28772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5489}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9462","reported_seq":9589,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.708943+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:03.708943+0000","last_peered":"2024-09-10T15:56:03.708943+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:03.708943+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9462,"log_dups_size":0,"ondisk_log_size":9462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38748160,"num_objects":9460,"num_object_clones":0,"num_object_copies":18920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6654,"num_write_kb":26616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5627},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9414","reported_seq":9577,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.685001+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:03.685001+0000","last_peered":"2024-09-10T15:56:03.685001+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:03.685001+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9414,"log_dups_size":0,"ondisk_log_size":9414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5832,"num_write_kb":23328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5489},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9437","reported_seq":9572,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.719248+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:03.719248+0000","last_peered":"2024-09-10T15:56:03.719248+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:03.719248+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9437,"log_dups_size":0,"ondisk_log_size":9437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38637568,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6043,"num_write_kb":24172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5556},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9612","reported_seq":9750,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.663237+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:03.663237+0000","last_peered":"2024-09-10T15:56:03.663237+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:03.663237+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9612,"log_dups_size":0,"ondisk_log_size":9612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39322083,"num_objects":9601,"num_object_clones":0,"num_object_copies":19202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6431,"num_write_kb":25724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5598},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":616395235,"num_objects":150488,"num_object_clones":0,"num_object_copies":300976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":150488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112090,"num_write_kb":448360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4731,"num_bytes_recovered":19369984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1252339712,"data_stored":1252256613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150569,"ondisk_log_size":150569,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626468,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504508,"kb_used_data":284076,"kb_used_omap":4,"kb_used_meta":220411,"kb_avail":93867332,"statfs":{"total":96636764160,"available":96120147968,"internally_reserved":0,"allocated":290893824,"data_stored":290555511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":225701550},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149751,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819060,"kb_used_data":512548,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93552780,"statfs":{"total":96636764160,"available":95798046720,"internally_reserved":0,"allocated":524849152,"data_stored":524583099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313847469},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,1,6,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700336,"kb_used_data":425760,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93671504,"statfs":{"total":96636764160,"available":95919620096,"internally_reserved":0,"allocated":435978240,"data_stored":435712187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":525393920,"data_stored":525390307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":290639872,"data_stored":290563999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":436305920,"data_stored":436302307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:07.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:56:07.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'9420", 'reported_seq': 9496, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:03.672546+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:56:03.672546+0000', 'last_peered': '2024-09-10T15:56:03.672546+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:56:03.672546+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 9420, 'log_dups_size': 0, 'ondisk_log_size': 9420, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38584320, 'num_objects': 9420, 'num_object_clones': 0, 'num_object_copies': 18840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9420, 'num_write_kb': 37680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5447}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:07.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:56:07.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'9390", 'reported_seq': 9513, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:04.592825+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:56:04.592825+0000', 'last_peered': '2024-09-10T15:56:04.592825+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:56:04.592825+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 9390, 'log_dups_size': 0, 'ondisk_log_size': 9390, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38445056, 'num_objects': 9386, 'num_object_clones': 0, 'num_object_copies': 18772, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9386, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7658, 'num_write_kb': 30632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 5584}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:07.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:56:07.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'9483", 'reported_seq': 11088, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:56:03.694436+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:56:03.694436+0000', 'last_peered': '2024-09-10T15:56:03.694436+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:56:03.694436+0000', 'last_undegraded': '2024-09-10T15:56:03.694436+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 9483, 'log_dups_size': 0, 'ondisk_log_size': 9483, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38821888, 'num_objects': 9478, 'num_object_clones': 0, 'num_object_copies': 18956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3133, 'num_objects_unfound': 0, 'num_objects_dirty': 9478, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6201, 'num_write_kb': 24804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 735, 'num_bytes_recovered': 3010560, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6345}, {'shards': '0,2', 'objects': 3133}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:07.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:56:07.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'9156", 'reported_seq': 9296, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:03.596031+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:56:03.596031+0000', 'last_peered': '2024-09-10T15:56:03.596031+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:56:03.596031+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 9156, 'log_dups_size': 0, 'ondisk_log_size': 9156, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37478400, 'num_objects': 9150, 'num_object_clones': 0, 'num_object_copies': 18300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9150, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5806, 'num_write_kb': 23224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5316}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:07.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:56:07.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'9282", 'reported_seq': 9408, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:03.561588+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:56:03.561588+0000', 'last_peered': '2024-09-10T15:56:03.561588+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:56:03.561588+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 9282, 'log_dups_size': 0, 'ondisk_log_size': 9282, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37986304, 'num_objects': 9274, 'num_object_clones': 0, 'num_object_copies': 18548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9274, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6146, 'num_write_kb': 24584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5411}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:07.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:56:07.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'9472", 'reported_seq': 9596, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:04.679806+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:56:04.679806+0000', 'last_peered': '2024-09-10T15:56:04.679806+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:56:04.679806+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 9472, 'log_dups_size': 0, 'ondisk_log_size': 9472, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38776832, 'num_objects': 9467, 'num_object_clones': 0, 'num_object_copies': 18934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7446, 'num_write_kb': 29784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 5635}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:07.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:56:07.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'9324", 'reported_seq': 9450, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:04.688009+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:56:04.688009+0000', 'last_peered': '2024-09-10T15:56:04.688009+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:56:04.688009+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 9324, 'log_dups_size': 0, 'ondisk_log_size': 9324, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38166528, 'num_objects': 9318, 'num_object_clones': 0, 'num_object_copies': 18636, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9318, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7193, 'num_write_kb': 28772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 5489}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:07.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:56:07.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'9462", 'reported_seq': 9589, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:03.708943+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:56:03.708943+0000', 'last_peered': '2024-09-10T15:56:03.708943+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:56:03.708943+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 9462, 'log_dups_size': 0, 'ondisk_log_size': 9462, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38748160, 'num_objects': 9460, 'num_object_clones': 0, 'num_object_copies': 18920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9460, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6654, 'num_write_kb': 26616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5627}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:07.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:56:07.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'9414", 'reported_seq': 9577, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:03.685001+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:56:03.685001+0000', 'last_peered': '2024-09-10T15:56:03.685001+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:56:03.685001+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 9414, 'log_dups_size': 0, 'ondisk_log_size': 9414, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38526976, 'num_objects': 9406, 'num_object_clones': 0, 'num_object_copies': 18812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5832, 'num_write_kb': 23328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5489}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:07.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:56:07.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'9437", 'reported_seq': 9572, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:03.719248+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:56:03.719248+0000', 'last_peered': '2024-09-10T15:56:03.719248+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:56:03.719248+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 9437, 'log_dups_size': 0, 'ondisk_log_size': 9437, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38637568, 'num_objects': 9433, 'num_object_clones': 0, 'num_object_copies': 18866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9433, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6043, 'num_write_kb': 24172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5556}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:07.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:56:07.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'9612", 'reported_seq': 9750, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:03.663237+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:56:03.663237+0000', 'last_peered': '2024-09-10T15:56:03.663237+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:56:03.663237+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 9612, 'log_dups_size': 0, 'ondisk_log_size': 9612, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39322083, 'num_objects': 9601, 'num_object_clones': 0, 'num_object_copies': 19202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9601, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6431, 'num_write_kb': 25724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5598}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:07.555 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:56:07.828 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 210 20 152270 152250 2.83161 1.71484 0.0411982 0.0275822 2024-09-10T15:56:08.266 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:08.267 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/300980 objects degraded (12.885%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":294,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":150490,"data_bytes":616395254,"bytes_used":2072477696,"bytes_avail":287837814784,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":300980,"degraded_ratio":0.12884909296298758,"misplaced_objects":3133,"misplaced_total":300980,"misplaced_ratio":0.010409329523556382,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":54934,"recovering_keys_per_sec":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"write_bytes_sec":1822743,"read_op_per_sec":0,"write_op_per_sec":445},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:56:08.292 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:56:08.293 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:08.828 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 211 20 152709 152689 2.82631 1.71484 0.0257848 0.0276355 2024-09-10T15:56:08.894 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:08.897 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:08.919 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-10T15:56:08.490111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616395254,"num_objects":150490,"num_object_clones":0,"num_object_copies":300980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":150490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112092,"num_write_kb":448362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4733,"num_bytes_recovered":19370003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150572,"ondisk_log_size":150572,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2023904,"kb_used_data":1222384,"kb_used_omap":12,"kb_used_meta":801459,"kb_avail":281091616,"statfs":{"total":289910292480,"available":287837814784,"internally_reserved":0,"allocated":1251721216,"data_stored":1250850797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":820694021},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,7,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-161,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4569,"num_write_kb":18276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004250"},"pg_stats":[{"pgid":"2.f","version":"46'9420","reported_seq":9496,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.672546+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:03.672546+0000","last_peered":"2024-09-10T15:56:03.672546+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:03.672546+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9420,"log_dups_size":0,"ondisk_log_size":9420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38584320,"num_objects":9420,"num_object_clones":0,"num_object_copies":18840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9420,"num_write_kb":37680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5447},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9609","reported_seq":9716,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:04.702833+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:04.702833+0000","last_peered":"2024-09-10T15:56:04.702833+0000","last_clean":"2024-09-10T15:56:04.702833+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:04.702833+0000","last_undegraded":"2024-09-10T15:56:04.702833+0000","last_fullsized":"2024-09-10T15:56:04.702833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9609,"log_dups_size":0,"ondisk_log_size":9609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39333888,"num_objects":9603,"num_object_clones":0,"num_object_copies":19206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7945,"num_write_kb":31780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9390","reported_seq":9513,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:04.592825+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:04.592825+0000","last_peered":"2024-09-10T15:56:04.592825+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:04.592825+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9390,"log_dups_size":0,"ondisk_log_size":9390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38445056,"num_objects":9386,"num_object_clones":0,"num_object_copies":18772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7658,"num_write_kb":30632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5584}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9373","reported_seq":9452,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:04.721788+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:04.721788+0000","last_peered":"2024-09-10T15:56:04.721788+0000","last_clean":"2024-09-10T15:56:04.721788+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:04.721788+0000","last_undegraded":"2024-09-10T15:56:04.721788+0000","last_fullsized":"2024-09-10T15:56:04.721788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9373,"log_dups_size":0,"ondisk_log_size":9373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38391808,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9373,"num_write_kb":37492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9213","reported_seq":9338,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:03.648566+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:03.648566+0000","last_peered":"2024-09-10T15:56:03.648566+0000","last_clean":"2024-09-10T15:56:03.648566+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:03.648566+0000","last_undegraded":"2024-09-10T15:56:03.648566+0000","last_fullsized":"2024-09-10T15:56:03.648566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9213,"log_dups_size":0,"ondisk_log_size":9213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37707776,"num_objects":9206,"num_object_clones":0,"num_object_copies":18412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6284,"num_write_kb":25136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9483","reported_seq":11088,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:03.694436+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:03.694436+0000","last_peered":"2024-09-10T15:56:03.694436+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:03.694436+0000","last_undegraded":"2024-09-10T15:56:03.694436+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9483,"log_dups_size":0,"ondisk_log_size":9483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38821888,"num_objects":9478,"num_object_clones":0,"num_object_copies":18956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":9478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6201,"num_write_kb":24804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":735,"num_bytes_recovered":3010560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6345},{"shards":"0,2","objects":3133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9156","reported_seq":9296,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.596031+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:03.596031+0000","last_peered":"2024-09-10T15:56:03.596031+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:03.596031+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9156,"log_dups_size":0,"ondisk_log_size":9156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37478400,"num_objects":9150,"num_object_clones":0,"num_object_copies":18300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5806,"num_write_kb":23224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5316},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9472","reported_seq":9603,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:03.621269+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:03.621269+0000","last_peered":"2024-09-10T15:56:03.621269+0000","last_clean":"2024-09-10T15:56:03.621269+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:03.621269+0000","last_undegraded":"2024-09-10T15:56:03.621269+0000","last_fullsized":"2024-09-10T15:56:03.621269+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38789120,"num_objects":9470,"num_object_clones":0,"num_object_copies":18940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6231,"num_write_kb":24924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9450","reported_seq":17577,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:03.882528+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:03.882528+0000","last_peered":"2024-09-10T15:56:03.882528+0000","last_clean":"2024-09-10T15:56:03.882528+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:03.882528+0000","last_undegraded":"2024-09-10T15:56:03.882528+0000","last_fullsized":"2024-09-10T15:56:03.882528+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9450,"log_dups_size":0,"ondisk_log_size":9450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38678528,"num_objects":9443,"num_object_clones":0,"num_object_copies":18886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7427,"num_write_kb":29708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9282","reported_seq":9408,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.561588+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:03.561588+0000","last_peered":"2024-09-10T15:56:03.561588+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:03.561588+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9282,"log_dups_size":0,"ondisk_log_size":9282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37986304,"num_objects":9274,"num_object_clones":0,"num_object_copies":18548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6146,"num_write_kb":24584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5411},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9472","reported_seq":9596,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:04.679806+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:04.679806+0000","last_peered":"2024-09-10T15:56:04.679806+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:04.679806+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38776832,"num_objects":9467,"num_object_clones":0,"num_object_copies":18934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7446,"num_write_kb":29784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5635}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9324","reported_seq":9450,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:04.688009+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:04.688009+0000","last_peered":"2024-09-10T15:56:04.688009+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:04.688009+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9324,"log_dups_size":0,"ondisk_log_size":9324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38166528,"num_objects":9318,"num_object_clones":0,"num_object_copies":18636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7193,"num_write_kb":28772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5489}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9462","reported_seq":9589,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.708943+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:03.708943+0000","last_peered":"2024-09-10T15:56:03.708943+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:03.708943+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9462,"log_dups_size":0,"ondisk_log_size":9462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38748160,"num_objects":9460,"num_object_clones":0,"num_object_copies":18920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6654,"num_write_kb":26616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5627},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9414","reported_seq":9577,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.685001+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:03.685001+0000","last_peered":"2024-09-10T15:56:03.685001+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:03.685001+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9414,"log_dups_size":0,"ondisk_log_size":9414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5832,"num_write_kb":23328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5489},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9437","reported_seq":9572,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.719248+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:03.719248+0000","last_peered":"2024-09-10T15:56:03.719248+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:03.719248+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9437,"log_dups_size":0,"ondisk_log_size":9437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38637568,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6043,"num_write_kb":24172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5556},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9612","reported_seq":9750,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:03.663237+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:03.663237+0000","last_peered":"2024-09-10T15:56:03.663237+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:03.663237+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9612,"log_dups_size":0,"ondisk_log_size":9612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39322083,"num_objects":9601,"num_object_clones":0,"num_object_copies":19202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6431,"num_write_kb":25724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5598},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":616395235,"num_objects":150488,"num_object_clones":0,"num_object_copies":300976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":150488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112090,"num_write_kb":448360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4731,"num_bytes_recovered":19369984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1252339712,"data_stored":1252256613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150569,"ondisk_log_size":150569,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626468,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504508,"kb_used_data":284076,"kb_used_omap":4,"kb_used_meta":220411,"kb_avail":93867332,"statfs":{"total":96636764160,"available":96120147968,"internally_reserved":0,"allocated":290893824,"data_stored":290555511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":225701550},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149751,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819060,"kb_used_data":512548,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93552780,"statfs":{"total":96636764160,"available":95798046720,"internally_reserved":0,"allocated":524849152,"data_stored":524583099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313847469},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,1,6,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700336,"kb_used_data":425760,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93671504,"statfs":{"total":96636764160,"available":95919620096,"internally_reserved":0,"allocated":435978240,"data_stored":435712187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":525393920,"data_stored":525390307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":290639872,"data_stored":290563999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":436305920,"data_stored":436302307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:09.828 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 212 20 153185 153165 2.82175 1.85938 0.0389222 0.0276794 2024-09-10T15:56:10.828 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 213 20 153675 153655 2.81749 1.91406 0.0249536 0.0277222 2024-09-10T15:56:11.828 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 214 20 154156 154136 2.8131 1.87891 0.0232706 0.0277633 2024-09-10T15:56:11.921 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:12.485 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:12.576 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:12.576 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-10T15:56:10.490951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":626020854,"num_objects":152840,"num_object_clones":0,"num_object_copies":305680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3052,"num_objects_unfound":0,"num_objects_dirty":152840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114442,"num_write_kb":457762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4814,"num_bytes_recovered":19701779,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152922,"ondisk_log_size":152922,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2039716,"kb_used_data":1240944,"kb_used_omap":12,"kb_used_meta":798707,"kb_avail":281075804,"statfs":{"total":289910292480,"available":287821623296,"internally_reserved":0,"allocated":1270726656,"data_stored":1269856237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":817875973},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,3,5,14,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28340224,"num_objects":6919,"num_object_clones":0,"num_object_copies":13838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-242,"num_objects_unfound":0,"num_objects_dirty":6919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6919,"num_write_kb":27676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":991232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004307"},"pg_stats":[{"pgid":"2.f","version":"46'9566","reported_seq":9642,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.774781+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:08.774781+0000","last_peered":"2024-09-10T15:56:08.774781+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:08.774781+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9566,"log_dups_size":0,"ondisk_log_size":9566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39182336,"num_objects":9566,"num_object_clones":0,"num_object_copies":19132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9566,"num_write_kb":38264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5593},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9745","reported_seq":9852,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:09.765077+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:09.765077+0000","last_peered":"2024-09-10T15:56:09.765077+0000","last_clean":"2024-09-10T15:56:09.765077+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:09.765077+0000","last_undegraded":"2024-09-10T15:56:09.765077+0000","last_fullsized":"2024-09-10T15:56:09.765077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9745,"log_dups_size":0,"ondisk_log_size":9745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8081,"num_write_kb":32324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9547","reported_seq":9669,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:09.780007+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:09.780007+0000","last_peered":"2024-09-10T15:56:09.780007+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:09.780007+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9547,"log_dups_size":0,"ondisk_log_size":9547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39088128,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7815,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5741}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9528","reported_seq":9607,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:09.684505+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:09.684505+0000","last_peered":"2024-09-10T15:56:09.684505+0000","last_clean":"2024-09-10T15:56:09.684505+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:09.684505+0000","last_undegraded":"2024-09-10T15:56:09.684505+0000","last_fullsized":"2024-09-10T15:56:09.684505+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9528,"log_dups_size":0,"ondisk_log_size":9528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39026688,"num_objects":9528,"num_object_clones":0,"num_object_copies":19056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9528,"num_write_kb":38112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9354","reported_seq":9480,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:08.780144+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:08.780144+0000","last_peered":"2024-09-10T15:56:08.780144+0000","last_clean":"2024-09-10T15:56:08.780144+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:08.780144+0000","last_undegraded":"2024-09-10T15:56:08.780144+0000","last_fullsized":"2024-09-10T15:56:08.780144+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9354,"log_dups_size":0,"ondisk_log_size":9354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38285312,"num_objects":9347,"num_object_clones":0,"num_object_copies":18694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6425,"num_write_kb":25700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9606","reported_seq":11373,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:08.714333+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:08.714333+0000","last_peered":"2024-09-10T15:56:08.714333+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:08.714333+0000","last_undegraded":"2024-09-10T15:56:08.714333+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9606,"log_dups_size":0,"ondisk_log_size":9606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39325696,"num_objects":9601,"num_object_clones":0,"num_object_copies":19202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3052,"num_objects_unfound":0,"num_objects_dirty":9601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6324,"num_write_kb":25296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":3342336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6549},{"shards":"0,2","objects":3052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9301","reported_seq":9441,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.736153+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:08.736153+0000","last_peered":"2024-09-10T15:56:08.736153+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:08.736153+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9301,"log_dups_size":0,"ondisk_log_size":9301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072320,"num_objects":9295,"num_object_clones":0,"num_object_copies":18590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5951,"num_write_kb":23804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5461},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9625","reported_seq":9756,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:08.605665+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:08.605665+0000","last_peered":"2024-09-10T15:56:08.605665+0000","last_clean":"2024-09-10T15:56:08.605665+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:08.605665+0000","last_undegraded":"2024-09-10T15:56:08.605665+0000","last_fullsized":"2024-09-10T15:56:08.605665+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9625,"log_dups_size":0,"ondisk_log_size":9625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39415808,"num_objects":9623,"num_object_clones":0,"num_object_copies":19246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6384,"num_write_kb":25536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9594","reported_seq":17721,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:08.845620+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:08.845620+0000","last_peered":"2024-09-10T15:56:08.845620+0000","last_clean":"2024-09-10T15:56:08.845620+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:08.845620+0000","last_undegraded":"2024-09-10T15:56:08.845620+0000","last_fullsized":"2024-09-10T15:56:08.845620+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9594,"log_dups_size":0,"ondisk_log_size":9594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39268352,"num_objects":9587,"num_object_clones":0,"num_object_copies":19174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7571,"num_write_kb":30284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9433","reported_seq":9559,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.706256+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:08.706256+0000","last_peered":"2024-09-10T15:56:08.706256+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:08.706256+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9433,"log_dups_size":0,"ondisk_log_size":9433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38604800,"num_objects":9425,"num_object_clones":0,"num_object_copies":18850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6297,"num_write_kb":25188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5562},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9639","reported_seq":9763,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:09.732765+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:09.732765+0000","last_peered":"2024-09-10T15:56:09.732765+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:09.732765+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9639,"log_dups_size":0,"ondisk_log_size":9639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39460864,"num_objects":9634,"num_object_clones":0,"num_object_copies":19268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7613,"num_write_kb":30452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9472","reported_seq":9598,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:09.762368+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:09.762368+0000","last_peered":"2024-09-10T15:56:09.762368+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:09.762368+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38772736,"num_objects":9466,"num_object_clones":0,"num_object_copies":18932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7341,"num_write_kb":29364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5637}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9623","reported_seq":9748,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.783438+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:08.783438+0000","last_peered":"2024-09-10T15:56:08.783438+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:08.783438+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9623,"log_dups_size":0,"ondisk_log_size":9623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6815,"num_write_kb":27260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5788},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9551","reported_seq":9714,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.743417+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:08.743417+0000","last_peered":"2024-09-10T15:56:08.743417+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:08.743417+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9551,"log_dups_size":0,"ondisk_log_size":9551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39088128,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5969,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5626},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9595","reported_seq":9734,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.694338+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:08.694338+0000","last_peered":"2024-09-10T15:56:08.694338+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:08.694338+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9595,"log_dups_size":0,"ondisk_log_size":9595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6201,"num_write_kb":24804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9740","reported_seq":9878,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.776726+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:08.776726+0000","last_peered":"2024-09-10T15:56:08.776726+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:08.776726+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9740,"log_dups_size":0,"ondisk_log_size":9740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39846371,"num_objects":9729,"num_object_clones":0,"num_object_copies":19458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5726},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":626020835,"num_objects":152838,"num_object_clones":0,"num_object_copies":305676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3052,"num_objects_unfound":0,"num_objects_dirty":152838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114440,"num_write_kb":457760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4812,"num_bytes_recovered":19701760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1271902208,"data_stored":1271819109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152919,"ondisk_log_size":152919,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517196,"kb_used_data":291196,"kb_used_omap":4,"kb_used_meta":225979,"kb_avail":93854644,"statfs":{"total":96636764160,"available":96107155456,"internally_reserved":0,"allocated":298184704,"data_stored":297846391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231403182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":830080,"kb_used_data":519404,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93541760,"statfs":{"total":96636764160,"available":95786762240,"internally_reserved":0,"allocated":531869696,"data_stored":531603643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318107309},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,9,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692440,"kb_used_data":430344,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93679400,"statfs":{"total":96636764160,"available":95927705600,"internally_reserved":0,"allocated":440672256,"data_stored":440406203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532529152,"data_stored":532525539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298131456,"data_stored":298055583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441241600,"data_stored":441237987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:12.577 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:12.828 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 215 20 154561 154541 2.80738 1.58203 0.0700675 0.0278194 2024-09-10T15:56:13.129 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:13.132 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:13.152 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-10T15:56:12.491684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":626020854,"num_objects":152840,"num_object_clones":0,"num_object_copies":305680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3052,"num_objects_unfound":0,"num_objects_dirty":152840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114442,"num_write_kb":457762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4814,"num_bytes_recovered":19701779,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152922,"ondisk_log_size":152922,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2039716,"kb_used_data":1240944,"kb_used_omap":12,"kb_used_meta":798707,"kb_avail":281075804,"statfs":{"total":289910292480,"available":287821623296,"internally_reserved":0,"allocated":1270726656,"data_stored":1269856237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":817875973},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,3,5,14,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18931712,"num_objects":4622,"num_object_clones":0,"num_object_copies":9244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-161,"num_objects_unfound":0,"num_objects_dirty":4622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4622,"num_write_kb":18488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004359"},"pg_stats":[{"pgid":"2.f","version":"46'9566","reported_seq":9642,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.774781+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:08.774781+0000","last_peered":"2024-09-10T15:56:08.774781+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:08.774781+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9566,"log_dups_size":0,"ondisk_log_size":9566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39182336,"num_objects":9566,"num_object_clones":0,"num_object_copies":19132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9566,"num_write_kb":38264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5593},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9745","reported_seq":9852,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:09.765077+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:09.765077+0000","last_peered":"2024-09-10T15:56:09.765077+0000","last_clean":"2024-09-10T15:56:09.765077+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:09.765077+0000","last_undegraded":"2024-09-10T15:56:09.765077+0000","last_fullsized":"2024-09-10T15:56:09.765077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9745,"log_dups_size":0,"ondisk_log_size":9745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8081,"num_write_kb":32324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9547","reported_seq":9669,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:09.780007+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:09.780007+0000","last_peered":"2024-09-10T15:56:09.780007+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:09.780007+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9547,"log_dups_size":0,"ondisk_log_size":9547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39088128,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7815,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5741}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9528","reported_seq":9607,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:09.684505+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:09.684505+0000","last_peered":"2024-09-10T15:56:09.684505+0000","last_clean":"2024-09-10T15:56:09.684505+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:09.684505+0000","last_undegraded":"2024-09-10T15:56:09.684505+0000","last_fullsized":"2024-09-10T15:56:09.684505+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9528,"log_dups_size":0,"ondisk_log_size":9528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39026688,"num_objects":9528,"num_object_clones":0,"num_object_copies":19056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9528,"num_write_kb":38112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9354","reported_seq":9480,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:08.780144+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:08.780144+0000","last_peered":"2024-09-10T15:56:08.780144+0000","last_clean":"2024-09-10T15:56:08.780144+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:08.780144+0000","last_undegraded":"2024-09-10T15:56:08.780144+0000","last_fullsized":"2024-09-10T15:56:08.780144+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9354,"log_dups_size":0,"ondisk_log_size":9354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38285312,"num_objects":9347,"num_object_clones":0,"num_object_copies":18694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6425,"num_write_kb":25700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9606","reported_seq":11373,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:08.714333+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:08.714333+0000","last_peered":"2024-09-10T15:56:08.714333+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:08.714333+0000","last_undegraded":"2024-09-10T15:56:08.714333+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9606,"log_dups_size":0,"ondisk_log_size":9606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39325696,"num_objects":9601,"num_object_clones":0,"num_object_copies":19202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3052,"num_objects_unfound":0,"num_objects_dirty":9601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6324,"num_write_kb":25296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":3342336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6549},{"shards":"0,2","objects":3052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9301","reported_seq":9441,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.736153+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:08.736153+0000","last_peered":"2024-09-10T15:56:08.736153+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:08.736153+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9301,"log_dups_size":0,"ondisk_log_size":9301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072320,"num_objects":9295,"num_object_clones":0,"num_object_copies":18590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5951,"num_write_kb":23804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5461},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9625","reported_seq":9756,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:08.605665+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:08.605665+0000","last_peered":"2024-09-10T15:56:08.605665+0000","last_clean":"2024-09-10T15:56:08.605665+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:08.605665+0000","last_undegraded":"2024-09-10T15:56:08.605665+0000","last_fullsized":"2024-09-10T15:56:08.605665+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9625,"log_dups_size":0,"ondisk_log_size":9625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39415808,"num_objects":9623,"num_object_clones":0,"num_object_copies":19246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6384,"num_write_kb":25536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9594","reported_seq":17721,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:08.845620+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:08.845620+0000","last_peered":"2024-09-10T15:56:08.845620+0000","last_clean":"2024-09-10T15:56:08.845620+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:08.845620+0000","last_undegraded":"2024-09-10T15:56:08.845620+0000","last_fullsized":"2024-09-10T15:56:08.845620+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9594,"log_dups_size":0,"ondisk_log_size":9594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39268352,"num_objects":9587,"num_object_clones":0,"num_object_copies":19174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7571,"num_write_kb":30284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9433","reported_seq":9559,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.706256+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:08.706256+0000","last_peered":"2024-09-10T15:56:08.706256+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:08.706256+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9433,"log_dups_size":0,"ondisk_log_size":9433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38604800,"num_objects":9425,"num_object_clones":0,"num_object_copies":18850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6297,"num_write_kb":25188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5562},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9639","reported_seq":9763,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:09.732765+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:09.732765+0000","last_peered":"2024-09-10T15:56:09.732765+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:09.732765+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9639,"log_dups_size":0,"ondisk_log_size":9639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39460864,"num_objects":9634,"num_object_clones":0,"num_object_copies":19268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7613,"num_write_kb":30452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9472","reported_seq":9598,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:09.762368+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:09.762368+0000","last_peered":"2024-09-10T15:56:09.762368+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:09.762368+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38772736,"num_objects":9466,"num_object_clones":0,"num_object_copies":18932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7341,"num_write_kb":29364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5637}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9623","reported_seq":9748,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.783438+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:08.783438+0000","last_peered":"2024-09-10T15:56:08.783438+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:08.783438+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9623,"log_dups_size":0,"ondisk_log_size":9623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6815,"num_write_kb":27260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5788},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9551","reported_seq":9714,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.743417+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:08.743417+0000","last_peered":"2024-09-10T15:56:08.743417+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:08.743417+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9551,"log_dups_size":0,"ondisk_log_size":9551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39088128,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5969,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5626},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9595","reported_seq":9734,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.694338+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:08.694338+0000","last_peered":"2024-09-10T15:56:08.694338+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:08.694338+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9595,"log_dups_size":0,"ondisk_log_size":9595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6201,"num_write_kb":24804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9740","reported_seq":9878,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.776726+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:08.776726+0000","last_peered":"2024-09-10T15:56:08.776726+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:08.776726+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9740,"log_dups_size":0,"ondisk_log_size":9740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39846371,"num_objects":9729,"num_object_clones":0,"num_object_copies":19458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5726},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":626020835,"num_objects":152838,"num_object_clones":0,"num_object_copies":305676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3052,"num_objects_unfound":0,"num_objects_dirty":152838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114440,"num_write_kb":457760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4812,"num_bytes_recovered":19701760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1271902208,"data_stored":1271819109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152919,"ondisk_log_size":152919,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517196,"kb_used_data":291196,"kb_used_omap":4,"kb_used_meta":225979,"kb_avail":93854644,"statfs":{"total":96636764160,"available":96107155456,"internally_reserved":0,"allocated":298184704,"data_stored":297846391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231403182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":830080,"kb_used_data":519404,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93541760,"statfs":{"total":96636764160,"available":95786762240,"internally_reserved":0,"allocated":531869696,"data_stored":531603643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318107309},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,9,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692440,"kb_used_data":430344,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93679400,"statfs":{"total":96636764160,"available":95927705600,"internally_reserved":0,"allocated":440672256,"data_stored":440406203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532529152,"data_stored":532525539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298131456,"data_stored":298055583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441241600,"data_stored":441237987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:13.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:56:13.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'9566", 'reported_seq': 9642, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:08.774781+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:56:08.774781+0000', 'last_peered': '2024-09-10T15:56:08.774781+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:56:08.774781+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 9566, 'log_dups_size': 0, 'ondisk_log_size': 9566, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39182336, 'num_objects': 9566, 'num_object_clones': 0, 'num_object_copies': 19132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9566, 'num_write_kb': 38264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5593}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:13.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:56:13.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'9547", 'reported_seq': 9669, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:09.780007+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:56:09.780007+0000', 'last_peered': '2024-09-10T15:56:09.780007+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:56:09.780007+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 9547, 'log_dups_size': 0, 'ondisk_log_size': 9547, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39088128, 'num_objects': 9543, 'num_object_clones': 0, 'num_object_copies': 19086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9543, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7815, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 5741}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:13.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:56:13.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'9606", 'reported_seq': 11373, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:56:08.714333+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:56:08.714333+0000', 'last_peered': '2024-09-10T15:56:08.714333+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:56:08.714333+0000', 'last_undegraded': '2024-09-10T15:56:08.714333+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 9606, 'log_dups_size': 0, 'ondisk_log_size': 9606, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39325696, 'num_objects': 9601, 'num_object_clones': 0, 'num_object_copies': 19202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3052, 'num_objects_unfound': 0, 'num_objects_dirty': 9601, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6324, 'num_write_kb': 25296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 816, 'num_bytes_recovered': 3342336, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6549}, {'shards': '0,2', 'objects': 3052}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:13.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:56:13.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'9301", 'reported_seq': 9441, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:08.736153+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:56:08.736153+0000', 'last_peered': '2024-09-10T15:56:08.736153+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:56:08.736153+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 9301, 'log_dups_size': 0, 'ondisk_log_size': 9301, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38072320, 'num_objects': 9295, 'num_object_clones': 0, 'num_object_copies': 18590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5951, 'num_write_kb': 23804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5461}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:13.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:56:13.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'9433", 'reported_seq': 9559, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:08.706256+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:56:08.706256+0000', 'last_peered': '2024-09-10T15:56:08.706256+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:56:08.706256+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 9433, 'log_dups_size': 0, 'ondisk_log_size': 9433, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38604800, 'num_objects': 9425, 'num_object_clones': 0, 'num_object_copies': 18850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9425, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6297, 'num_write_kb': 25188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5562}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:13.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:56:13.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'9639", 'reported_seq': 9763, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:09.732765+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:56:09.732765+0000', 'last_peered': '2024-09-10T15:56:09.732765+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:56:09.732765+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 9639, 'log_dups_size': 0, 'ondisk_log_size': 9639, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39460864, 'num_objects': 9634, 'num_object_clones': 0, 'num_object_copies': 19268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7613, 'num_write_kb': 30452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 5802}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:13.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:56:13.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'9472", 'reported_seq': 9598, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:09.762368+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:56:09.762368+0000', 'last_peered': '2024-09-10T15:56:09.762368+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:56:09.762368+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 9472, 'log_dups_size': 0, 'ondisk_log_size': 9472, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38772736, 'num_objects': 9466, 'num_object_clones': 0, 'num_object_copies': 18932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9466, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7341, 'num_write_kb': 29364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 5637}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:13.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:56:13.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'9623", 'reported_seq': 9748, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:08.783438+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:56:08.783438+0000', 'last_peered': '2024-09-10T15:56:08.783438+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:56:08.783438+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 9623, 'log_dups_size': 0, 'ondisk_log_size': 9623, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39407616, 'num_objects': 9621, 'num_object_clones': 0, 'num_object_copies': 19242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9621, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6815, 'num_write_kb': 27260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5788}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:13.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:56:13.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'9551", 'reported_seq': 9714, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:08.743417+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:56:08.743417+0000', 'last_peered': '2024-09-10T15:56:08.743417+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:56:08.743417+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 9551, 'log_dups_size': 0, 'ondisk_log_size': 9551, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39088128, 'num_objects': 9543, 'num_object_clones': 0, 'num_object_copies': 19086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9543, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5969, 'num_write_kb': 23876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5626}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:13.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:56:13.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'9595", 'reported_seq': 9734, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:08.694338+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:56:08.694338+0000', 'last_peered': '2024-09-10T15:56:08.694338+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:56:08.694338+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 9595, 'log_dups_size': 0, 'ondisk_log_size': 9595, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39284736, 'num_objects': 9591, 'num_object_clones': 0, 'num_object_copies': 19182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6201, 'num_write_kb': 24804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5714}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:13.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:56:13.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'9740", 'reported_seq': 9878, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:08.776726+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:56:08.776726+0000', 'last_peered': '2024-09-10T15:56:08.776726+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:56:08.776726+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 9740, 'log_dups_size': 0, 'ondisk_log_size': 9740, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39846371, 'num_objects': 9729, 'num_object_clones': 0, 'num_object_copies': 19458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9729, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6559, 'num_write_kb': 26236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5726}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:13.156 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:56:13.828 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 216 20 154989 154969 2.80212 1.67188 0.0390443 0.0278739 2024-09-10T15:56:13.881 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:13.881 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/305680 objects degraded (12.687%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":299,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152840,"data_bytes":626020854,"bytes_used":2088669184,"bytes_avail":287821623296,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":305680,"degraded_ratio":0.126867966500916,"misplaced_objects":3052,"misplaced_total":305680,"misplaced_ratio":0.0099842973043705839,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":54934,"recovering_keys_per_sec":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"write_bytes_sec":1577069,"read_op_per_sec":0,"write_op_per_sec":385},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:56:13.906 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:56:13.906 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:14.460 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:14.463 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:14.485 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-10T15:56:12.491684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":626020854,"num_objects":152840,"num_object_clones":0,"num_object_copies":305680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3052,"num_objects_unfound":0,"num_objects_dirty":152840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114442,"num_write_kb":457762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4814,"num_bytes_recovered":19701779,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152922,"ondisk_log_size":152922,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2039716,"kb_used_data":1240944,"kb_used_omap":12,"kb_used_meta":798707,"kb_avail":281075804,"statfs":{"total":289910292480,"available":287821623296,"internally_reserved":0,"allocated":1270726656,"data_stored":1269856237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":817875973},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,3,5,14,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18931712,"num_objects":4622,"num_object_clones":0,"num_object_copies":9244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-161,"num_objects_unfound":0,"num_objects_dirty":4622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4622,"num_write_kb":18488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004359"},"pg_stats":[{"pgid":"2.f","version":"46'9566","reported_seq":9642,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.774781+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:08.774781+0000","last_peered":"2024-09-10T15:56:08.774781+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:08.774781+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9566,"log_dups_size":0,"ondisk_log_size":9566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39182336,"num_objects":9566,"num_object_clones":0,"num_object_copies":19132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9566,"num_write_kb":38264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5593},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9745","reported_seq":9852,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:09.765077+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:09.765077+0000","last_peered":"2024-09-10T15:56:09.765077+0000","last_clean":"2024-09-10T15:56:09.765077+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:09.765077+0000","last_undegraded":"2024-09-10T15:56:09.765077+0000","last_fullsized":"2024-09-10T15:56:09.765077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9745,"log_dups_size":0,"ondisk_log_size":9745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8081,"num_write_kb":32324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9547","reported_seq":9669,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:09.780007+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:09.780007+0000","last_peered":"2024-09-10T15:56:09.780007+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:09.780007+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9547,"log_dups_size":0,"ondisk_log_size":9547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39088128,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7815,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5741}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9528","reported_seq":9607,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:09.684505+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:09.684505+0000","last_peered":"2024-09-10T15:56:09.684505+0000","last_clean":"2024-09-10T15:56:09.684505+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:09.684505+0000","last_undegraded":"2024-09-10T15:56:09.684505+0000","last_fullsized":"2024-09-10T15:56:09.684505+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9528,"log_dups_size":0,"ondisk_log_size":9528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39026688,"num_objects":9528,"num_object_clones":0,"num_object_copies":19056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9528,"num_write_kb":38112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9354","reported_seq":9480,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:08.780144+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:08.780144+0000","last_peered":"2024-09-10T15:56:08.780144+0000","last_clean":"2024-09-10T15:56:08.780144+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:08.780144+0000","last_undegraded":"2024-09-10T15:56:08.780144+0000","last_fullsized":"2024-09-10T15:56:08.780144+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9354,"log_dups_size":0,"ondisk_log_size":9354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38285312,"num_objects":9347,"num_object_clones":0,"num_object_copies":18694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6425,"num_write_kb":25700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9606","reported_seq":11373,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:08.714333+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:08.714333+0000","last_peered":"2024-09-10T15:56:08.714333+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:08.714333+0000","last_undegraded":"2024-09-10T15:56:08.714333+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9606,"log_dups_size":0,"ondisk_log_size":9606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39325696,"num_objects":9601,"num_object_clones":0,"num_object_copies":19202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3052,"num_objects_unfound":0,"num_objects_dirty":9601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6324,"num_write_kb":25296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":3342336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6549},{"shards":"0,2","objects":3052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9301","reported_seq":9441,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.736153+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:08.736153+0000","last_peered":"2024-09-10T15:56:08.736153+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:08.736153+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9301,"log_dups_size":0,"ondisk_log_size":9301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072320,"num_objects":9295,"num_object_clones":0,"num_object_copies":18590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5951,"num_write_kb":23804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5461},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9625","reported_seq":9756,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:08.605665+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:08.605665+0000","last_peered":"2024-09-10T15:56:08.605665+0000","last_clean":"2024-09-10T15:56:08.605665+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:08.605665+0000","last_undegraded":"2024-09-10T15:56:08.605665+0000","last_fullsized":"2024-09-10T15:56:08.605665+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9625,"log_dups_size":0,"ondisk_log_size":9625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39415808,"num_objects":9623,"num_object_clones":0,"num_object_copies":19246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6384,"num_write_kb":25536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9594","reported_seq":17721,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:08.845620+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:08.845620+0000","last_peered":"2024-09-10T15:56:08.845620+0000","last_clean":"2024-09-10T15:56:08.845620+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:08.845620+0000","last_undegraded":"2024-09-10T15:56:08.845620+0000","last_fullsized":"2024-09-10T15:56:08.845620+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9594,"log_dups_size":0,"ondisk_log_size":9594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39268352,"num_objects":9587,"num_object_clones":0,"num_object_copies":19174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7571,"num_write_kb":30284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9433","reported_seq":9559,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.706256+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:08.706256+0000","last_peered":"2024-09-10T15:56:08.706256+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:08.706256+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9433,"log_dups_size":0,"ondisk_log_size":9433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38604800,"num_objects":9425,"num_object_clones":0,"num_object_copies":18850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6297,"num_write_kb":25188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5562},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9639","reported_seq":9763,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:09.732765+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:09.732765+0000","last_peered":"2024-09-10T15:56:09.732765+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:09.732765+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9639,"log_dups_size":0,"ondisk_log_size":9639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39460864,"num_objects":9634,"num_object_clones":0,"num_object_copies":19268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7613,"num_write_kb":30452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9472","reported_seq":9598,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:09.762368+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:09.762368+0000","last_peered":"2024-09-10T15:56:09.762368+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:09.762368+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9472,"log_dups_size":0,"ondisk_log_size":9472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38772736,"num_objects":9466,"num_object_clones":0,"num_object_copies":18932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7341,"num_write_kb":29364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5637}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9623","reported_seq":9748,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.783438+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:08.783438+0000","last_peered":"2024-09-10T15:56:08.783438+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:08.783438+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9623,"log_dups_size":0,"ondisk_log_size":9623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6815,"num_write_kb":27260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5788},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9551","reported_seq":9714,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.743417+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:08.743417+0000","last_peered":"2024-09-10T15:56:08.743417+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:08.743417+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9551,"log_dups_size":0,"ondisk_log_size":9551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39088128,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5969,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5626},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9595","reported_seq":9734,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.694338+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:08.694338+0000","last_peered":"2024-09-10T15:56:08.694338+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:08.694338+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9595,"log_dups_size":0,"ondisk_log_size":9595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6201,"num_write_kb":24804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9740","reported_seq":9878,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:08.776726+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:08.776726+0000","last_peered":"2024-09-10T15:56:08.776726+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:08.776726+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9740,"log_dups_size":0,"ondisk_log_size":9740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39846371,"num_objects":9729,"num_object_clones":0,"num_object_copies":19458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5726},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":626020835,"num_objects":152838,"num_object_clones":0,"num_object_copies":305676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":3052,"num_objects_unfound":0,"num_objects_dirty":152838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114440,"num_write_kb":457760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4812,"num_bytes_recovered":19701760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1271902208,"data_stored":1271819109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152919,"ondisk_log_size":152919,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517196,"kb_used_data":291196,"kb_used_omap":4,"kb_used_meta":225979,"kb_avail":93854644,"statfs":{"total":96636764160,"available":96107155456,"internally_reserved":0,"allocated":298184704,"data_stored":297846391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231403182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":830080,"kb_used_data":519404,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93541760,"statfs":{"total":96636764160,"available":95786762240,"internally_reserved":0,"allocated":531869696,"data_stored":531603643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318107309},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,9,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692440,"kb_used_data":430344,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93679400,"statfs":{"total":96636764160,"available":95927705600,"internally_reserved":0,"allocated":440672256,"data_stored":440406203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532529152,"data_stored":532525539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298131456,"data_stored":298055583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441241600,"data_stored":441237987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:14.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 217 20 155437 155417 2.79727 1.75 0.0525412 0.0279219 2024-09-10T15:56:15.828 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 218 20 155929 155909 2.79326 1.92188 0.0712434 0.0279623 2024-09-10T15:56:16.828 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 219 20 156422 156402 2.78929 1.92578 0.0282706 0.0280021 2024-09-10T15:56:17.487 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:17.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:2024-09-10T15:56:17.828441+0000 min lat: 0.00198913 max lat: 2.66872 avg lat: 0.0280479 2024-09-10T15:56:17.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-10T15:56:17.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 220 20 156880 156860 2.78475 1.78906 0.0283609 0.0280479 2024-09-10T15:56:18.018 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:18.019 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:18.041 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-10T15:56:16.493180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":635458038,"num_objects":155144,"num_object_clones":0,"num_object_copies":310288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2976,"num_objects_unfound":0,"num_objects_dirty":155144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116746,"num_write_kb":466978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4890,"num_bytes_recovered":20013075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155226,"ondisk_log_size":155226,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2072116,"kb_used_data":1259460,"kb_used_omap":12,"kb_used_meta":812595,"kb_avail":281043404,"statfs":{"total":289910292480,"available":287788445696,"internally_reserved":0,"allocated":1289687040,"data_stored":1288816621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":832097285},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,10,5,8,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22069248,"num_objects":5388,"num_object_clones":0,"num_object_copies":10776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-157,"num_objects_unfound":0,"num_objects_dirty":5388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5388,"num_write_kb":21552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":643072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004541"},"pg_stats":[{"pgid":"2.f","version":"46'9701","reported_seq":9777,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.755500+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:13.755500+0000","last_peered":"2024-09-10T15:56:13.755500+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:13.755500+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9701,"log_dups_size":0,"ondisk_log_size":9701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39735296,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9701,"num_write_kb":38804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5728},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9893","reported_seq":10000,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:14.737681+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:14.737681+0000","last_peered":"2024-09-10T15:56:14.737681+0000","last_clean":"2024-09-10T15:56:14.737681+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:14.737681+0000","last_undegraded":"2024-09-10T15:56:14.737681+0000","last_fullsized":"2024-09-10T15:56:14.737681+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9893,"log_dups_size":0,"ondisk_log_size":9893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40497152,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8229,"num_write_kb":32916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9689","reported_seq":9809,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:14.774406+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:14.774406+0000","last_peered":"2024-09-10T15:56:14.774406+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:14.774406+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9689,"log_dups_size":0,"ondisk_log_size":9689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39669760,"num_objects":9685,"num_object_clones":0,"num_object_copies":19370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7957,"num_write_kb":31828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5883}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9674","reported_seq":9753,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:14.756554+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:14.756554+0000","last_peered":"2024-09-10T15:56:14.756554+0000","last_clean":"2024-09-10T15:56:14.756554+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:14.756554+0000","last_undegraded":"2024-09-10T15:56:14.756554+0000","last_fullsized":"2024-09-10T15:56:14.756554+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9674,"log_dups_size":0,"ondisk_log_size":9674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39624704,"num_objects":9674,"num_object_clones":0,"num_object_copies":19348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9674,"num_write_kb":38696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9487","reported_seq":9613,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:13.774890+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:13.774890+0000","last_peered":"2024-09-10T15:56:13.774890+0000","last_clean":"2024-09-10T15:56:13.774890+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:13.774890+0000","last_undegraded":"2024-09-10T15:56:13.774890+0000","last_fullsized":"2024-09-10T15:56:13.774890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9487,"log_dups_size":0,"ondisk_log_size":9487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38830080,"num_objects":9480,"num_object_clones":0,"num_object_copies":18960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6558,"num_write_kb":26232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9738","reported_seq":11657,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:13.844177+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:13.844177+0000","last_peered":"2024-09-10T15:56:13.844177+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:13.844177+0000","last_undegraded":"2024-09-10T15:56:13.844177+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9738,"log_dups_size":0,"ondisk_log_size":9738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39866368,"num_objects":9733,"num_object_clones":0,"num_object_copies":19466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2976,"num_objects_unfound":0,"num_objects_dirty":9733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":892,"num_bytes_recovered":3653632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6757},{"shards":"0,2","objects":2976}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9454","reported_seq":9594,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.787508+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:13.787508+0000","last_peered":"2024-09-10T15:56:13.787508+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:13.787508+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9454,"log_dups_size":0,"ondisk_log_size":9454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38699008,"num_objects":9448,"num_object_clones":0,"num_object_copies":18896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6104,"num_write_kb":24416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5614},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9766","reported_seq":9897,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:13.741431+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:13.741431+0000","last_peered":"2024-09-10T15:56:13.741431+0000","last_clean":"2024-09-10T15:56:13.741431+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:13.741431+0000","last_undegraded":"2024-09-10T15:56:13.741431+0000","last_fullsized":"2024-09-10T15:56:13.741431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9766,"log_dups_size":0,"ondisk_log_size":9766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39993344,"num_objects":9764,"num_object_clones":0,"num_object_copies":19528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6525,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9749","reported_seq":17876,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:14.039504+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:14.039504+0000","last_peered":"2024-09-10T15:56:14.039504+0000","last_clean":"2024-09-10T15:56:14.039504+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:14.039504+0000","last_undegraded":"2024-09-10T15:56:14.039504+0000","last_fullsized":"2024-09-10T15:56:14.039504+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9749,"log_dups_size":0,"ondisk_log_size":9749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903232,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7726,"num_write_kb":30904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9596","reported_seq":9722,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.858266+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:13.858266+0000","last_peered":"2024-09-10T15:56:13.858266+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:13.858266+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9596,"log_dups_size":0,"ondisk_log_size":9596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39272448,"num_objects":9588,"num_object_clones":0,"num_object_copies":19176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6460,"num_write_kb":25840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5725},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9774","reported_seq":9898,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:14.676344+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:14.676344+0000","last_peered":"2024-09-10T15:56:14.676344+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:14.676344+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9774,"log_dups_size":0,"ondisk_log_size":9774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40013824,"num_objects":9769,"num_object_clones":0,"num_object_copies":19538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7748,"num_write_kb":30992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5937}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9623","reported_seq":9749,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:14.763139+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:14.763139+0000","last_peered":"2024-09-10T15:56:14.763139+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:14.763139+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9623,"log_dups_size":0,"ondisk_log_size":9623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39391232,"num_objects":9617,"num_object_clones":0,"num_object_copies":19234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7492,"num_write_kb":29968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5788}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9760","reported_seq":9887,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.905117+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:13.905117+0000","last_peered":"2024-09-10T15:56:13.905117+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:13.905117+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9760,"log_dups_size":0,"ondisk_log_size":9760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39968768,"num_objects":9758,"num_object_clones":0,"num_object_copies":19516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6952,"num_write_kb":27808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5925},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9688","reported_seq":9850,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.837310+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:13.837310+0000","last_peered":"2024-09-10T15:56:13.837310+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:13.837310+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9688,"log_dups_size":0,"ondisk_log_size":9688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39649280,"num_objects":9680,"num_object_clones":0,"num_object_copies":19360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6106,"num_write_kb":24424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5763},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9745","reported_seq":9884,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.802422+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:13.802422+0000","last_peered":"2024-09-10T15:56:13.802422+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:13.802422+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9745,"log_dups_size":0,"ondisk_log_size":9745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6351,"num_write_kb":25404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5864},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9886","reported_seq":10023,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.863021+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:13.863021+0000","last_peered":"2024-09-10T15:56:13.863021+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:13.863021+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9886,"log_dups_size":0,"ondisk_log_size":9886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40444387,"num_objects":9875,"num_object_clones":0,"num_object_copies":19750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6705,"num_write_kb":26820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5872},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":635458019,"num_objects":155142,"num_object_clones":0,"num_object_copies":310284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2976,"num_objects_unfound":0,"num_objects_dirty":155142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116744,"num_write_kb":466976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4888,"num_bytes_recovered":20013056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1291075584,"data_stored":1290992485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155223,"ondisk_log_size":155223,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626470,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528488,"kb_used_data":298328,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93843352,"statfs":{"total":96636764160,"available":96095592448,"internally_reserved":0,"allocated":305487872,"data_stored":305149559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235663022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149753,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":842428,"kb_used_data":526188,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93529412,"statfs":{"total":96636764160,"available":95774117888,"internally_reserved":0,"allocated":538816512,"data_stored":538550459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":323808941},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,0,3,1,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701200,"kb_used_data":434944,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93670640,"statfs":{"total":96636764160,"available":95918735360,"internally_reserved":0,"allocated":445382656,"data_stored":445116603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":539586560,"data_stored":539582947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305557504,"data_stored":305481631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":445931520,"data_stored":445927907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:18.043 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:18.602 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:18.603 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:18.631 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-10T15:56:18.494023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":635458038,"num_objects":155144,"num_object_clones":0,"num_object_copies":310288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2976,"num_objects_unfound":0,"num_objects_dirty":155144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116746,"num_write_kb":466978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4890,"num_bytes_recovered":20013075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155226,"ondisk_log_size":155226,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2072116,"kb_used_data":1259460,"kb_used_omap":12,"kb_used_meta":812595,"kb_avail":281043404,"statfs":{"total":289910292480,"available":287788445696,"internally_reserved":0,"allocated":1289687040,"data_stored":1288816621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":832097285},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,10,5,8,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19062784,"num_objects":4654,"num_object_clones":0,"num_object_copies":9308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-157,"num_objects_unfound":0,"num_objects_dirty":4654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4654,"num_write_kb":18616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":643072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004675"},"pg_stats":[{"pgid":"2.f","version":"46'9701","reported_seq":9777,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.755500+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:13.755500+0000","last_peered":"2024-09-10T15:56:13.755500+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:13.755500+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9701,"log_dups_size":0,"ondisk_log_size":9701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39735296,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9701,"num_write_kb":38804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5728},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9893","reported_seq":10000,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:14.737681+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:14.737681+0000","last_peered":"2024-09-10T15:56:14.737681+0000","last_clean":"2024-09-10T15:56:14.737681+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:14.737681+0000","last_undegraded":"2024-09-10T15:56:14.737681+0000","last_fullsized":"2024-09-10T15:56:14.737681+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9893,"log_dups_size":0,"ondisk_log_size":9893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40497152,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8229,"num_write_kb":32916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9689","reported_seq":9809,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:14.774406+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:14.774406+0000","last_peered":"2024-09-10T15:56:14.774406+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:14.774406+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9689,"log_dups_size":0,"ondisk_log_size":9689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39669760,"num_objects":9685,"num_object_clones":0,"num_object_copies":19370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7957,"num_write_kb":31828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5883}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9674","reported_seq":9753,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:14.756554+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:14.756554+0000","last_peered":"2024-09-10T15:56:14.756554+0000","last_clean":"2024-09-10T15:56:14.756554+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:14.756554+0000","last_undegraded":"2024-09-10T15:56:14.756554+0000","last_fullsized":"2024-09-10T15:56:14.756554+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9674,"log_dups_size":0,"ondisk_log_size":9674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39624704,"num_objects":9674,"num_object_clones":0,"num_object_copies":19348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9674,"num_write_kb":38696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9487","reported_seq":9613,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:13.774890+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:13.774890+0000","last_peered":"2024-09-10T15:56:13.774890+0000","last_clean":"2024-09-10T15:56:13.774890+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:13.774890+0000","last_undegraded":"2024-09-10T15:56:13.774890+0000","last_fullsized":"2024-09-10T15:56:13.774890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9487,"log_dups_size":0,"ondisk_log_size":9487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38830080,"num_objects":9480,"num_object_clones":0,"num_object_copies":18960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6558,"num_write_kb":26232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9738","reported_seq":11657,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:13.844177+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:13.844177+0000","last_peered":"2024-09-10T15:56:13.844177+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:13.844177+0000","last_undegraded":"2024-09-10T15:56:13.844177+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9738,"log_dups_size":0,"ondisk_log_size":9738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39866368,"num_objects":9733,"num_object_clones":0,"num_object_copies":19466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2976,"num_objects_unfound":0,"num_objects_dirty":9733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":892,"num_bytes_recovered":3653632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6757},{"shards":"0,2","objects":2976}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9454","reported_seq":9594,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.787508+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:13.787508+0000","last_peered":"2024-09-10T15:56:13.787508+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:13.787508+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9454,"log_dups_size":0,"ondisk_log_size":9454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38699008,"num_objects":9448,"num_object_clones":0,"num_object_copies":18896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6104,"num_write_kb":24416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5614},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9766","reported_seq":9897,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:13.741431+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:13.741431+0000","last_peered":"2024-09-10T15:56:13.741431+0000","last_clean":"2024-09-10T15:56:13.741431+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:13.741431+0000","last_undegraded":"2024-09-10T15:56:13.741431+0000","last_fullsized":"2024-09-10T15:56:13.741431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9766,"log_dups_size":0,"ondisk_log_size":9766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39993344,"num_objects":9764,"num_object_clones":0,"num_object_copies":19528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6525,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9749","reported_seq":17876,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:14.039504+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:14.039504+0000","last_peered":"2024-09-10T15:56:14.039504+0000","last_clean":"2024-09-10T15:56:14.039504+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:14.039504+0000","last_undegraded":"2024-09-10T15:56:14.039504+0000","last_fullsized":"2024-09-10T15:56:14.039504+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9749,"log_dups_size":0,"ondisk_log_size":9749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903232,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7726,"num_write_kb":30904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9596","reported_seq":9722,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.858266+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:13.858266+0000","last_peered":"2024-09-10T15:56:13.858266+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:13.858266+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9596,"log_dups_size":0,"ondisk_log_size":9596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39272448,"num_objects":9588,"num_object_clones":0,"num_object_copies":19176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6460,"num_write_kb":25840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5725},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9774","reported_seq":9898,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:14.676344+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:14.676344+0000","last_peered":"2024-09-10T15:56:14.676344+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:14.676344+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9774,"log_dups_size":0,"ondisk_log_size":9774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40013824,"num_objects":9769,"num_object_clones":0,"num_object_copies":19538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7748,"num_write_kb":30992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5937}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9623","reported_seq":9749,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:14.763139+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:14.763139+0000","last_peered":"2024-09-10T15:56:14.763139+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:14.763139+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9623,"log_dups_size":0,"ondisk_log_size":9623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39391232,"num_objects":9617,"num_object_clones":0,"num_object_copies":19234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7492,"num_write_kb":29968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5788}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9760","reported_seq":9887,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.905117+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:13.905117+0000","last_peered":"2024-09-10T15:56:13.905117+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:13.905117+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9760,"log_dups_size":0,"ondisk_log_size":9760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39968768,"num_objects":9758,"num_object_clones":0,"num_object_copies":19516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6952,"num_write_kb":27808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5925},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9688","reported_seq":9850,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.837310+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:13.837310+0000","last_peered":"2024-09-10T15:56:13.837310+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:13.837310+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9688,"log_dups_size":0,"ondisk_log_size":9688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39649280,"num_objects":9680,"num_object_clones":0,"num_object_copies":19360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6106,"num_write_kb":24424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5763},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9745","reported_seq":9884,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.802422+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:13.802422+0000","last_peered":"2024-09-10T15:56:13.802422+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:13.802422+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9745,"log_dups_size":0,"ondisk_log_size":9745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6351,"num_write_kb":25404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5864},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9886","reported_seq":10023,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.863021+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:13.863021+0000","last_peered":"2024-09-10T15:56:13.863021+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:13.863021+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9886,"log_dups_size":0,"ondisk_log_size":9886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40444387,"num_objects":9875,"num_object_clones":0,"num_object_copies":19750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6705,"num_write_kb":26820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5872},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":635458019,"num_objects":155142,"num_object_clones":0,"num_object_copies":310284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2976,"num_objects_unfound":0,"num_objects_dirty":155142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116744,"num_write_kb":466976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4888,"num_bytes_recovered":20013056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1291075584,"data_stored":1290992485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155223,"ondisk_log_size":155223,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626470,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528488,"kb_used_data":298328,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93843352,"statfs":{"total":96636764160,"available":96095592448,"internally_reserved":0,"allocated":305487872,"data_stored":305149559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235663022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149753,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":842428,"kb_used_data":526188,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93529412,"statfs":{"total":96636764160,"available":95774117888,"internally_reserved":0,"allocated":538816512,"data_stored":538550459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":323808941},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,0,3,1,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701200,"kb_used_data":434944,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93670640,"statfs":{"total":96636764160,"available":95918735360,"internally_reserved":0,"allocated":445382656,"data_stored":445116603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":539586560,"data_stored":539582947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305557504,"data_stored":305481631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":445931520,"data_stored":445927907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:18.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:56:18.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'9701", 'reported_seq': 9777, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:13.755500+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:56:13.755500+0000', 'last_peered': '2024-09-10T15:56:13.755500+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:56:13.755500+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 9701, 'log_dups_size': 0, 'ondisk_log_size': 9701, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39735296, 'num_objects': 9701, 'num_object_clones': 0, 'num_object_copies': 19402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9701, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9701, 'num_write_kb': 38804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5728}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:18.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:56:18.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'9689", 'reported_seq': 9809, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:14.774406+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:56:14.774406+0000', 'last_peered': '2024-09-10T15:56:14.774406+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:56:14.774406+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 9689, 'log_dups_size': 0, 'ondisk_log_size': 9689, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39669760, 'num_objects': 9685, 'num_object_clones': 0, 'num_object_copies': 19370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9685, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7957, 'num_write_kb': 31828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 5883}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:18.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:56:18.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'9738", 'reported_seq': 11657, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:56:13.844177+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:56:13.844177+0000', 'last_peered': '2024-09-10T15:56:13.844177+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:56:13.844177+0000', 'last_undegraded': '2024-09-10T15:56:13.844177+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 9738, 'log_dups_size': 0, 'ondisk_log_size': 9738, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39866368, 'num_objects': 9733, 'num_object_clones': 0, 'num_object_copies': 19466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2976, 'num_objects_unfound': 0, 'num_objects_dirty': 9733, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6456, 'num_write_kb': 25824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 892, 'num_bytes_recovered': 3653632, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6757}, {'shards': '0,2', 'objects': 2976}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:18.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:56:18.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'9454", 'reported_seq': 9594, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:13.787508+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:56:13.787508+0000', 'last_peered': '2024-09-10T15:56:13.787508+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:56:13.787508+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 9454, 'log_dups_size': 0, 'ondisk_log_size': 9454, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38699008, 'num_objects': 9448, 'num_object_clones': 0, 'num_object_copies': 18896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6104, 'num_write_kb': 24416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5614}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:18.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:56:18.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'9596", 'reported_seq': 9722, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:13.858266+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:56:13.858266+0000', 'last_peered': '2024-09-10T15:56:13.858266+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:56:13.858266+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 9596, 'log_dups_size': 0, 'ondisk_log_size': 9596, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39272448, 'num_objects': 9588, 'num_object_clones': 0, 'num_object_copies': 19176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6460, 'num_write_kb': 25840, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5725}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:18.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:56:18.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'9774", 'reported_seq': 9898, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:14.676344+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:56:14.676344+0000', 'last_peered': '2024-09-10T15:56:14.676344+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:56:14.676344+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 9774, 'log_dups_size': 0, 'ondisk_log_size': 9774, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40013824, 'num_objects': 9769, 'num_object_clones': 0, 'num_object_copies': 19538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9769, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7748, 'num_write_kb': 30992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 5937}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:18.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:56:18.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'9623", 'reported_seq': 9749, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:14.763139+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:56:14.763139+0000', 'last_peered': '2024-09-10T15:56:14.763139+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:56:14.763139+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 9623, 'log_dups_size': 0, 'ondisk_log_size': 9623, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39391232, 'num_objects': 9617, 'num_object_clones': 0, 'num_object_copies': 19234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9617, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7492, 'num_write_kb': 29968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 5788}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:18.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:56:18.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'9760", 'reported_seq': 9887, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:13.905117+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:56:13.905117+0000', 'last_peered': '2024-09-10T15:56:13.905117+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:56:13.905117+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 9760, 'log_dups_size': 0, 'ondisk_log_size': 9760, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39968768, 'num_objects': 9758, 'num_object_clones': 0, 'num_object_copies': 19516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9758, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6952, 'num_write_kb': 27808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5925}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:18.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:56:18.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'9688", 'reported_seq': 9850, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:13.837310+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:56:13.837310+0000', 'last_peered': '2024-09-10T15:56:13.837310+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:56:13.837310+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 9688, 'log_dups_size': 0, 'ondisk_log_size': 9688, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39649280, 'num_objects': 9680, 'num_object_clones': 0, 'num_object_copies': 19360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9680, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6106, 'num_write_kb': 24424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5763}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:18.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:56:18.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'9745", 'reported_seq': 9884, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:13.802422+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:56:13.802422+0000', 'last_peered': '2024-09-10T15:56:13.802422+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:56:13.802422+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 9745, 'log_dups_size': 0, 'ondisk_log_size': 9745, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39899136, 'num_objects': 9741, 'num_object_clones': 0, 'num_object_copies': 19482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6351, 'num_write_kb': 25404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5864}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:18.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:56:18.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'9886", 'reported_seq': 10023, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:13.863021+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:56:13.863021+0000', 'last_peered': '2024-09-10T15:56:13.863021+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:56:13.863021+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 9886, 'log_dups_size': 0, 'ondisk_log_size': 9886, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40444387, 'num_objects': 9875, 'num_object_clones': 0, 'num_object_copies': 19750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9875, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6705, 'num_write_kb': 26820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5872}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:18.636 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:56:18.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 221 20 157303 157283 2.77962 1.65234 0.0190371 0.0280973 2024-09-10T15:56:19.341 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:19.341 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/310288 objects degraded (12.498%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":305,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":155144,"data_bytes":635458038,"bytes_used":2121846784,"bytes_avail":287788445696,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":310288,"degraded_ratio":0.12498388593822513,"misplaced_objects":2976,"misplaced_total":310288,"misplaced_ratio":0.0095910895684009692,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":53568,"recovering_keys_per_sec":0,"num_objects_recovered":157,"num_bytes_recovered":643072,"num_keys_recovered":0,"write_bytes_sec":1587946,"read_op_per_sec":0,"write_op_per_sec":387},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-10T15:54:58.468984+0000","services":{}},"progress_events":{}} 2024-09-10T15:56:19.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:56:19.362 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:19.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 222 20 157717 157697 2.77439 1.61719 0.0272536 0.028152 2024-09-10T15:56:19.929 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:19.931 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:19.958 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-10T15:56:18.494023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":635458038,"num_objects":155144,"num_object_clones":0,"num_object_copies":310288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2976,"num_objects_unfound":0,"num_objects_dirty":155144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116746,"num_write_kb":466978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4890,"num_bytes_recovered":20013075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155226,"ondisk_log_size":155226,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2072116,"kb_used_data":1259460,"kb_used_omap":12,"kb_used_meta":812595,"kb_avail":281043404,"statfs":{"total":289910292480,"available":287788445696,"internally_reserved":0,"allocated":1289687040,"data_stored":1288816621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":832097285},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,10,5,8,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19062784,"num_objects":4654,"num_object_clones":0,"num_object_copies":9308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-157,"num_objects_unfound":0,"num_objects_dirty":4654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4654,"num_write_kb":18616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":643072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004675"},"pg_stats":[{"pgid":"2.f","version":"46'9701","reported_seq":9777,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.755500+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:13.755500+0000","last_peered":"2024-09-10T15:56:13.755500+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:13.755500+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9701,"log_dups_size":0,"ondisk_log_size":9701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39735296,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9701,"num_write_kb":38804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5728},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9893","reported_seq":10000,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:14.737681+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:14.737681+0000","last_peered":"2024-09-10T15:56:14.737681+0000","last_clean":"2024-09-10T15:56:14.737681+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:14.737681+0000","last_undegraded":"2024-09-10T15:56:14.737681+0000","last_fullsized":"2024-09-10T15:56:14.737681+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":9893,"log_dups_size":0,"ondisk_log_size":9893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40497152,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8229,"num_write_kb":32916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9689","reported_seq":9809,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:14.774406+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:14.774406+0000","last_peered":"2024-09-10T15:56:14.774406+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:14.774406+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9689,"log_dups_size":0,"ondisk_log_size":9689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39669760,"num_objects":9685,"num_object_clones":0,"num_object_copies":19370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7957,"num_write_kb":31828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":5883}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9674","reported_seq":9753,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:14.756554+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:14.756554+0000","last_peered":"2024-09-10T15:56:14.756554+0000","last_clean":"2024-09-10T15:56:14.756554+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:14.756554+0000","last_undegraded":"2024-09-10T15:56:14.756554+0000","last_fullsized":"2024-09-10T15:56:14.756554+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9674,"log_dups_size":0,"ondisk_log_size":9674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39624704,"num_objects":9674,"num_object_clones":0,"num_object_copies":19348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9674,"num_write_kb":38696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9487","reported_seq":9613,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:13.774890+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:13.774890+0000","last_peered":"2024-09-10T15:56:13.774890+0000","last_clean":"2024-09-10T15:56:13.774890+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:13.774890+0000","last_undegraded":"2024-09-10T15:56:13.774890+0000","last_fullsized":"2024-09-10T15:56:13.774890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9487,"log_dups_size":0,"ondisk_log_size":9487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38830080,"num_objects":9480,"num_object_clones":0,"num_object_copies":18960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6558,"num_write_kb":26232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9738","reported_seq":11657,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:13.844177+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:13.844177+0000","last_peered":"2024-09-10T15:56:13.844177+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:13.844177+0000","last_undegraded":"2024-09-10T15:56:13.844177+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9738,"log_dups_size":0,"ondisk_log_size":9738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39866368,"num_objects":9733,"num_object_clones":0,"num_object_copies":19466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2976,"num_objects_unfound":0,"num_objects_dirty":9733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":892,"num_bytes_recovered":3653632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6757},{"shards":"0,2","objects":2976}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9454","reported_seq":9594,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.787508+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:13.787508+0000","last_peered":"2024-09-10T15:56:13.787508+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:13.787508+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9454,"log_dups_size":0,"ondisk_log_size":9454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38699008,"num_objects":9448,"num_object_clones":0,"num_object_copies":18896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6104,"num_write_kb":24416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5614},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9766","reported_seq":9897,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:13.741431+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:13.741431+0000","last_peered":"2024-09-10T15:56:13.741431+0000","last_clean":"2024-09-10T15:56:13.741431+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:13.741431+0000","last_undegraded":"2024-09-10T15:56:13.741431+0000","last_fullsized":"2024-09-10T15:56:13.741431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9766,"log_dups_size":0,"ondisk_log_size":9766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39993344,"num_objects":9764,"num_object_clones":0,"num_object_copies":19528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6525,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9749","reported_seq":17876,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:14.039504+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:14.039504+0000","last_peered":"2024-09-10T15:56:14.039504+0000","last_clean":"2024-09-10T15:56:14.039504+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:14.039504+0000","last_undegraded":"2024-09-10T15:56:14.039504+0000","last_fullsized":"2024-09-10T15:56:14.039504+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9749,"log_dups_size":0,"ondisk_log_size":9749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903232,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7726,"num_write_kb":30904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9596","reported_seq":9722,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.858266+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:13.858266+0000","last_peered":"2024-09-10T15:56:13.858266+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:13.858266+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9596,"log_dups_size":0,"ondisk_log_size":9596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39272448,"num_objects":9588,"num_object_clones":0,"num_object_copies":19176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6460,"num_write_kb":25840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5725},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671759+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671759+0000","last_peered":"2024-09-10T15:55:20.671759+0000","last_clean":"2024-09-10T15:55:20.671759+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671759+0000","last_undegraded":"2024-09-10T15:55:20.671759+0000","last_fullsized":"2024-09-10T15:55:20.671759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9774","reported_seq":9898,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:14.676344+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:14.676344+0000","last_peered":"2024-09-10T15:56:14.676344+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:14.676344+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9774,"log_dups_size":0,"ondisk_log_size":9774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40013824,"num_objects":9769,"num_object_clones":0,"num_object_copies":19538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7748,"num_write_kb":30992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":5937}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9623","reported_seq":9749,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:14.763139+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:14.763139+0000","last_peered":"2024-09-10T15:56:14.763139+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:14.763139+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9623,"log_dups_size":0,"ondisk_log_size":9623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39391232,"num_objects":9617,"num_object_clones":0,"num_object_copies":19234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7492,"num_write_kb":29968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5788}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9760","reported_seq":9887,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.905117+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:13.905117+0000","last_peered":"2024-09-10T15:56:13.905117+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:13.905117+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9760,"log_dups_size":0,"ondisk_log_size":9760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39968768,"num_objects":9758,"num_object_clones":0,"num_object_copies":19516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6952,"num_write_kb":27808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5925},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9688","reported_seq":9850,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.837310+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:13.837310+0000","last_peered":"2024-09-10T15:56:13.837310+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:13.837310+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9688,"log_dups_size":0,"ondisk_log_size":9688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39649280,"num_objects":9680,"num_object_clones":0,"num_object_copies":19360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6106,"num_write_kb":24424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5763},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9745","reported_seq":9884,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.802422+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:13.802422+0000","last_peered":"2024-09-10T15:56:13.802422+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:13.802422+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9745,"log_dups_size":0,"ondisk_log_size":9745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6351,"num_write_kb":25404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5864},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9886","reported_seq":10023,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:13.863021+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:13.863021+0000","last_peered":"2024-09-10T15:56:13.863021+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:13.863021+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":9886,"log_dups_size":0,"ondisk_log_size":9886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40444387,"num_objects":9875,"num_object_clones":0,"num_object_copies":19750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6705,"num_write_kb":26820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5872},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":635458019,"num_objects":155142,"num_object_clones":0,"num_object_copies":310284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2976,"num_objects_unfound":0,"num_objects_dirty":155142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116744,"num_write_kb":466976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4888,"num_bytes_recovered":20013056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1291075584,"data_stored":1290992485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155223,"ondisk_log_size":155223,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626470,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528488,"kb_used_data":298328,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93843352,"statfs":{"total":96636764160,"available":96095592448,"internally_reserved":0,"allocated":305487872,"data_stored":305149559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235663022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149753,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":842428,"kb_used_data":526188,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93529412,"statfs":{"total":96636764160,"available":95774117888,"internally_reserved":0,"allocated":538816512,"data_stored":538550459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":323808941},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,0,3,1,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701200,"kb_used_data":434944,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93670640,"statfs":{"total":96636764160,"available":95918735360,"internally_reserved":0,"allocated":445382656,"data_stored":445116603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":539586560,"data_stored":539582947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305557504,"data_stored":305481631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":445931520,"data_stored":445927907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:20.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 223 20 158189 158169 2.77021 1.84375 0.0161125 0.0281939 2024-09-10T15:56:21.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 224 20 158673 158653 2.76628 1.89062 0.0213674 0.0282337 2024-09-10T15:56:22.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 225 20 159156 159136 2.76238 1.88672 0.0263561 0.0282743 2024-09-10T15:56:22.960 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:23.530 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:23.559 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-10T15:56:22.495414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644973046,"num_objects":157467,"num_object_clones":0,"num_object_copies":314934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2905,"num_objects_unfound":0,"num_objects_dirty":157467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119069,"num_write_kb":476270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4961,"num_bytes_recovered":20303891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157549,"ondisk_log_size":157549,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2104108,"kb_used_data":1278972,"kb_used_omap":12,"kb_used_meta":825075,"kb_avail":281011412,"statfs":{"total":289910292480,"available":287755685888,"internally_reserved":0,"allocated":1309667328,"data_stored":1308796909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":844876805},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,1,0,15,9,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18952192,"num_objects":4627,"num_object_clones":0,"num_object_copies":9254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-147,"num_objects_unfound":0,"num_objects_dirty":4627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4627,"num_write_kb":18508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":602112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004462"},"pg_stats":[{"pgid":"2.f","version":"46'9845","reported_seq":9921,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.882116+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:18.882116+0000","last_peered":"2024-09-10T15:56:18.882116+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:18.882116+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9845,"log_dups_size":0,"ondisk_log_size":9845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9845,"num_write_kb":39380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5872},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10028","reported_seq":10135,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:19.805350+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:19.805350+0000","last_peered":"2024-09-10T15:56:19.805350+0000","last_clean":"2024-09-10T15:56:19.805350+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:19.805350+0000","last_undegraded":"2024-09-10T15:56:19.805350+0000","last_fullsized":"2024-09-10T15:56:19.805350+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41050112,"num_objects":10022,"num_object_clones":0,"num_object_copies":20044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8364,"num_write_kb":33456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9826","reported_seq":9949,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:19.818510+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:19.818510+0000","last_peered":"2024-09-10T15:56:19.818510+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:19.818510+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9826,"log_dups_size":0,"ondisk_log_size":9826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40230912,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8094,"num_write_kb":32376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":6020}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9828","reported_seq":9907,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:19.763242+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:19.763242+0000","last_peered":"2024-09-10T15:56:19.763242+0000","last_clean":"2024-09-10T15:56:19.763242+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:19.763242+0000","last_undegraded":"2024-09-10T15:56:19.763242+0000","last_fullsized":"2024-09-10T15:56:19.763242+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9828,"log_dups_size":0,"ondisk_log_size":9828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9828,"num_write_kb":39312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9634","reported_seq":9760,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:18.906646+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:18.906646+0000","last_peered":"2024-09-10T15:56:18.906646+0000","last_clean":"2024-09-10T15:56:18.906646+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:18.906646+0000","last_undegraded":"2024-09-10T15:56:18.906646+0000","last_fullsized":"2024-09-10T15:56:18.906646+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9634,"log_dups_size":0,"ondisk_log_size":9634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39432192,"num_objects":9627,"num_object_clones":0,"num_object_copies":19254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6705,"num_write_kb":26820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9895","reported_seq":11956,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:18.893819+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:18.893819+0000","last_peered":"2024-09-10T15:56:18.893819+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:18.893819+0000","last_undegraded":"2024-09-10T15:56:18.893819+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9895,"log_dups_size":0,"ondisk_log_size":9895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40509440,"num_objects":9890,"num_object_clones":0,"num_object_copies":19780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2905,"num_objects_unfound":0,"num_objects_dirty":9890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6613,"num_write_kb":26452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":963,"num_bytes_recovered":3944448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6985},{"shards":"0,2","objects":2905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9597","reported_seq":9737,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.867145+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:18.867145+0000","last_peered":"2024-09-10T15:56:18.867145+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:18.867145+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9597,"log_dups_size":0,"ondisk_log_size":9597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6247,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5757},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9948","reported_seq":10079,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:18.808386+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:18.808386+0000","last_peered":"2024-09-10T15:56:18.808386+0000","last_clean":"2024-09-10T15:56:18.808386+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:18.808386+0000","last_undegraded":"2024-09-10T15:56:18.808386+0000","last_fullsized":"2024-09-10T15:56:18.808386+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9948,"log_dups_size":0,"ondisk_log_size":9948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40738816,"num_objects":9946,"num_object_clones":0,"num_object_copies":19892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6707,"num_write_kb":26828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9879","reported_seq":18006,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:18.974469+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:18.974469+0000","last_peered":"2024-09-10T15:56:18.974469+0000","last_clean":"2024-09-10T15:56:18.974469+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:18.974469+0000","last_undegraded":"2024-09-10T15:56:18.974469+0000","last_fullsized":"2024-09-10T15:56:18.974469+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9879,"log_dups_size":0,"ondisk_log_size":9879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40435712,"num_objects":9872,"num_object_clones":0,"num_object_copies":19744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7856,"num_write_kb":31424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9741","reported_seq":9867,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.896654+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:18.896654+0000","last_peered":"2024-09-10T15:56:18.896654+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:18.896654+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9741,"log_dups_size":0,"ondisk_log_size":9741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39866368,"num_objects":9733,"num_object_clones":0,"num_object_copies":19466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6605,"num_write_kb":26420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5870},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9907","reported_seq":10031,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:19.787232+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:19.787232+0000","last_peered":"2024-09-10T15:56:19.787232+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:19.787232+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9907,"log_dups_size":0,"ondisk_log_size":9907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40558592,"num_objects":9902,"num_object_clones":0,"num_object_copies":19804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7881,"num_write_kb":31524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":6070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9768","reported_seq":9894,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:19.780177+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:19.780177+0000","last_peered":"2024-09-10T15:56:19.780177+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:19.780177+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9768,"log_dups_size":0,"ondisk_log_size":9768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39985152,"num_objects":9762,"num_object_clones":0,"num_object_copies":19524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7637,"num_write_kb":30548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5933}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9910","reported_seq":10036,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.878697+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:18.878697+0000","last_peered":"2024-09-10T15:56:18.878697+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:18.878697+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40583168,"num_objects":9908,"num_object_clones":0,"num_object_copies":19816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7102,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6075},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9841","reported_seq":10004,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.738596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:18.738596+0000","last_peered":"2024-09-10T15:56:18.738596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:18.738596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9841,"log_dups_size":0,"ondisk_log_size":9841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40275968,"num_objects":9833,"num_object_clones":0,"num_object_copies":19666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6259,"num_write_kb":25036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5916},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9879","reported_seq":10018,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.852365+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:18.852365+0000","last_peered":"2024-09-10T15:56:18.852365+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:18.852365+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9879,"log_dups_size":0,"ondisk_log_size":9879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40448000,"num_objects":9875,"num_object_clones":0,"num_object_copies":19750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6485,"num_write_kb":25940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5998},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10020","reported_seq":10158,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.887132+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:18.887132+0000","last_peered":"2024-09-10T15:56:18.887132+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:18.887132+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40993251,"num_objects":10009,"num_object_clones":0,"num_object_copies":20018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6839,"num_write_kb":27356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6006},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":644973027,"num_objects":157465,"num_object_clones":0,"num_object_copies":314930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2905,"num_objects_unfound":0,"num_objects_dirty":157465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119067,"num_write_kb":476268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4959,"num_bytes_recovered":20303872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1310330880,"data_stored":1310247781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157546,"ondisk_log_size":157546,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539644,"kb_used_data":305324,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93832196,"statfs":{"total":96636764160,"available":96084168704,"internally_reserved":0,"allocated":312651776,"data_stored":312313463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239922862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149754,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":853632,"kb_used_data":533232,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93518208,"statfs":{"total":96636764160,"available":95762644992,"internally_reserved":0,"allocated":546029568,"data_stored":545763515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":328068781},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,8,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710832,"kb_used_data":440416,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93661008,"statfs":{"total":96636764160,"available":95908872192,"internally_reserved":0,"allocated":450985984,"data_stored":450719931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":546873344,"data_stored":546869731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312856576,"data_stored":312780703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":450600960,"data_stored":450597347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:23.560 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:23.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 226 20 159580 159560 2.75748 1.65625 0.0476499 0.0283238 2024-09-10T15:56:24.101 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:24.103 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:24.126 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-10T15:56:22.495414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644973046,"num_objects":157467,"num_object_clones":0,"num_object_copies":314934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2905,"num_objects_unfound":0,"num_objects_dirty":157467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119069,"num_write_kb":476270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4961,"num_bytes_recovered":20303891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157549,"ondisk_log_size":157549,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2104108,"kb_used_data":1278972,"kb_used_omap":12,"kb_used_meta":825075,"kb_avail":281011412,"statfs":{"total":289910292480,"available":287755685888,"internally_reserved":0,"allocated":1309667328,"data_stored":1308796909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":844876805},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,1,0,15,9,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18952192,"num_objects":4627,"num_object_clones":0,"num_object_copies":9254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-147,"num_objects_unfound":0,"num_objects_dirty":4627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4627,"num_write_kb":18508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":602112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004462"},"pg_stats":[{"pgid":"2.f","version":"46'9845","reported_seq":9921,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.882116+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:18.882116+0000","last_peered":"2024-09-10T15:56:18.882116+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:18.882116+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9845,"log_dups_size":0,"ondisk_log_size":9845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9845,"num_write_kb":39380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5872},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10028","reported_seq":10135,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:19.805350+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:19.805350+0000","last_peered":"2024-09-10T15:56:19.805350+0000","last_clean":"2024-09-10T15:56:19.805350+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:19.805350+0000","last_undegraded":"2024-09-10T15:56:19.805350+0000","last_fullsized":"2024-09-10T15:56:19.805350+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41050112,"num_objects":10022,"num_object_clones":0,"num_object_copies":20044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8364,"num_write_kb":33456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9826","reported_seq":9949,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:19.818510+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:19.818510+0000","last_peered":"2024-09-10T15:56:19.818510+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:19.818510+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9826,"log_dups_size":0,"ondisk_log_size":9826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40230912,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8094,"num_write_kb":32376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":6020}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9828","reported_seq":9907,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:19.763242+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:19.763242+0000","last_peered":"2024-09-10T15:56:19.763242+0000","last_clean":"2024-09-10T15:56:19.763242+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:19.763242+0000","last_undegraded":"2024-09-10T15:56:19.763242+0000","last_fullsized":"2024-09-10T15:56:19.763242+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9828,"log_dups_size":0,"ondisk_log_size":9828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9828,"num_write_kb":39312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9634","reported_seq":9760,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:18.906646+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:18.906646+0000","last_peered":"2024-09-10T15:56:18.906646+0000","last_clean":"2024-09-10T15:56:18.906646+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:18.906646+0000","last_undegraded":"2024-09-10T15:56:18.906646+0000","last_fullsized":"2024-09-10T15:56:18.906646+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9634,"log_dups_size":0,"ondisk_log_size":9634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39432192,"num_objects":9627,"num_object_clones":0,"num_object_copies":19254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6705,"num_write_kb":26820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9895","reported_seq":11956,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:18.893819+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:18.893819+0000","last_peered":"2024-09-10T15:56:18.893819+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:18.893819+0000","last_undegraded":"2024-09-10T15:56:18.893819+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":9895,"log_dups_size":0,"ondisk_log_size":9895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40509440,"num_objects":9890,"num_object_clones":0,"num_object_copies":19780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2905,"num_objects_unfound":0,"num_objects_dirty":9890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6613,"num_write_kb":26452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":963,"num_bytes_recovered":3944448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6985},{"shards":"0,2","objects":2905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9597","reported_seq":9737,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.867145+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:18.867145+0000","last_peered":"2024-09-10T15:56:18.867145+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:18.867145+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9597,"log_dups_size":0,"ondisk_log_size":9597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6247,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5757},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9948","reported_seq":10079,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:18.808386+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:18.808386+0000","last_peered":"2024-09-10T15:56:18.808386+0000","last_clean":"2024-09-10T15:56:18.808386+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:18.808386+0000","last_undegraded":"2024-09-10T15:56:18.808386+0000","last_fullsized":"2024-09-10T15:56:18.808386+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":9948,"log_dups_size":0,"ondisk_log_size":9948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40738816,"num_objects":9946,"num_object_clones":0,"num_object_copies":19892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6707,"num_write_kb":26828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9879","reported_seq":18006,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:18.974469+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:18.974469+0000","last_peered":"2024-09-10T15:56:18.974469+0000","last_clean":"2024-09-10T15:56:18.974469+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:18.974469+0000","last_undegraded":"2024-09-10T15:56:18.974469+0000","last_fullsized":"2024-09-10T15:56:18.974469+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":9879,"log_dups_size":0,"ondisk_log_size":9879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40435712,"num_objects":9872,"num_object_clones":0,"num_object_copies":19744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7856,"num_write_kb":31424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9741","reported_seq":9867,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.896654+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:18.896654+0000","last_peered":"2024-09-10T15:56:18.896654+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:18.896654+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9741,"log_dups_size":0,"ondisk_log_size":9741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39866368,"num_objects":9733,"num_object_clones":0,"num_object_copies":19466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6605,"num_write_kb":26420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5870},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9907","reported_seq":10031,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:19.787232+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:19.787232+0000","last_peered":"2024-09-10T15:56:19.787232+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:19.787232+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9907,"log_dups_size":0,"ondisk_log_size":9907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40558592,"num_objects":9902,"num_object_clones":0,"num_object_copies":19804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7881,"num_write_kb":31524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":6070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9768","reported_seq":9894,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:19.780177+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:19.780177+0000","last_peered":"2024-09-10T15:56:19.780177+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:19.780177+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9768,"log_dups_size":0,"ondisk_log_size":9768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39985152,"num_objects":9762,"num_object_clones":0,"num_object_copies":19524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7637,"num_write_kb":30548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5933}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9910","reported_seq":10036,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.878697+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:18.878697+0000","last_peered":"2024-09-10T15:56:18.878697+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:18.878697+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40583168,"num_objects":9908,"num_object_clones":0,"num_object_copies":19816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7102,"num_write_kb":28408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6075},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9841","reported_seq":10004,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.738596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:18.738596+0000","last_peered":"2024-09-10T15:56:18.738596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:18.738596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9841,"log_dups_size":0,"ondisk_log_size":9841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40275968,"num_objects":9833,"num_object_clones":0,"num_object_copies":19666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6259,"num_write_kb":25036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5916},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9879","reported_seq":10018,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.852365+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:18.852365+0000","last_peered":"2024-09-10T15:56:18.852365+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:18.852365+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":9879,"log_dups_size":0,"ondisk_log_size":9879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40448000,"num_objects":9875,"num_object_clones":0,"num_object_copies":19750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6485,"num_write_kb":25940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5998},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10020","reported_seq":10158,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:18.887132+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:18.887132+0000","last_peered":"2024-09-10T15:56:18.887132+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:18.887132+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40993251,"num_objects":10009,"num_object_clones":0,"num_object_copies":20018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6839,"num_write_kb":27356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6006},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":644973027,"num_objects":157465,"num_object_clones":0,"num_object_copies":314930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2905,"num_objects_unfound":0,"num_objects_dirty":157465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119067,"num_write_kb":476268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4959,"num_bytes_recovered":20303872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1310330880,"data_stored":1310247781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157546,"ondisk_log_size":157546,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539644,"kb_used_data":305324,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93832196,"statfs":{"total":96636764160,"available":96084168704,"internally_reserved":0,"allocated":312651776,"data_stored":312313463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239922862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149754,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":853632,"kb_used_data":533232,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93518208,"statfs":{"total":96636764160,"available":95762644992,"internally_reserved":0,"allocated":546029568,"data_stored":545763515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":328068781},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,8,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710832,"kb_used_data":440416,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93661008,"statfs":{"total":96636764160,"available":95908872192,"internally_reserved":0,"allocated":450985984,"data_stored":450719931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":546873344,"data_stored":546869731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312856576,"data_stored":312780703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":450600960,"data_stored":450597347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:24.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:56:24.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'9845", 'reported_seq': 9921, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:18.882116+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:56:18.882116+0000', 'last_peered': '2024-09-10T15:56:18.882116+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:56:18.882116+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 9845, 'log_dups_size': 0, 'ondisk_log_size': 9845, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40325120, 'num_objects': 9845, 'num_object_clones': 0, 'num_object_copies': 19690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9845, 'num_write_kb': 39380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5872}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:24.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:56:24.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'9826", 'reported_seq': 9949, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:19.818510+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:56:19.818510+0000', 'last_peered': '2024-09-10T15:56:19.818510+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:56:19.818510+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 9826, 'log_dups_size': 0, 'ondisk_log_size': 9826, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40230912, 'num_objects': 9822, 'num_object_clones': 0, 'num_object_copies': 19644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8094, 'num_write_kb': 32376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 6020}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:24.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:56:24.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'9895", 'reported_seq': 11956, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:56:18.893819+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:56:18.893819+0000', 'last_peered': '2024-09-10T15:56:18.893819+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:56:18.893819+0000', 'last_undegraded': '2024-09-10T15:56:18.893819+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 9895, 'log_dups_size': 0, 'ondisk_log_size': 9895, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40509440, 'num_objects': 9890, 'num_object_clones': 0, 'num_object_copies': 19780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2905, 'num_objects_unfound': 0, 'num_objects_dirty': 9890, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6613, 'num_write_kb': 26452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 963, 'num_bytes_recovered': 3944448, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6985}, {'shards': '0,2', 'objects': 2905}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:24.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:56:24.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'9597", 'reported_seq': 9737, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:18.867145+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:56:18.867145+0000', 'last_peered': '2024-09-10T15:56:18.867145+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:56:18.867145+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 9597, 'log_dups_size': 0, 'ondisk_log_size': 9597, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39284736, 'num_objects': 9591, 'num_object_clones': 0, 'num_object_copies': 19182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6247, 'num_write_kb': 24988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5757}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:24.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:56:24.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'9741", 'reported_seq': 9867, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:18.896654+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:56:18.896654+0000', 'last_peered': '2024-09-10T15:56:18.896654+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:56:18.896654+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 9741, 'log_dups_size': 0, 'ondisk_log_size': 9741, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39866368, 'num_objects': 9733, 'num_object_clones': 0, 'num_object_copies': 19466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9733, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6605, 'num_write_kb': 26420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5870}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:24.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:56:24.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'9907", 'reported_seq': 10031, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:19.787232+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:56:19.787232+0000', 'last_peered': '2024-09-10T15:56:19.787232+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:56:19.787232+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 9907, 'log_dups_size': 0, 'ondisk_log_size': 9907, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40558592, 'num_objects': 9902, 'num_object_clones': 0, 'num_object_copies': 19804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9902, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7881, 'num_write_kb': 31524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 6070}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:24.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:56:24.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'9768", 'reported_seq': 9894, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:19.780177+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:56:19.780177+0000', 'last_peered': '2024-09-10T15:56:19.780177+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:56:19.780177+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 9768, 'log_dups_size': 0, 'ondisk_log_size': 9768, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39985152, 'num_objects': 9762, 'num_object_clones': 0, 'num_object_copies': 19524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7637, 'num_write_kb': 30548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 5933}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:24.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:56:24.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'9910", 'reported_seq': 10036, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:18.878697+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:56:18.878697+0000', 'last_peered': '2024-09-10T15:56:18.878697+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:56:18.878697+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 9910, 'log_dups_size': 0, 'ondisk_log_size': 9910, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40583168, 'num_objects': 9908, 'num_object_clones': 0, 'num_object_copies': 19816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9908, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7102, 'num_write_kb': 28408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6075}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:24.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:56:24.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'9841", 'reported_seq': 10004, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:18.738596+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:56:18.738596+0000', 'last_peered': '2024-09-10T15:56:18.738596+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:56:18.738596+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 9841, 'log_dups_size': 0, 'ondisk_log_size': 9841, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40275968, 'num_objects': 9833, 'num_object_clones': 0, 'num_object_copies': 19666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9833, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6259, 'num_write_kb': 25036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5916}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:24.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:56:24.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'9879", 'reported_seq': 10018, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:18.852365+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:56:18.852365+0000', 'last_peered': '2024-09-10T15:56:18.852365+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:56:18.852365+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 9879, 'log_dups_size': 0, 'ondisk_log_size': 9879, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40448000, 'num_objects': 9875, 'num_object_clones': 0, 'num_object_copies': 19750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9875, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6485, 'num_write_kb': 25940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5998}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:24.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:56:24.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'10020", 'reported_seq': 10158, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:18.887132+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:56:18.887132+0000', 'last_peered': '2024-09-10T15:56:18.887132+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:56:18.887132+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40993251, 'num_objects': 10009, 'num_object_clones': 0, 'num_object_copies': 20018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10009, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6839, 'num_write_kb': 27356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6006}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:24.130 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:56:24.819 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:24.819 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/318160 objects degraded (12.189%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":310,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":159080,"data_bytes":651579894,"bytes_used":2177503232,"bytes_avail":287732789248,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":318160,"degraded_ratio":0.12189150113150618,"misplaced_objects":2829,"misplaced_total":318160,"misplaced_ratio":0.0088917525773195873,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":76088,"recovering_keys_per_sec":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"write_bytes_sec":2129116,"read_op_per_sec":0,"write_op_per_sec":519},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T15:56:24.496337+0000","services":{}},"progress_events":{}} 2024-09-10T15:56:24.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 227 20 160005 159985 2.75265 1.66016 0.0310546 0.0283741 2024-09-10T15:56:24.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:56:24.842 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:25.390 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:25.415 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-10T15:56:24.496211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651579894,"num_objects":159080,"num_object_clones":0,"num_object_copies":318160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2829,"num_objects_unfound":0,"num_objects_dirty":159080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120682,"num_write_kb":482722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5037,"num_bytes_recovered":20615187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159062,"ondisk_log_size":159062,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2126468,"kb_used_data":1293012,"kb_used_omap":12,"kb_used_meta":833395,"kb_avail":280989052,"statfs":{"total":289910292480,"available":287732789248,"internally_reserved":0,"allocated":1324044288,"data_stored":1323173869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":853396485},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,7,1,9,10,13,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25559040,"num_objects":6240,"num_object_clones":0,"num_object_copies":12480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-223,"num_objects_unfound":0,"num_objects_dirty":6240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6240,"num_write_kb":24960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004526"},"pg_stats":[{"pgid":"2.f","version":"46'9989","reported_seq":10065,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.950932+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:23.950932+0000","last_peered":"2024-09-10T15:56:23.950932+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:23.950932+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9989,"log_dups_size":0,"ondisk_log_size":9989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40914944,"num_objects":9989,"num_object_clones":0,"num_object_copies":19978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9989,"num_write_kb":39956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6016},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10028","reported_seq":10135,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:19.805350+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:19.805350+0000","last_peered":"2024-09-10T15:56:19.805350+0000","last_clean":"2024-09-10T15:56:19.805350+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:19.805350+0000","last_undegraded":"2024-09-10T15:56:19.805350+0000","last_fullsized":"2024-09-10T15:56:19.805350+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41050112,"num_objects":10022,"num_object_clones":0,"num_object_copies":20044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8364,"num_write_kb":33456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9826","reported_seq":9949,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:19.818510+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:19.818510+0000","last_peered":"2024-09-10T15:56:19.818510+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:19.818510+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9826,"log_dups_size":0,"ondisk_log_size":9826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40230912,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8094,"num_write_kb":32376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":6020}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9828","reported_seq":9907,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:19.763242+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:19.763242+0000","last_peered":"2024-09-10T15:56:19.763242+0000","last_clean":"2024-09-10T15:56:19.763242+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:19.763242+0000","last_undegraded":"2024-09-10T15:56:19.763242+0000","last_fullsized":"2024-09-10T15:56:19.763242+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9828,"log_dups_size":0,"ondisk_log_size":9828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9828,"num_write_kb":39312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9798","reported_seq":9924,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:23.958047+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:23.958047+0000","last_peered":"2024-09-10T15:56:23.958047+0000","last_clean":"2024-09-10T15:56:23.958047+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:23.958047+0000","last_undegraded":"2024-09-10T15:56:23.958047+0000","last_fullsized":"2024-09-10T15:56:23.958047+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9798,"log_dups_size":0,"ondisk_log_size":9798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40103936,"num_objects":9791,"num_object_clones":0,"num_object_copies":19582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6869,"num_write_kb":27476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10047","reported_seq":12259,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:23.995392+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:23.995392+0000","last_peered":"2024-09-10T15:56:23.995392+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:23.995392+0000","last_undegraded":"2024-09-10T15:56:23.995392+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41132032,"num_objects":10042,"num_object_clones":0,"num_object_copies":20084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2829,"num_objects_unfound":0,"num_objects_dirty":10042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6765,"num_write_kb":27060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":4255744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7213},{"shards":"0,2","objects":2829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9754","reported_seq":9894,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.909934+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:23.909934+0000","last_peered":"2024-09-10T15:56:23.909934+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:23.909934+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9754,"log_dups_size":0,"ondisk_log_size":9754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39927808,"num_objects":9748,"num_object_clones":0,"num_object_copies":19496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6404,"num_write_kb":25616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5914},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10089","reported_seq":10220,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:23.955729+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:23.955729+0000","last_peered":"2024-09-10T15:56:23.955729+0000","last_clean":"2024-09-10T15:56:23.955729+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:23.955729+0000","last_undegraded":"2024-09-10T15:56:23.955729+0000","last_fullsized":"2024-09-10T15:56:23.955729+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41316352,"num_objects":10087,"num_object_clones":0,"num_object_copies":20174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6848,"num_write_kb":27392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10012","reported_seq":18139,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:24.140203+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:24.140203+0000","last_peered":"2024-09-10T15:56:24.140203+0000","last_clean":"2024-09-10T15:56:24.140203+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:24.140203+0000","last_undegraded":"2024-09-10T15:56:24.140203+0000","last_fullsized":"2024-09-10T15:56:24.140203+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40980480,"num_objects":10005,"num_object_clones":0,"num_object_copies":20010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7989,"num_write_kb":31956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9901","reported_seq":10027,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.924058+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:23.924058+0000","last_peered":"2024-09-10T15:56:23.924058+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:23.924058+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9901,"log_dups_size":0,"ondisk_log_size":9901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40521728,"num_objects":9893,"num_object_clones":0,"num_object_copies":19786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6765,"num_write_kb":27060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6030},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9907","reported_seq":10031,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:19.787232+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:19.787232+0000","last_peered":"2024-09-10T15:56:19.787232+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:19.787232+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":9907,"log_dups_size":0,"ondisk_log_size":9907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40558592,"num_objects":9902,"num_object_clones":0,"num_object_copies":19804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7881,"num_write_kb":31524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":6070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9768","reported_seq":9894,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:19.780177+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:19.780177+0000","last_peered":"2024-09-10T15:56:19.780177+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:19.780177+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9768,"log_dups_size":0,"ondisk_log_size":9768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39985152,"num_objects":9762,"num_object_clones":0,"num_object_copies":19524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7637,"num_write_kb":30548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5933}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10051","reported_seq":10178,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.870762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:23.870762+0000","last_peered":"2024-09-10T15:56:23.870762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:23.870762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41160704,"num_objects":10049,"num_object_clones":0,"num_object_copies":20098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7243,"num_write_kb":28972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6216},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9991","reported_seq":10154,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.929042+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:23.929042+0000","last_peered":"2024-09-10T15:56:23.929042+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:23.929042+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9991,"log_dups_size":0,"ondisk_log_size":9991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40890368,"num_objects":9983,"num_object_clones":0,"num_object_copies":19966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6409,"num_write_kb":25636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6066},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10009","reported_seq":10148,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:24.023831+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:24.023831+0000","last_peered":"2024-09-10T15:56:24.023831+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:24.023831+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40980480,"num_objects":10005,"num_object_clones":0,"num_object_copies":20010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6615,"num_write_kb":26460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6128},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10161","reported_seq":10299,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.978583+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:23.978583+0000","last_peered":"2024-09-10T15:56:23.978583+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:23.978583+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41570787,"num_objects":10150,"num_object_clones":0,"num_object_copies":20300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6980,"num_write_kb":27920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6147},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":651579875,"num_objects":159078,"num_object_clones":0,"num_object_copies":318156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2829,"num_objects_unfound":0,"num_objects_dirty":159078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120680,"num_write_kb":482720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5035,"num_bytes_recovered":20615168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1324916736,"data_stored":1324833637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159059,"ondisk_log_size":159059,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626472,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550904,"kb_used_data":312424,"kb_used_omap":4,"kb_used_meta":238459,"kb_avail":93820936,"statfs":{"total":96636764160,"available":96072638464,"internally_reserved":0,"allocated":319922176,"data_stored":319583863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244182702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149755,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864732,"kb_used_data":540172,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93507108,"statfs":{"total":96636764160,"available":95751278592,"internally_reserved":0,"allocated":553136128,"data_stored":552870075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332328621},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,0,3,6,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710832,"kb_used_data":440416,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93661008,"statfs":{"total":96636764160,"available":95908872192,"internally_reserved":0,"allocated":450985984,"data_stored":450719931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554135552,"data_stored":554131939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320180224,"data_stored":320104351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":450600960,"data_stored":450597347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:25.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 228 20 160443 160423 2.74808 1.71094 0.0635208 0.0284226 2024-09-10T15:56:26.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 229 20 160927 160907 2.74433 1.89062 0.0298749 0.0284608 2024-09-10T15:56:27.830 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 230 20 161373 161353 2.73998 1.74219 0.0243067 0.0285058 2024-09-10T15:56:28.418 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:28.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 231 20 161831 161811 2.73586 1.78906 0.0900319 0.0285484 2024-09-10T15:56:28.979 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:28.981 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:29.006 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-10T15:56:28.497753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654410230,"num_objects":159771,"num_object_clones":0,"num_object_copies":319542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2829,"num_objects_unfound":0,"num_objects_dirty":159771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121373,"num_write_kb":485486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5037,"num_bytes_recovered":20615187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159653,"ondisk_log_size":159653,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2135180,"kb_used_data":1297560,"kb_used_omap":12,"kb_used_meta":837555,"kb_avail":280980340,"statfs":{"total":289910292480,"available":287723868160,"internally_reserved":0,"allocated":1328701440,"data_stored":1327831021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":857656325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,6,2,6,14,13,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18952192,"num_objects":4627,"num_object_clones":0,"num_object_copies":9254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-147,"num_objects_unfound":0,"num_objects_dirty":4627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4627,"num_write_kb":18508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":602112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004573"},"pg_stats":[{"pgid":"2.f","version":"46'9989","reported_seq":10065,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.950932+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:23.950932+0000","last_peered":"2024-09-10T15:56:23.950932+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:23.950932+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9989,"log_dups_size":0,"ondisk_log_size":9989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40914944,"num_objects":9989,"num_object_clones":0,"num_object_copies":19978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9989,"num_write_kb":39956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6016},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10172","reported_seq":10279,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:24.772092+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:24.772092+0000","last_peered":"2024-09-10T15:56:24.772092+0000","last_clean":"2024-09-10T15:56:24.772092+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:24.772092+0000","last_undegraded":"2024-09-10T15:56:24.772092+0000","last_fullsized":"2024-09-10T15:56:24.772092+0000","mapping_epoch":36,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41639936,"num_objects":10166,"num_object_clones":0,"num_object_copies":20332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8508,"num_write_kb":34032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9950","reported_seq":10069,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:24.815769+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:24.815769+0000","last_peered":"2024-09-10T15:56:24.815769+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:24.815769+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9950,"log_dups_size":0,"ondisk_log_size":9950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40738816,"num_objects":9946,"num_object_clones":0,"num_object_copies":19892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8218,"num_write_kb":32872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":6144}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9967","reported_seq":10046,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:24.824315+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:24.824315+0000","last_peered":"2024-09-10T15:56:24.824315+0000","last_clean":"2024-09-10T15:56:24.824315+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:24.824315+0000","last_undegraded":"2024-09-10T15:56:24.824315+0000","last_fullsized":"2024-09-10T15:56:24.824315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9967,"log_dups_size":0,"ondisk_log_size":9967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40824832,"num_objects":9967,"num_object_clones":0,"num_object_copies":19934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9967,"num_write_kb":39868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9798","reported_seq":9924,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:23.958047+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:23.958047+0000","last_peered":"2024-09-10T15:56:23.958047+0000","last_clean":"2024-09-10T15:56:23.958047+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:23.958047+0000","last_undegraded":"2024-09-10T15:56:23.958047+0000","last_fullsized":"2024-09-10T15:56:23.958047+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9798,"log_dups_size":0,"ondisk_log_size":9798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40103936,"num_objects":9791,"num_object_clones":0,"num_object_copies":19582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6869,"num_write_kb":27476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10047","reported_seq":12259,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:23.995392+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:23.995392+0000","last_peered":"2024-09-10T15:56:23.995392+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:23.995392+0000","last_undegraded":"2024-09-10T15:56:23.995392+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41132032,"num_objects":10042,"num_object_clones":0,"num_object_copies":20084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2829,"num_objects_unfound":0,"num_objects_dirty":10042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6765,"num_write_kb":27060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":4255744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7213},{"shards":"0,2","objects":2829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9754","reported_seq":9894,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.909934+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:23.909934+0000","last_peered":"2024-09-10T15:56:23.909934+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:23.909934+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9754,"log_dups_size":0,"ondisk_log_size":9754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39927808,"num_objects":9748,"num_object_clones":0,"num_object_copies":19496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6404,"num_write_kb":25616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5914},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10089","reported_seq":10220,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:23.955729+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:23.955729+0000","last_peered":"2024-09-10T15:56:23.955729+0000","last_clean":"2024-09-10T15:56:23.955729+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:23.955729+0000","last_undegraded":"2024-09-10T15:56:23.955729+0000","last_fullsized":"2024-09-10T15:56:23.955729+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41316352,"num_objects":10087,"num_object_clones":0,"num_object_copies":20174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6848,"num_write_kb":27392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10012","reported_seq":18139,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:24.140203+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:24.140203+0000","last_peered":"2024-09-10T15:56:24.140203+0000","last_clean":"2024-09-10T15:56:24.140203+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:24.140203+0000","last_undegraded":"2024-09-10T15:56:24.140203+0000","last_fullsized":"2024-09-10T15:56:24.140203+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40980480,"num_objects":10005,"num_object_clones":0,"num_object_copies":20010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7989,"num_write_kb":31956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9901","reported_seq":10027,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.924058+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:23.924058+0000","last_peered":"2024-09-10T15:56:23.924058+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:23.924058+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9901,"log_dups_size":0,"ondisk_log_size":9901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40521728,"num_objects":9893,"num_object_clones":0,"num_object_copies":19786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6765,"num_write_kb":27060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6030},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10046","reported_seq":10170,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:24.805448+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:24.805448+0000","last_peered":"2024-09-10T15:56:24.805448+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:24.805448+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41127936,"num_objects":10041,"num_object_clones":0,"num_object_copies":20082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8020,"num_write_kb":32080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":6209}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9913","reported_seq":10038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:24.792780+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:24.792780+0000","last_peered":"2024-09-10T15:56:24.792780+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:24.792780+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9913,"log_dups_size":0,"ondisk_log_size":9913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40579072,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7782,"num_write_kb":31128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":6078}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10051","reported_seq":10178,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.870762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:23.870762+0000","last_peered":"2024-09-10T15:56:23.870762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:23.870762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41160704,"num_objects":10049,"num_object_clones":0,"num_object_copies":20098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7243,"num_write_kb":28972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6216},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9991","reported_seq":10154,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.929042+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:23.929042+0000","last_peered":"2024-09-10T15:56:23.929042+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:23.929042+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9991,"log_dups_size":0,"ondisk_log_size":9991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40890368,"num_objects":9983,"num_object_clones":0,"num_object_copies":19966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6409,"num_write_kb":25636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6066},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10009","reported_seq":10148,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:24.023831+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:24.023831+0000","last_peered":"2024-09-10T15:56:24.023831+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:24.023831+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40980480,"num_objects":10005,"num_object_clones":0,"num_object_copies":20010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6615,"num_write_kb":26460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6128},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10161","reported_seq":10299,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.978583+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:23.978583+0000","last_peered":"2024-09-10T15:56:23.978583+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:23.978583+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41570787,"num_objects":10150,"num_object_clones":0,"num_object_copies":20300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6980,"num_write_kb":27920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6147},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":654410211,"num_objects":159769,"num_object_clones":0,"num_object_copies":319538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2829,"num_objects_unfound":0,"num_objects_dirty":159769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121371,"num_write_kb":485484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5035,"num_bytes_recovered":20615168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1329573888,"data_stored":1329490789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159650,"ondisk_log_size":159650,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626472,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550904,"kb_used_data":312424,"kb_used_omap":4,"kb_used_meta":238459,"kb_avail":93820936,"statfs":{"total":96636764160,"available":96072638464,"internally_reserved":0,"allocated":319922176,"data_stored":319583863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244182702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149755,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864732,"kb_used_data":540172,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93507108,"statfs":{"total":96636764160,"available":95751278592,"internally_reserved":0,"allocated":553136128,"data_stored":552870075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332328621},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,0,3,6,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719544,"kb_used_data":444964,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93652296,"statfs":{"total":96636764160,"available":95899951104,"internally_reserved":0,"allocated":455643136,"data_stored":455377083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554135552,"data_stored":554131939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320180224,"data_stored":320104351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":455258112,"data_stored":455254499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:29.007 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:29.580 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:29.583 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:29.602 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-10T15:56:28.497753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654410230,"num_objects":159771,"num_object_clones":0,"num_object_copies":319542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2829,"num_objects_unfound":0,"num_objects_dirty":159771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121373,"num_write_kb":485486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5037,"num_bytes_recovered":20615187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159653,"ondisk_log_size":159653,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2135180,"kb_used_data":1297560,"kb_used_omap":12,"kb_used_meta":837555,"kb_avail":280980340,"statfs":{"total":289910292480,"available":287723868160,"internally_reserved":0,"allocated":1328701440,"data_stored":1327831021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":857656325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,6,2,6,14,13,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18952192,"num_objects":4627,"num_object_clones":0,"num_object_copies":9254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-147,"num_objects_unfound":0,"num_objects_dirty":4627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4627,"num_write_kb":18508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":602112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004573"},"pg_stats":[{"pgid":"2.f","version":"46'9989","reported_seq":10065,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.950932+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:23.950932+0000","last_peered":"2024-09-10T15:56:23.950932+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:23.950932+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9989,"log_dups_size":0,"ondisk_log_size":9989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40914944,"num_objects":9989,"num_object_clones":0,"num_object_copies":19978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9989,"num_write_kb":39956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6016},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10172","reported_seq":10279,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:24.772092+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:24.772092+0000","last_peered":"2024-09-10T15:56:24.772092+0000","last_clean":"2024-09-10T15:56:24.772092+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:24.772092+0000","last_undegraded":"2024-09-10T15:56:24.772092+0000","last_fullsized":"2024-09-10T15:56:24.772092+0000","mapping_epoch":36,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41639936,"num_objects":10166,"num_object_clones":0,"num_object_copies":20332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8508,"num_write_kb":34032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9950","reported_seq":10069,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:24.815769+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:24.815769+0000","last_peered":"2024-09-10T15:56:24.815769+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:24.815769+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":9950,"log_dups_size":0,"ondisk_log_size":9950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40738816,"num_objects":9946,"num_object_clones":0,"num_object_copies":19892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8218,"num_write_kb":32872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":6144}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'9967","reported_seq":10046,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:24.824315+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:24.824315+0000","last_peered":"2024-09-10T15:56:24.824315+0000","last_clean":"2024-09-10T15:56:24.824315+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:24.824315+0000","last_undegraded":"2024-09-10T15:56:24.824315+0000","last_fullsized":"2024-09-10T15:56:24.824315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":9967,"log_dups_size":0,"ondisk_log_size":9967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40824832,"num_objects":9967,"num_object_clones":0,"num_object_copies":19934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9967,"num_write_kb":39868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9798","reported_seq":9924,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:23.958047+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:23.958047+0000","last_peered":"2024-09-10T15:56:23.958047+0000","last_clean":"2024-09-10T15:56:23.958047+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:23.958047+0000","last_undegraded":"2024-09-10T15:56:23.958047+0000","last_fullsized":"2024-09-10T15:56:23.958047+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9798,"log_dups_size":0,"ondisk_log_size":9798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40103936,"num_objects":9791,"num_object_clones":0,"num_object_copies":19582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6869,"num_write_kb":27476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10047","reported_seq":12259,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:23.995392+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:23.995392+0000","last_peered":"2024-09-10T15:56:23.995392+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:23.995392+0000","last_undegraded":"2024-09-10T15:56:23.995392+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41132032,"num_objects":10042,"num_object_clones":0,"num_object_copies":20084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2829,"num_objects_unfound":0,"num_objects_dirty":10042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6765,"num_write_kb":27060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1039,"num_bytes_recovered":4255744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7213},{"shards":"0,2","objects":2829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9754","reported_seq":9894,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.909934+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:23.909934+0000","last_peered":"2024-09-10T15:56:23.909934+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:23.909934+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9754,"log_dups_size":0,"ondisk_log_size":9754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39927808,"num_objects":9748,"num_object_clones":0,"num_object_copies":19496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6404,"num_write_kb":25616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5914},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10089","reported_seq":10220,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:23.955729+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:23.955729+0000","last_peered":"2024-09-10T15:56:23.955729+0000","last_clean":"2024-09-10T15:56:23.955729+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:23.955729+0000","last_undegraded":"2024-09-10T15:56:23.955729+0000","last_fullsized":"2024-09-10T15:56:23.955729+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41316352,"num_objects":10087,"num_object_clones":0,"num_object_copies":20174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6848,"num_write_kb":27392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10012","reported_seq":18139,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:24.140203+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:24.140203+0000","last_peered":"2024-09-10T15:56:24.140203+0000","last_clean":"2024-09-10T15:56:24.140203+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:24.140203+0000","last_undegraded":"2024-09-10T15:56:24.140203+0000","last_fullsized":"2024-09-10T15:56:24.140203+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40980480,"num_objects":10005,"num_object_clones":0,"num_object_copies":20010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7989,"num_write_kb":31956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9901","reported_seq":10027,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.924058+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:23.924058+0000","last_peered":"2024-09-10T15:56:23.924058+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:23.924058+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":9901,"log_dups_size":0,"ondisk_log_size":9901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40521728,"num_objects":9893,"num_object_clones":0,"num_object_copies":19786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6765,"num_write_kb":27060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6030},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10046","reported_seq":10170,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:24.805448+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:24.805448+0000","last_peered":"2024-09-10T15:56:24.805448+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:24.805448+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41127936,"num_objects":10041,"num_object_clones":0,"num_object_copies":20082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8020,"num_write_kb":32080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":6209}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9913","reported_seq":10038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:24.792780+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:24.792780+0000","last_peered":"2024-09-10T15:56:24.792780+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:24.792780+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":9913,"log_dups_size":0,"ondisk_log_size":9913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40579072,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7782,"num_write_kb":31128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":6078}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10051","reported_seq":10178,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.870762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:23.870762+0000","last_peered":"2024-09-10T15:56:23.870762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:23.870762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41160704,"num_objects":10049,"num_object_clones":0,"num_object_copies":20098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7243,"num_write_kb":28972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6216},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9991","reported_seq":10154,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.929042+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:23.929042+0000","last_peered":"2024-09-10T15:56:23.929042+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:23.929042+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":9991,"log_dups_size":0,"ondisk_log_size":9991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40890368,"num_objects":9983,"num_object_clones":0,"num_object_copies":19966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6409,"num_write_kb":25636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6066},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10009","reported_seq":10148,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:24.023831+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:24.023831+0000","last_peered":"2024-09-10T15:56:24.023831+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:24.023831+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40980480,"num_objects":10005,"num_object_clones":0,"num_object_copies":20010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6615,"num_write_kb":26460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6128},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10161","reported_seq":10299,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:23.978583+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:23.978583+0000","last_peered":"2024-09-10T15:56:23.978583+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:23.978583+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41570787,"num_objects":10150,"num_object_clones":0,"num_object_copies":20300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6980,"num_write_kb":27920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6147},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":654410211,"num_objects":159769,"num_object_clones":0,"num_object_copies":319538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2829,"num_objects_unfound":0,"num_objects_dirty":159769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121371,"num_write_kb":485484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5035,"num_bytes_recovered":20615168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1329573888,"data_stored":1329490789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159650,"ondisk_log_size":159650,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626472,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550904,"kb_used_data":312424,"kb_used_omap":4,"kb_used_meta":238459,"kb_avail":93820936,"statfs":{"total":96636764160,"available":96072638464,"internally_reserved":0,"allocated":319922176,"data_stored":319583863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244182702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149755,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864732,"kb_used_data":540172,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93507108,"statfs":{"total":96636764160,"available":95751278592,"internally_reserved":0,"allocated":553136128,"data_stored":552870075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332328621},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,0,3,6,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719544,"kb_used_data":444964,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93652296,"statfs":{"total":96636764160,"available":95899951104,"internally_reserved":0,"allocated":455643136,"data_stored":455377083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554135552,"data_stored":554131939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320180224,"data_stored":320104351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":455258112,"data_stored":455254499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:29.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:56:29.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'9989", 'reported_seq': 10065, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:23.950932+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:56:23.950932+0000', 'last_peered': '2024-09-10T15:56:23.950932+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:56:23.950932+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 9989, 'log_dups_size': 0, 'ondisk_log_size': 9989, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40914944, 'num_objects': 9989, 'num_object_clones': 0, 'num_object_copies': 19978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9989, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9989, 'num_write_kb': 39956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6016}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:29.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:56:29.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'9950", 'reported_seq': 10069, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:24.815769+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:56:24.815769+0000', 'last_peered': '2024-09-10T15:56:24.815769+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:56:24.815769+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 9950, 'log_dups_size': 0, 'ondisk_log_size': 9950, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40738816, 'num_objects': 9946, 'num_object_clones': 0, 'num_object_copies': 19892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9946, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8218, 'num_write_kb': 32872, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 6144}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:56:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'10047", 'reported_seq': 12259, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:56:23.995392+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:56:23.995392+0000', 'last_peered': '2024-09-10T15:56:23.995392+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:56:23.995392+0000', 'last_undegraded': '2024-09-10T15:56:23.995392+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41132032, 'num_objects': 10042, 'num_object_clones': 0, 'num_object_copies': 20084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2829, 'num_objects_unfound': 0, 'num_objects_dirty': 10042, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6765, 'num_write_kb': 27060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1039, 'num_bytes_recovered': 4255744, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7213}, {'shards': '0,2', 'objects': 2829}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:56:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'9754", 'reported_seq': 9894, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:23.909934+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:56:23.909934+0000', 'last_peered': '2024-09-10T15:56:23.909934+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:56:23.909934+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 9754, 'log_dups_size': 0, 'ondisk_log_size': 9754, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39927808, 'num_objects': 9748, 'num_object_clones': 0, 'num_object_copies': 19496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9748, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6404, 'num_write_kb': 25616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5914}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:56:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'9901", 'reported_seq': 10027, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:23.924058+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:56:23.924058+0000', 'last_peered': '2024-09-10T15:56:23.924058+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:56:23.924058+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 9901, 'log_dups_size': 0, 'ondisk_log_size': 9901, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40521728, 'num_objects': 9893, 'num_object_clones': 0, 'num_object_copies': 19786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6765, 'num_write_kb': 27060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6030}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:56:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'10046", 'reported_seq': 10170, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:24.805448+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:56:24.805448+0000', 'last_peered': '2024-09-10T15:56:24.805448+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:56:24.805448+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41127936, 'num_objects': 10041, 'num_object_clones': 0, 'num_object_copies': 20082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8020, 'num_write_kb': 32080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 6209}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:56:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'9913", 'reported_seq': 10038, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:24.792780+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:56:24.792780+0000', 'last_peered': '2024-09-10T15:56:24.792780+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:56:24.792780+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 9913, 'log_dups_size': 0, 'ondisk_log_size': 9913, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40579072, 'num_objects': 9907, 'num_object_clones': 0, 'num_object_copies': 19814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9907, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7782, 'num_write_kb': 31128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 6078}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:56:29.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'10051", 'reported_seq': 10178, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:23.870762+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:56:23.870762+0000', 'last_peered': '2024-09-10T15:56:23.870762+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:56:23.870762+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41160704, 'num_objects': 10049, 'num_object_clones': 0, 'num_object_copies': 20098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10049, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7243, 'num_write_kb': 28972, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6216}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:29.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:56:29.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'9991", 'reported_seq': 10154, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:23.929042+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:56:23.929042+0000', 'last_peered': '2024-09-10T15:56:23.929042+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:56:23.929042+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 9991, 'log_dups_size': 0, 'ondisk_log_size': 9991, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40890368, 'num_objects': 9983, 'num_object_clones': 0, 'num_object_copies': 19966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9983, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6409, 'num_write_kb': 25636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6066}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:29.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:56:29.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'10009", 'reported_seq': 10148, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:24.023831+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:56:24.023831+0000', 'last_peered': '2024-09-10T15:56:24.023831+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:56:24.023831+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40980480, 'num_objects': 10005, 'num_object_clones': 0, 'num_object_copies': 20010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10005, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6615, 'num_write_kb': 26460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6128}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:29.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:56:29.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'10161", 'reported_seq': 10299, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:23.978583+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:56:23.978583+0000', 'last_peered': '2024-09-10T15:56:23.978583+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:56:23.978583+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'100", 'ondisk_log_start': "37'100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41570787, 'num_objects': 10150, 'num_object_clones': 0, 'num_object_copies': 20300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10150, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6980, 'num_write_kb': 27920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6147}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:29.605 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:56:29.829 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 232 20 162243 162223 2.731 1.60938 0.0266702 0.0285983 2024-09-10T15:56:30.298 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:30.298 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/319542 objects degraded (12.136%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":316,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":159771,"data_bytes":654410230,"bytes_used":2186424320,"bytes_avail":287723868160,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":319542,"degraded_ratio":0.12136432769401205,"misplaced_objects":2829,"misplaced_total":319542,"misplaced_ratio":0.0088532962803011817,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":50156,"recovering_keys_per_sec":0,"num_objects_recovered":147,"num_bytes_recovered":602112,"num_keys_recovered":0,"write_bytes_sec":1578747,"read_op_per_sec":0,"write_op_per_sec":385},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T15:56:24.496337+0000","services":{}},"progress_events":{}} 2024-09-10T15:56:30.335 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:56:30.336 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:30.830 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 233 20 162655 162635 2.72619 1.60938 0.0180415 0.0286484 2024-09-10T15:56:30.913 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:30.913 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:30.938 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-10T15:56:30.498587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663675382,"num_objects":162033,"num_object_clones":0,"num_object_copies":324066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2748,"num_objects_unfound":0,"num_objects_dirty":162033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123635,"num_write_kb":494534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5118,"num_bytes_recovered":20946963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160515,"ondisk_log_size":160515,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2142688,"kb_used_data":1316144,"kb_used_omap":12,"kb_used_meta":826483,"kb_avail":280972832,"statfs":{"total":289910292480,"available":287716179968,"internally_reserved":0,"allocated":1347731456,"data_stored":1346861037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":846318597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,7,7,5,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28217344,"num_objects":6889,"num_object_clones":0,"num_object_copies":13778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-228,"num_objects_unfound":0,"num_objects_dirty":6889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6889,"num_write_kb":27556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":933888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004564"},"pg_stats":[{"pgid":"2.f","version":"46'10142","reported_seq":10218,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.102763+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:29.102763+0000","last_peered":"2024-09-10T15:56:29.102763+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:29.102763+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41541632,"num_objects":10142,"num_object_clones":0,"num_object_copies":20284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10142,"num_write_kb":40568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6169},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10320","reported_seq":10427,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.688425+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:29.688425+0000","last_peered":"2024-09-10T15:56:29.688425+0000","last_clean":"2024-09-10T15:56:29.688425+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:29.688425+0000","last_undegraded":"2024-09-10T15:56:29.688425+0000","last_fullsized":"2024-09-10T15:56:29.688425+0000","mapping_epoch":36,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42246144,"num_objects":10314,"num_object_clones":0,"num_object_copies":20628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8656,"num_write_kb":34624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10097","reported_seq":10220,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.769943+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:29.769943+0000","last_peered":"2024-09-10T15:56:29.769943+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:29.769943+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41340928,"num_objects":10093,"num_object_clones":0,"num_object_copies":20186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8365,"num_write_kb":33460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":6291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10119","reported_seq":10198,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.749985+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:29.749985+0000","last_peered":"2024-09-10T15:56:29.749985+0000","last_clean":"2024-09-10T15:56:29.749985+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:29.749985+0000","last_undegraded":"2024-09-10T15:56:29.749985+0000","last_fullsized":"2024-09-10T15:56:29.749985+0000","mapping_epoch":36,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41447424,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10119,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9947","reported_seq":10073,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.017119+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:29.017119+0000","last_peered":"2024-09-10T15:56:29.017119+0000","last_clean":"2024-09-10T15:56:29.017119+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:29.017119+0000","last_undegraded":"2024-09-10T15:56:29.017119+0000","last_fullsized":"2024-09-10T15:56:29.017119+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9947,"log_dups_size":0,"ondisk_log_size":9947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40714240,"num_objects":9940,"num_object_clones":0,"num_object_copies":19880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7018,"num_write_kb":28072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10186","reported_seq":12559,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:29.088289+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:29.088289+0000","last_peered":"2024-09-10T15:56:29.088289+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:29.088289+0000","last_undegraded":"2024-09-10T15:56:29.088289+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41701376,"num_objects":10181,"num_object_clones":0,"num_object_copies":20362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2748,"num_objects_unfound":0,"num_objects_dirty":10181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6904,"num_write_kb":27616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1120,"num_bytes_recovered":4587520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7433},{"shards":"0,2","objects":2748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9896","reported_seq":10036,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.068223+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:29.068223+0000","last_peered":"2024-09-10T15:56:29.068223+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:29.068223+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9896,"log_dups_size":0,"ondisk_log_size":9896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40509440,"num_objects":9890,"num_object_clones":0,"num_object_copies":19780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6546,"num_write_kb":26184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6056},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10234","reported_seq":10365,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.075204+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:29.075204+0000","last_peered":"2024-09-10T15:56:29.075204+0000","last_clean":"2024-09-10T15:56:29.075204+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:29.075204+0000","last_undegraded":"2024-09-10T15:56:29.075204+0000","last_fullsized":"2024-09-10T15:56:29.075204+0000","mapping_epoch":35,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41910272,"num_objects":10232,"num_object_clones":0,"num_object_copies":20464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6993,"num_write_kb":27972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10144","reported_seq":18270,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.179412+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:29.179412+0000","last_peered":"2024-09-10T15:56:29.179412+0000","last_clean":"2024-09-10T15:56:29.179412+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:29.179412+0000","last_undegraded":"2024-09-10T15:56:29.179412+0000","last_fullsized":"2024-09-10T15:56:29.179412+0000","mapping_epoch":45,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41521152,"num_objects":10137,"num_object_clones":0,"num_object_copies":20274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8121,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10051","reported_seq":10178,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.063081+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:29.063081+0000","last_peered":"2024-09-10T15:56:29.063081+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:29.063081+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41136128,"num_objects":10043,"num_object_clones":0,"num_object_copies":20086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6915,"num_write_kb":27660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6180},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10188","reported_seq":10311,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.842037+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:29.842037+0000","last_peered":"2024-09-10T15:56:29.842037+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:29.842037+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41709568,"num_objects":10183,"num_object_clones":0,"num_object_copies":20366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8162,"num_write_kb":32648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":6351}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10029","reported_seq":10155,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.751786+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:29.751786+0000","last_peered":"2024-09-10T15:56:29.751786+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:29.751786+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41054208,"num_objects":10023,"num_object_clones":0,"num_object_copies":20046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7898,"num_write_kb":31592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":6194}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10193","reported_seq":10320,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.101337+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:29.101337+0000","last_peered":"2024-09-10T15:56:29.101337+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:29.101337+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41742336,"num_objects":10191,"num_object_clones":0,"num_object_copies":20382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7385,"num_write_kb":29540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6358},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10126","reported_seq":10289,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.055706+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:29.055706+0000","last_peered":"2024-09-10T15:56:29.055706+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:29.055706+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41443328,"num_objects":10118,"num_object_clones":0,"num_object_copies":20236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6544,"num_write_kb":26176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6201},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10136","reported_seq":10275,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:28.984146+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:28.984146+0000","last_peered":"2024-09-10T15:56:28.984146+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:28.984146+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41500672,"num_objects":10132,"num_object_clones":0,"num_object_copies":20264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6742,"num_write_kb":26968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6255},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10304","reported_seq":10442,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:28.991070+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:28.991070+0000","last_peered":"2024-09-10T15:56:28.991070+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:28.991070+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42156515,"num_objects":10293,"num_object_clones":0,"num_object_copies":20586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7123,"num_write_kb":28492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6290},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":663675363,"num_objects":162031,"num_object_clones":0,"num_object_copies":324062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2748,"num_objects_unfound":0,"num_objects_dirty":162031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123633,"num_write_kb":494532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5116,"num_bytes_recovered":20946944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1348501504,"data_stored":1348418405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160512,"ondisk_log_size":160512,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626473,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549376,"kb_used_data":319216,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93822464,"statfs":{"total":96636764160,"available":96074203136,"internally_reserved":0,"allocated":326877184,"data_stored":326538871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235663022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149756,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":882556,"kb_used_data":548268,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93489284,"statfs":{"total":96636764160,"available":95733026816,"internally_reserved":0,"allocated":561426432,"data_stored":561160379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342290093},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,2,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710756,"kb_used_data":448660,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93661084,"statfs":{"total":96636764160,"available":95908950016,"internally_reserved":0,"allocated":459427840,"data_stored":459161787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561217536,"data_stored":561213923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327450624,"data_stored":327374751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":459833344,"data_stored":459829731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:31.830 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 234 20 163125 163105 2.72238 1.83594 0.0297647 0.0286905 2024-09-10T15:56:32.830 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 235 20 163586 163566 2.71846 1.80078 0.00999899 0.0287298 2024-09-10T15:56:33.830 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 236 20 164024 164004 2.71419 1.71094 0.00702864 0.0287639 2024-09-10T15:56:33.940 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:34.483 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:34.484 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:34.516 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-10T15:56:32.499471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663675382,"num_objects":162033,"num_object_clones":0,"num_object_copies":324066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2748,"num_objects_unfound":0,"num_objects_dirty":162033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123635,"num_write_kb":494534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5118,"num_bytes_recovered":20946963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160515,"ondisk_log_size":160515,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2142688,"kb_used_data":1316144,"kb_used_omap":12,"kb_used_meta":826483,"kb_avail":280972832,"statfs":{"total":289910292480,"available":287716179968,"internally_reserved":0,"allocated":1347731456,"data_stored":1346861037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":846318597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,7,7,5,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18702336,"num_objects":4566,"num_object_clones":0,"num_object_copies":9132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-157,"num_objects_unfound":0,"num_objects_dirty":4566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4566,"num_write_kb":18264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":643072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004835"},"pg_stats":[{"pgid":"2.f","version":"46'10142","reported_seq":10218,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.102763+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:29.102763+0000","last_peered":"2024-09-10T15:56:29.102763+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:29.102763+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41541632,"num_objects":10142,"num_object_clones":0,"num_object_copies":20284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10142,"num_write_kb":40568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6169},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10320","reported_seq":10427,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.688425+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:29.688425+0000","last_peered":"2024-09-10T15:56:29.688425+0000","last_clean":"2024-09-10T15:56:29.688425+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:29.688425+0000","last_undegraded":"2024-09-10T15:56:29.688425+0000","last_fullsized":"2024-09-10T15:56:29.688425+0000","mapping_epoch":36,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42246144,"num_objects":10314,"num_object_clones":0,"num_object_copies":20628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8656,"num_write_kb":34624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10097","reported_seq":10220,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.769943+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:29.769943+0000","last_peered":"2024-09-10T15:56:29.769943+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:29.769943+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41340928,"num_objects":10093,"num_object_clones":0,"num_object_copies":20186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8365,"num_write_kb":33460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":6291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10119","reported_seq":10198,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.749985+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:29.749985+0000","last_peered":"2024-09-10T15:56:29.749985+0000","last_clean":"2024-09-10T15:56:29.749985+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:29.749985+0000","last_undegraded":"2024-09-10T15:56:29.749985+0000","last_fullsized":"2024-09-10T15:56:29.749985+0000","mapping_epoch":36,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41447424,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10119,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9947","reported_seq":10073,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.017119+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:29.017119+0000","last_peered":"2024-09-10T15:56:29.017119+0000","last_clean":"2024-09-10T15:56:29.017119+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:29.017119+0000","last_undegraded":"2024-09-10T15:56:29.017119+0000","last_fullsized":"2024-09-10T15:56:29.017119+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":9947,"log_dups_size":0,"ondisk_log_size":9947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40714240,"num_objects":9940,"num_object_clones":0,"num_object_copies":19880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7018,"num_write_kb":28072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10186","reported_seq":12559,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:29.088289+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:29.088289+0000","last_peered":"2024-09-10T15:56:29.088289+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:29.088289+0000","last_undegraded":"2024-09-10T15:56:29.088289+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41701376,"num_objects":10181,"num_object_clones":0,"num_object_copies":20362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2748,"num_objects_unfound":0,"num_objects_dirty":10181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6904,"num_write_kb":27616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1120,"num_bytes_recovered":4587520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7433},{"shards":"0,2","objects":2748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9896","reported_seq":10036,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.068223+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:29.068223+0000","last_peered":"2024-09-10T15:56:29.068223+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:29.068223+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":9896,"log_dups_size":0,"ondisk_log_size":9896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40509440,"num_objects":9890,"num_object_clones":0,"num_object_copies":19780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6546,"num_write_kb":26184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6056},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10234","reported_seq":10365,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.075204+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:29.075204+0000","last_peered":"2024-09-10T15:56:29.075204+0000","last_clean":"2024-09-10T15:56:29.075204+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:29.075204+0000","last_undegraded":"2024-09-10T15:56:29.075204+0000","last_fullsized":"2024-09-10T15:56:29.075204+0000","mapping_epoch":35,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41910272,"num_objects":10232,"num_object_clones":0,"num_object_copies":20464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6993,"num_write_kb":27972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10144","reported_seq":18270,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.179412+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:29.179412+0000","last_peered":"2024-09-10T15:56:29.179412+0000","last_clean":"2024-09-10T15:56:29.179412+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:29.179412+0000","last_undegraded":"2024-09-10T15:56:29.179412+0000","last_fullsized":"2024-09-10T15:56:29.179412+0000","mapping_epoch":45,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41521152,"num_objects":10137,"num_object_clones":0,"num_object_copies":20274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8121,"num_write_kb":32484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10051","reported_seq":10178,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.063081+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:29.063081+0000","last_peered":"2024-09-10T15:56:29.063081+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:29.063081+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41136128,"num_objects":10043,"num_object_clones":0,"num_object_copies":20086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6915,"num_write_kb":27660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6180},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10188","reported_seq":10311,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.842037+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:29.842037+0000","last_peered":"2024-09-10T15:56:29.842037+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:29.842037+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41709568,"num_objects":10183,"num_object_clones":0,"num_object_copies":20366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8162,"num_write_kb":32648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":6351}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10029","reported_seq":10155,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.751786+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:29.751786+0000","last_peered":"2024-09-10T15:56:29.751786+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:29.751786+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41054208,"num_objects":10023,"num_object_clones":0,"num_object_copies":20046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7898,"num_write_kb":31592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":6194}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10193","reported_seq":10320,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.101337+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:29.101337+0000","last_peered":"2024-09-10T15:56:29.101337+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:29.101337+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41742336,"num_objects":10191,"num_object_clones":0,"num_object_copies":20382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7385,"num_write_kb":29540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6358},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10126","reported_seq":10289,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.055706+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:29.055706+0000","last_peered":"2024-09-10T15:56:29.055706+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:29.055706+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41443328,"num_objects":10118,"num_object_clones":0,"num_object_copies":20236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6544,"num_write_kb":26176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6201},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10136","reported_seq":10275,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:28.984146+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:28.984146+0000","last_peered":"2024-09-10T15:56:28.984146+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:28.984146+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41500672,"num_objects":10132,"num_object_clones":0,"num_object_copies":20264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6742,"num_write_kb":26968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6255},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10304","reported_seq":10442,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:28.991070+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:28.991070+0000","last_peered":"2024-09-10T15:56:28.991070+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:28.991070+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42156515,"num_objects":10293,"num_object_clones":0,"num_object_copies":20586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7123,"num_write_kb":28492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6290},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":663675363,"num_objects":162031,"num_object_clones":0,"num_object_copies":324062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2748,"num_objects_unfound":0,"num_objects_dirty":162031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123633,"num_write_kb":494532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5116,"num_bytes_recovered":20946944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1348501504,"data_stored":1348418405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160512,"ondisk_log_size":160512,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626473,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549376,"kb_used_data":319216,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93822464,"statfs":{"total":96636764160,"available":96074203136,"internally_reserved":0,"allocated":326877184,"data_stored":326538871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235663022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149756,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":882556,"kb_used_data":548268,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93489284,"statfs":{"total":96636764160,"available":95733026816,"internally_reserved":0,"allocated":561426432,"data_stored":561160379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342290093},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,2,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710756,"kb_used_data":448660,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93661084,"statfs":{"total":96636764160,"available":95908950016,"internally_reserved":0,"allocated":459427840,"data_stored":459161787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561217536,"data_stored":561213923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327450624,"data_stored":327374751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":459833344,"data_stored":459829731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:34.518 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:34.830 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 237 20 164400 164380 2.70894 1.46875 0.0141482 0.0288326 2024-09-10T15:56:35.084 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:35.087 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:35.105 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-10T15:56:34.500105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669860342,"num_objects":163543,"num_object_clones":0,"num_object_copies":327086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2676,"num_objects_unfound":0,"num_objects_dirty":163543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125145,"num_write_kb":500574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5190,"num_bytes_recovered":21241875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2164472,"kb_used_data":1329608,"kb_used_omap":12,"kb_used_meta":834803,"kb_avail":280951048,"statfs":{"total":289910292480,"available":287693873152,"internally_reserved":0,"allocated":1361518592,"data_stored":1360648173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":854838277},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,10,11,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24887296,"num_objects":6076,"num_object_clones":0,"num_object_copies":12152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-229,"num_objects_unfound":0,"num_objects_dirty":6076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6076,"num_write_kb":24304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":937984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004690"},"pg_stats":[{"pgid":"2.f","version":"46'10282","reported_seq":10358,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.112456+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:34.112456+0000","last_peered":"2024-09-10T15:56:34.112456+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:34.112456+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42115072,"num_objects":10282,"num_object_clones":0,"num_object_copies":20564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10282,"num_write_kb":41128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6309},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10320","reported_seq":10427,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.688425+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:29.688425+0000","last_peered":"2024-09-10T15:56:29.688425+0000","last_clean":"2024-09-10T15:56:29.688425+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:29.688425+0000","last_undegraded":"2024-09-10T15:56:29.688425+0000","last_fullsized":"2024-09-10T15:56:29.688425+0000","mapping_epoch":36,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42246144,"num_objects":10314,"num_object_clones":0,"num_object_copies":20628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8656,"num_write_kb":34624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10097","reported_seq":10220,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.769943+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:29.769943+0000","last_peered":"2024-09-10T15:56:29.769943+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:29.769943+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41340928,"num_objects":10093,"num_object_clones":0,"num_object_copies":20186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8365,"num_write_kb":33460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":6291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10119","reported_seq":10198,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.749985+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:29.749985+0000","last_peered":"2024-09-10T15:56:29.749985+0000","last_clean":"2024-09-10T15:56:29.749985+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:29.749985+0000","last_undegraded":"2024-09-10T15:56:29.749985+0000","last_fullsized":"2024-09-10T15:56:29.749985+0000","mapping_epoch":36,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41447424,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10119,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10078","reported_seq":10205,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:34.174399+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:34.174399+0000","last_peered":"2024-09-10T15:56:34.174399+0000","last_clean":"2024-09-10T15:56:34.174399+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:34.174399+0000","last_undegraded":"2024-09-10T15:56:34.174399+0000","last_fullsized":"2024-09-10T15:56:34.174399+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41250816,"num_objects":10071,"num_object_clones":0,"num_object_copies":20142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7149,"num_write_kb":28596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10331","reported_seq":12850,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:34.134168+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:34.134168+0000","last_peered":"2024-09-10T15:56:34.134168+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:34.134168+0000","last_undegraded":"2024-09-10T15:56:34.134168+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42295296,"num_objects":10326,"num_object_clones":0,"num_object_copies":20652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2676,"num_objects_unfound":0,"num_objects_dirty":10326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7049,"num_write_kb":28196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1192,"num_bytes_recovered":4882432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7650},{"shards":"0,2","objects":2676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10047","reported_seq":10187,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.127667+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:34.127667+0000","last_peered":"2024-09-10T15:56:34.127667+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:34.127667+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41127936,"num_objects":10041,"num_object_clones":0,"num_object_copies":20082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6697,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6207},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10354","reported_seq":10485,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:34.020310+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:34.020310+0000","last_peered":"2024-09-10T15:56:34.020310+0000","last_clean":"2024-09-10T15:56:34.020310+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:34.020310+0000","last_undegraded":"2024-09-10T15:56:34.020310+0000","last_fullsized":"2024-09-10T15:56:34.020310+0000","mapping_epoch":35,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42401792,"num_objects":10352,"num_object_clones":0,"num_object_copies":20704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7113,"num_write_kb":28452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10271","reported_seq":18398,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:34.241025+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:34.241025+0000","last_peered":"2024-09-10T15:56:34.241025+0000","last_clean":"2024-09-10T15:56:34.241025+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:34.241025+0000","last_undegraded":"2024-09-10T15:56:34.241025+0000","last_fullsized":"2024-09-10T15:56:34.241025+0000","mapping_epoch":45,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42041344,"num_objects":10264,"num_object_clones":0,"num_object_copies":20528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8248,"num_write_kb":32992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10193","reported_seq":10320,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.123453+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:34.123453+0000","last_peered":"2024-09-10T15:56:34.123453+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:34.123453+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41717760,"num_objects":10185,"num_object_clones":0,"num_object_copies":20370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7057,"num_write_kb":28228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6322},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10188","reported_seq":10311,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.842037+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:29.842037+0000","last_peered":"2024-09-10T15:56:29.842037+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:29.842037+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41709568,"num_objects":10183,"num_object_clones":0,"num_object_copies":20366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8162,"num_write_kb":32648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":6351}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10029","reported_seq":10155,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.751786+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:29.751786+0000","last_peered":"2024-09-10T15:56:29.751786+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:29.751786+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41054208,"num_objects":10023,"num_object_clones":0,"num_object_copies":20046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7898,"num_write_kb":31592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":6194}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10326","reported_seq":10453,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.036826+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:34.036826+0000","last_peered":"2024-09-10T15:56:34.036826+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:34.036826+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42287104,"num_objects":10324,"num_object_clones":0,"num_object_copies":20648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7518,"num_write_kb":30072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6491},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10259","reported_seq":10422,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.189204+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:34.189204+0000","last_peered":"2024-09-10T15:56:34.189204+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:34.189204+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41988096,"num_objects":10251,"num_object_clones":0,"num_object_copies":20502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6677,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6334},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10275","reported_seq":10414,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.074028+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:34.074028+0000","last_peered":"2024-09-10T15:56:34.074028+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:34.074028+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42070016,"num_objects":10271,"num_object_clones":0,"num_object_copies":20542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6881,"num_write_kb":27524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6394},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10453","reported_seq":10590,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.103392+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:34.103392+0000","last_peered":"2024-09-10T15:56:34.103392+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:34.103392+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42766819,"num_objects":10442,"num_object_clones":0,"num_object_copies":20884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7272,"num_write_kb":29088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6439},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":669860323,"num_objects":163541,"num_object_clones":0,"num_object_copies":327082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2676,"num_objects_unfound":0,"num_objects_dirty":163541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125143,"num_write_kb":500572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5188,"num_bytes_recovered":21241856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1362272256,"data_stored":1362189157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160922,"ondisk_log_size":160922,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626474,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560532,"kb_used_data":326212,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93811308,"statfs":{"total":96636764160,"available":96062779392,"internally_reserved":0,"allocated":334041088,"data_stored":333702775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239922862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149757,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":893184,"kb_used_data":554736,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93478656,"statfs":{"total":96636764160,"available":95722143744,"internally_reserved":0,"allocated":568049664,"data_stored":567783611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346549933},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,7,6,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710756,"kb_used_data":448660,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93661084,"statfs":{"total":96636764160,"available":95908950016,"internally_reserved":0,"allocated":459427840,"data_stored":459161787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568012800,"data_stored":568009187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334426112,"data_stored":334350239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":459833344,"data_stored":459829731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:35.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:56:35.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'10282", 'reported_seq': 10358, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:34.112456+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:56:34.112456+0000', 'last_peered': '2024-09-10T15:56:34.112456+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:56:34.112456+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'200", 'ondisk_log_start': "37'200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42115072, 'num_objects': 10282, 'num_object_clones': 0, 'num_object_copies': 20564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3973, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10282, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10282, 'num_write_kb': 41128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6309}, {'shards': '0,2', 'objects': 3973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:35.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:56:35.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'10097", 'reported_seq': 10220, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:29.769943+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:56:29.769943+0000', 'last_peered': '2024-09-10T15:56:29.769943+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:56:29.769943+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41340928, 'num_objects': 10093, 'num_object_clones': 0, 'num_object_copies': 20186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10093, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8365, 'num_write_kb': 33460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3800}, {'shards': '2', 'objects': 6291}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:35.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:56:35.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'10331", 'reported_seq': 12850, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:56:34.134168+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:56:34.134168+0000', 'last_peered': '2024-09-10T15:56:34.134168+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:56:34.134168+0000', 'last_undegraded': '2024-09-10T15:56:34.134168+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'300", 'ondisk_log_start': "37'300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42295296, 'num_objects': 10326, 'num_object_clones': 0, 'num_object_copies': 20652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2676, 'num_objects_unfound': 0, 'num_objects_dirty': 10326, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7049, 'num_write_kb': 28196, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1192, 'num_bytes_recovered': 4882432, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7650}, {'shards': '0,2', 'objects': 2676}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:35.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:56:35.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'10047", 'reported_seq': 10187, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:34.127667+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:56:34.127667+0000', 'last_peered': '2024-09-10T15:56:34.127667+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:56:34.127667+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41127936, 'num_objects': 10041, 'num_object_clones': 0, 'num_object_copies': 20082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6697, 'num_write_kb': 26788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6207}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:35.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:56:35.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'10193", 'reported_seq': 10320, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:34.123453+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:56:34.123453+0000', 'last_peered': '2024-09-10T15:56:34.123453+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:56:34.123453+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'100", 'ondisk_log_start': "37'100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41717760, 'num_objects': 10185, 'num_object_clones': 0, 'num_object_copies': 20370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7057, 'num_write_kb': 28228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6322}, {'shards': '0,2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:35.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:56:35.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'10188", 'reported_seq': 10311, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:29.842037+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:56:29.842037+0000', 'last_peered': '2024-09-10T15:56:29.842037+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:56:29.842037+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'100", 'ondisk_log_start': "37'100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41709568, 'num_objects': 10183, 'num_object_clones': 0, 'num_object_copies': 20366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10183, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8162, 'num_write_kb': 32648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3832}, {'shards': '2', 'objects': 6351}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:35.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:56:35.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'10029", 'reported_seq': 10155, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:29.751786+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:56:29.751786+0000', 'last_peered': '2024-09-10T15:56:29.751786+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:56:29.751786+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41054208, 'num_objects': 10023, 'num_object_clones': 0, 'num_object_copies': 20046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10023, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7898, 'num_write_kb': 31592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 6194}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:35.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:56:35.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'10326", 'reported_seq': 10453, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:34.036826+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:56:34.036826+0000', 'last_peered': '2024-09-10T15:56:34.036826+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:56:34.036826+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'300", 'ondisk_log_start': "37'300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42287104, 'num_objects': 10324, 'num_object_clones': 0, 'num_object_copies': 20648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10324, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7518, 'num_write_kb': 30072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6491}, {'shards': '0,2', 'objects': 3833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:35.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:56:35.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'10259", 'reported_seq': 10422, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:34.189204+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:56:34.189204+0000', 'last_peered': '2024-09-10T15:56:34.189204+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:56:34.189204+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'200", 'ondisk_log_start': "37'200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41988096, 'num_objects': 10251, 'num_object_clones': 0, 'num_object_copies': 20502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10251, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6677, 'num_write_kb': 26708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6334}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:35.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:56:35.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'10275", 'reported_seq': 10414, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:34.074028+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:56:34.074028+0000', 'last_peered': '2024-09-10T15:56:34.074028+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:56:34.074028+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'200", 'ondisk_log_start': "37'200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42070016, 'num_objects': 10271, 'num_object_clones': 0, 'num_object_copies': 20542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6881, 'num_write_kb': 27524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6394}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:35.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:56:35.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'10453", 'reported_seq': 10590, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:34.103392+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:56:34.103392+0000', 'last_peered': '2024-09-10T15:56:34.103392+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:56:34.103392+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42766819, 'num_objects': 10442, 'num_object_clones': 0, 'num_object_copies': 20884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7272, 'num_write_kb': 29088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6439}, {'shards': '0,2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:35.109 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:56:35.774 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:35.774 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38781/327086 objects degraded (11.857%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":321,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":163543,"data_bytes":669860342,"bytes_used":2216419328,"bytes_avail":287693873152,"bytes_total":289910292480,"degraded_objects":38781,"degraded_total":327086,"degraded_ratio":0.11856514800388888,"misplaced_objects":2676,"misplaced_total":327086,"misplaced_ratio":0.008181334572558898,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":78134,"recovering_keys_per_sec":0,"num_objects_recovered":229,"num_bytes_recovered":937984,"num_keys_recovered":0,"write_bytes_sec":2073130,"read_op_per_sec":0,"write_op_per_sec":506},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T15:56:24.496337+0000","services":{}},"progress_events":{}} 2024-09-10T15:56:35.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:56:35.797 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:35.830 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 238 20 164817 164797 2.70439 1.62891 0.0342776 0.0288816 2024-09-10T15:56:36.361 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:36.362 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:36.390 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-10T15:56:34.500105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669860342,"num_objects":163543,"num_object_clones":0,"num_object_copies":327086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2676,"num_objects_unfound":0,"num_objects_dirty":163543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125145,"num_write_kb":500574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5190,"num_bytes_recovered":21241875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2164472,"kb_used_data":1329608,"kb_used_omap":12,"kb_used_meta":834803,"kb_avail":280951048,"statfs":{"total":289910292480,"available":287693873152,"internally_reserved":0,"allocated":1361518592,"data_stored":1360648173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":854838277},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,10,11,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24887296,"num_objects":6076,"num_object_clones":0,"num_object_copies":12152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-229,"num_objects_unfound":0,"num_objects_dirty":6076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6076,"num_write_kb":24304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":937984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004690"},"pg_stats":[{"pgid":"2.f","version":"46'10282","reported_seq":10358,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.112456+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:34.112456+0000","last_peered":"2024-09-10T15:56:34.112456+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:34.112456+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42115072,"num_objects":10282,"num_object_clones":0,"num_object_copies":20564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10282,"num_write_kb":41128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6309},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10320","reported_seq":10427,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.688425+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:29.688425+0000","last_peered":"2024-09-10T15:56:29.688425+0000","last_clean":"2024-09-10T15:56:29.688425+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:29.688425+0000","last_undegraded":"2024-09-10T15:56:29.688425+0000","last_fullsized":"2024-09-10T15:56:29.688425+0000","mapping_epoch":36,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42246144,"num_objects":10314,"num_object_clones":0,"num_object_copies":20628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8656,"num_write_kb":34624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10097","reported_seq":10220,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.769943+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:29.769943+0000","last_peered":"2024-09-10T15:56:29.769943+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:29.769943+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41340928,"num_objects":10093,"num_object_clones":0,"num_object_copies":20186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8365,"num_write_kb":33460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":6291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10119","reported_seq":10198,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:29.749985+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:29.749985+0000","last_peered":"2024-09-10T15:56:29.749985+0000","last_clean":"2024-09-10T15:56:29.749985+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:29.749985+0000","last_undegraded":"2024-09-10T15:56:29.749985+0000","last_fullsized":"2024-09-10T15:56:29.749985+0000","mapping_epoch":36,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41447424,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10119,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10078","reported_seq":10205,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:34.174399+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:34.174399+0000","last_peered":"2024-09-10T15:56:34.174399+0000","last_clean":"2024-09-10T15:56:34.174399+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:34.174399+0000","last_undegraded":"2024-09-10T15:56:34.174399+0000","last_fullsized":"2024-09-10T15:56:34.174399+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41250816,"num_objects":10071,"num_object_clones":0,"num_object_copies":20142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7149,"num_write_kb":28596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10331","reported_seq":12850,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:34.134168+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:34.134168+0000","last_peered":"2024-09-10T15:56:34.134168+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:34.134168+0000","last_undegraded":"2024-09-10T15:56:34.134168+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42295296,"num_objects":10326,"num_object_clones":0,"num_object_copies":20652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2676,"num_objects_unfound":0,"num_objects_dirty":10326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7049,"num_write_kb":28196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1192,"num_bytes_recovered":4882432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7650},{"shards":"0,2","objects":2676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10047","reported_seq":10187,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.127667+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:34.127667+0000","last_peered":"2024-09-10T15:56:34.127667+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:34.127667+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41127936,"num_objects":10041,"num_object_clones":0,"num_object_copies":20082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6697,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6207},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10354","reported_seq":10485,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:34.020310+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:34.020310+0000","last_peered":"2024-09-10T15:56:34.020310+0000","last_clean":"2024-09-10T15:56:34.020310+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:34.020310+0000","last_undegraded":"2024-09-10T15:56:34.020310+0000","last_fullsized":"2024-09-10T15:56:34.020310+0000","mapping_epoch":35,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42401792,"num_objects":10352,"num_object_clones":0,"num_object_copies":20704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7113,"num_write_kb":28452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10271","reported_seq":18398,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:34.241025+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:34.241025+0000","last_peered":"2024-09-10T15:56:34.241025+0000","last_clean":"2024-09-10T15:56:34.241025+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:34.241025+0000","last_undegraded":"2024-09-10T15:56:34.241025+0000","last_fullsized":"2024-09-10T15:56:34.241025+0000","mapping_epoch":45,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42041344,"num_objects":10264,"num_object_clones":0,"num_object_copies":20528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8248,"num_write_kb":32992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10193","reported_seq":10320,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.123453+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:34.123453+0000","last_peered":"2024-09-10T15:56:34.123453+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:34.123453+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41717760,"num_objects":10185,"num_object_clones":0,"num_object_copies":20370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7057,"num_write_kb":28228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6322},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10188","reported_seq":10311,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.842037+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:29.842037+0000","last_peered":"2024-09-10T15:56:29.842037+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:29.842037+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41709568,"num_objects":10183,"num_object_clones":0,"num_object_copies":20366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8162,"num_write_kb":32648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":6351}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10029","reported_seq":10155,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:29.751786+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:29.751786+0000","last_peered":"2024-09-10T15:56:29.751786+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:29.751786+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41054208,"num_objects":10023,"num_object_clones":0,"num_object_copies":20046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7898,"num_write_kb":31592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":6194}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10326","reported_seq":10453,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.036826+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:34.036826+0000","last_peered":"2024-09-10T15:56:34.036826+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:34.036826+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42287104,"num_objects":10324,"num_object_clones":0,"num_object_copies":20648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7518,"num_write_kb":30072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6491},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10259","reported_seq":10422,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.189204+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:34.189204+0000","last_peered":"2024-09-10T15:56:34.189204+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:34.189204+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41988096,"num_objects":10251,"num_object_clones":0,"num_object_copies":20502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6677,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6334},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.679974+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.679974+0000","last_peered":"2024-09-10T15:55:20.679974+0000","last_clean":"2024-09-10T15:55:20.679974+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.679974+0000","last_undegraded":"2024-09-10T15:55:20.679974+0000","last_fullsized":"2024-09-10T15:55:20.679974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10275","reported_seq":10414,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.074028+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:34.074028+0000","last_peered":"2024-09-10T15:56:34.074028+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:34.074028+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42070016,"num_objects":10271,"num_object_clones":0,"num_object_copies":20542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6881,"num_write_kb":27524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6394},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10453","reported_seq":10590,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.103392+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:34.103392+0000","last_peered":"2024-09-10T15:56:34.103392+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:34.103392+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42766819,"num_objects":10442,"num_object_clones":0,"num_object_copies":20884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7272,"num_write_kb":29088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6439},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":669860323,"num_objects":163541,"num_object_clones":0,"num_object_copies":327082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2676,"num_objects_unfound":0,"num_objects_dirty":163541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125143,"num_write_kb":500572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5188,"num_bytes_recovered":21241856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1362272256,"data_stored":1362189157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160922,"ondisk_log_size":160922,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626474,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560532,"kb_used_data":326212,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93811308,"statfs":{"total":96636764160,"available":96062779392,"internally_reserved":0,"allocated":334041088,"data_stored":333702775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239922862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149757,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":893184,"kb_used_data":554736,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93478656,"statfs":{"total":96636764160,"available":95722143744,"internally_reserved":0,"allocated":568049664,"data_stored":567783611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346549933},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,7,6,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710756,"kb_used_data":448660,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93661084,"statfs":{"total":96636764160,"available":95908950016,"internally_reserved":0,"allocated":459427840,"data_stored":459161787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568012800,"data_stored":568009187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334426112,"data_stored":334350239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":459833344,"data_stored":459829731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:36.831 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout: 239 20 165259 165239 2.7003 1.72656 0.0442643 0.0289252 2024-09-10T15:56:37.831 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Total time run: 240.024 2024-09-10T15:56:37.831 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Total writes made: 165722 2024-09-10T15:56:37.831 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Write size: 4096 2024-09-10T15:56:37.831 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Object size: 4096 2024-09-10T15:56:37.831 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Bandwidth (MB/sec): 2.69703 2024-09-10T15:56:37.831 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Stddev Bandwidth: 1.86769 2024-09-10T15:56:37.831 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Max bandwidth (MB/sec): 12.2031 2024-09-10T15:56:37.831 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Min bandwidth (MB/sec): 0 2024-09-10T15:56:37.831 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Average IOPS: 690 2024-09-10T15:56:37.832 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Stddev IOPS: 478.13 2024-09-10T15:56:37.832 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Max IOPS: 3124 2024-09-10T15:56:37.832 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Min IOPS: 0 2024-09-10T15:56:37.832 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Average Latency(s): 0.0289616 2024-09-10T15:56:37.832 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Stddev Latency(s): 0.044084 2024-09-10T15:56:37.832 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Max latency(s): 2.66872 2024-09-10T15:56:37.832 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Min latency(s): 0.00198913 2024-09-10T15:56:37.840 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Cleaning up (deleting benchmark objects) 2024-09-10T15:56:39.393 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:39.981 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:39.982 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:40.007 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-10T15:56:38.501163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672633334,"num_objects":164220,"num_object_clones":0,"num_object_copies":328440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2676,"num_objects_unfound":0,"num_objects_dirty":164220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125822,"num_write_kb":503282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5190,"num_bytes_recovered":21241875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160902,"ondisk_log_size":160902,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2172844,"kb_used_data":1333820,"kb_used_omap":12,"kb_used_meta":838963,"kb_avail":280942676,"statfs":{"total":289910292480,"available":287685300224,"internally_reserved":0,"allocated":1365831680,"data_stored":1364961261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":859098117},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3,12,11,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18223104,"num_objects":4449,"num_object_clones":0,"num_object_copies":8898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-153,"num_objects_unfound":0,"num_objects_dirty":4449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4449,"num_write_kb":17796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":626688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004058"},"pg_stats":[{"pgid":"2.f","version":"46'10282","reported_seq":10358,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.112456+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:34.112456+0000","last_peered":"2024-09-10T15:56:34.112456+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:34.112456+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42115072,"num_objects":10282,"num_object_clones":0,"num_object_copies":20564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10282,"num_write_kb":41128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6309},{"shards":"0,2","objects":3973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10463","reported_seq":10568,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:34.915849+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:34.915849+0000","last_peered":"2024-09-10T15:56:34.915849+0000","last_clean":"2024-09-10T15:56:34.915849+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:34.915849+0000","last_undegraded":"2024-09-10T15:56:34.915849+0000","last_fullsized":"2024-09-10T15:56:34.915849+0000","mapping_epoch":36,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42831872,"num_objects":10457,"num_object_clones":0,"num_object_copies":20914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8799,"num_write_kb":35196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10227","reported_seq":10350,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.892757+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:34.892757+0000","last_peered":"2024-09-10T15:56:34.892757+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:34.892757+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41873408,"num_objects":10223,"num_object_clones":0,"num_object_copies":20446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8495,"num_write_kb":33980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3800},{"shards":"2","objects":6421}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10249","reported_seq":10328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:34.823483+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:34.823483+0000","last_peered":"2024-09-10T15:56:34.823483+0000","last_clean":"2024-09-10T15:56:34.823483+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:34.823483+0000","last_undegraded":"2024-09-10T15:56:34.823483+0000","last_fullsized":"2024-09-10T15:56:34.823483+0000","mapping_epoch":36,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41979904,"num_objects":10249,"num_object_clones":0,"num_object_copies":20498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10249,"num_write_kb":40996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10078","reported_seq":10205,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:34.174399+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:34.174399+0000","last_peered":"2024-09-10T15:56:34.174399+0000","last_clean":"2024-09-10T15:56:34.174399+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:34.174399+0000","last_undegraded":"2024-09-10T15:56:34.174399+0000","last_fullsized":"2024-09-10T15:56:34.174399+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41250816,"num_objects":10071,"num_object_clones":0,"num_object_copies":20142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7149,"num_write_kb":28596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10331","reported_seq":12850,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:34.134168+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:34.134168+0000","last_peered":"2024-09-10T15:56:34.134168+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:34.134168+0000","last_undegraded":"2024-09-10T15:56:34.134168+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42295296,"num_objects":10326,"num_object_clones":0,"num_object_copies":20652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2676,"num_objects_unfound":0,"num_objects_dirty":10326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7049,"num_write_kb":28196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1192,"num_bytes_recovered":4882432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7650},{"shards":"0,2","objects":2676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10047","reported_seq":10187,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.127667+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:34.127667+0000","last_peered":"2024-09-10T15:56:34.127667+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:34.127667+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41127936,"num_objects":10041,"num_object_clones":0,"num_object_copies":20082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6697,"num_write_kb":26788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6207},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10354","reported_seq":10485,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:34.020310+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:34.020310+0000","last_peered":"2024-09-10T15:56:34.020310+0000","last_clean":"2024-09-10T15:56:34.020310+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:34.020310+0000","last_undegraded":"2024-09-10T15:56:34.020310+0000","last_fullsized":"2024-09-10T15:56:34.020310+0000","mapping_epoch":35,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42401792,"num_objects":10352,"num_object_clones":0,"num_object_copies":20704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7113,"num_write_kb":28452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10271","reported_seq":18398,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:34.241025+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:34.241025+0000","last_peered":"2024-09-10T15:56:34.241025+0000","last_clean":"2024-09-10T15:56:34.241025+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:34.241025+0000","last_undegraded":"2024-09-10T15:56:34.241025+0000","last_fullsized":"2024-09-10T15:56:34.241025+0000","mapping_epoch":45,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42041344,"num_objects":10264,"num_object_clones":0,"num_object_copies":20528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8248,"num_write_kb":32992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10193","reported_seq":10320,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.123453+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:34.123453+0000","last_peered":"2024-09-10T15:56:34.123453+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:34.123453+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41717760,"num_objects":10185,"num_object_clones":0,"num_object_copies":20370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7057,"num_write_kb":28228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6322},{"shards":"0,2","objects":3863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10331","reported_seq":10456,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.905208+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:34.905208+0000","last_peered":"2024-09-10T15:56:34.905208+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:34.905208+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42295296,"num_objects":10326,"num_object_clones":0,"num_object_copies":20652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8305,"num_write_kb":33220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3832},{"shards":"2","objects":6494}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10160","reported_seq":10286,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.820372+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:34.820372+0000","last_peered":"2024-09-10T15:56:34.820372+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:34.820372+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41590784,"num_objects":10154,"num_object_clones":0,"num_object_copies":20308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8029,"num_write_kb":32116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":6325}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10326","reported_seq":10453,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.036826+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:34.036826+0000","last_peered":"2024-09-10T15:56:34.036826+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:34.036826+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42287104,"num_objects":10324,"num_object_clones":0,"num_object_copies":20648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7518,"num_write_kb":30072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6491},{"shards":"0,2","objects":3833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10259","reported_seq":10422,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.189204+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:34.189204+0000","last_peered":"2024-09-10T15:56:34.189204+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:34.189204+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41988096,"num_objects":10251,"num_object_clones":0,"num_object_copies":20502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6677,"num_write_kb":26708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6334},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10275","reported_seq":10414,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.074028+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:34.074028+0000","last_peered":"2024-09-10T15:56:34.074028+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:34.074028+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42070016,"num_objects":10271,"num_object_clones":0,"num_object_copies":20542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6881,"num_write_kb":27524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6394},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10453","reported_seq":10590,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:34.103392+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:34.103392+0000","last_peered":"2024-09-10T15:56:34.103392+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:34.103392+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42766819,"num_objects":10442,"num_object_clones":0,"num_object_copies":20884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7272,"num_write_kb":29088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6439},{"shards":"0,2","objects":4003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":672633315,"num_objects":164218,"num_object_clones":0,"num_object_copies":328436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38781,"num_objects_misplaced":2676,"num_objects_unfound":0,"num_objects_dirty":164218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":125820,"num_write_kb":503280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5188,"num_bytes_recovered":21241856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1366589440,"data_stored":1366506341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626474,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560532,"kb_used_data":326212,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93811308,"statfs":{"total":96636764160,"available":96062779392,"internally_reserved":0,"allocated":334041088,"data_stored":333702775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239922862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149757,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":893184,"kb_used_data":554736,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93478656,"statfs":{"total":96636764160,"available":95722143744,"internally_reserved":0,"allocated":568049664,"data_stored":567783611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346549933},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,7,6,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953528,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719128,"kb_used_data":452872,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93652712,"statfs":{"total":96636764160,"available":95900377088,"internally_reserved":0,"allocated":463740928,"data_stored":463474875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568012800,"data_stored":568009187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334426112,"data_stored":334350239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":464150528,"data_stored":464146915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:40.009 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:40.537 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:40.537 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:40.562 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-10T15:56:40.501598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":675758606,"num_objects":164984,"num_object_clones":0,"num_object_copies":329968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38761,"num_objects_misplaced":2588,"num_objects_unfound":0,"num_objects_dirty":164984,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128070,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5298,"num_bytes_recovered":21602323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160650,"ondisk_log_size":160650,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2196600,"kb_used_data":1345096,"kb_used_omap":12,"kb_used_meta":851443,"kb_avail":280918920,"statfs":{"total":289910292480,"available":287660974080,"internally_reserved":0,"allocated":1377378304,"data_stored":1376499741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":871877637},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,7,4,9,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21348376,"num_objects":5213,"num_object_clones":0,"num_object_copies":10426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-20,"num_objects_misplaced":-241,"num_objects_unfound":0,"num_objects_dirty":5213,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6697,"num_write_kb":23817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":987136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003844"},"pg_stats":[{"pgid":"2.f","version":"46'10426","reported_seq":10509,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.252086+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:39.252086+0000","last_peered":"2024-09-10T15:56:39.252086+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:39.252086+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42369024,"num_objects":10344,"num_object_clones":0,"num_object_copies":20688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10426,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6374},{"shards":"0,2","objects":3932}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10598","reported_seq":10705,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:39.812695+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:39.812695+0000","last_peered":"2024-09-10T15:56:39.812695+0000","last_clean":"2024-09-10T15:56:39.812695+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:39.812695+0000","last_undegraded":"2024-09-10T15:56:39.812695+0000","last_fullsized":"2024-09-10T15:56:39.812695+0000","mapping_epoch":36,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42909696,"num_objects":10476,"num_object_clones":0,"num_object_copies":20952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8934,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10372","reported_seq":10502,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.825205+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:39.825205+0000","last_peered":"2024-09-10T15:56:39.825205+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:39.825205+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42012696,"num_objects":10258,"num_object_clones":0,"num_object_copies":20516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10258,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8640,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3745},{"shards":"2","objects":6459}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10417","reported_seq":10496,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:39.917519+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:39.917519+0000","last_peered":"2024-09-10T15:56:39.917519+0000","last_clean":"2024-09-10T15:56:39.917519+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:39.917519+0000","last_undegraded":"2024-09-10T15:56:39.917519+0000","last_fullsized":"2024-09-10T15:56:39.917519+0000","mapping_epoch":36,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42045440,"num_objects":10265,"num_object_clones":0,"num_object_copies":20530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10417,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10202","reported_seq":10329,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:39.231786+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:39.231786+0000","last_peered":"2024-09-10T15:56:39.231786+0000","last_clean":"2024-09-10T15:56:39.231786+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:39.231786+0000","last_undegraded":"2024-09-10T15:56:39.231786+0000","last_fullsized":"2024-09-10T15:56:39.231786+0000","mapping_epoch":35,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41488384,"num_objects":10129,"num_object_clones":0,"num_object_copies":20258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7273,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10475","reported_seq":13170,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:39.229135+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:39.229135+0000","last_peered":"2024-09-10T15:56:39.229135+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:39.229135+0000","last_undegraded":"2024-09-10T15:56:39.229135+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42565632,"num_objects":10392,"num_object_clones":0,"num_object_copies":20784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2588,"num_objects_unfound":0,"num_objects_dirty":10392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7193,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1280,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7804},{"shards":"0,2","objects":2588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10182","reported_seq":10326,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.144570+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:39.144570+0000","last_peered":"2024-09-10T15:56:39.144570+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:39.144570+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41385984,"num_objects":10104,"num_object_clones":0,"num_object_copies":20208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6832,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6272},{"shards":"0,2","objects":3798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10510","reported_seq":10641,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:39.168831+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:39.168831+0000","last_peered":"2024-09-10T15:56:39.168831+0000","last_clean":"2024-09-10T15:56:39.168831+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:39.168831+0000","last_undegraded":"2024-09-10T15:56:39.168831+0000","last_fullsized":"2024-09-10T15:56:39.168831+0000","mapping_epoch":35,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42614784,"num_objects":10404,"num_object_clones":0,"num_object_copies":20808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7269,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10403","reported_seq":18530,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:39.292307+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:39.292307+0000","last_peered":"2024-09-10T15:56:39.292307+0000","last_clean":"2024-09-10T15:56:39.292307+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:39.292307+0000","last_undegraded":"2024-09-10T15:56:39.292307+0000","last_fullsized":"2024-09-10T15:56:39.292307+0000","mapping_epoch":45,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42254336,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8380,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10317","reported_seq":10448,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.209146+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:39.209146+0000","last_peered":"2024-09-10T15:56:39.209146+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:39.209146+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42029056,"num_objects":10261,"num_object_clones":0,"num_object_copies":20522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3863,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7181,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6400},{"shards":"0,2","objects":3839}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10444","reported_seq":10571,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.735990+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:39.735990+0000","last_peered":"2024-09-10T15:56:39.735990+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:39.735990+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42356736,"num_objects":10341,"num_object_clones":0,"num_object_copies":20682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8418,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3783},{"shards":"2","objects":6510}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10289","reported_seq":10415,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.839054+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:39.839054+0000","last_peered":"2024-09-10T15:56:39.839054+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:39.839054+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41644032,"num_objects":10167,"num_object_clones":0,"num_object_copies":20334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8158,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3770},{"shards":"2","objects":6338}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10461","reported_seq":10590,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.189886+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:39.189886+0000","last_peered":"2024-09-10T15:56:39.189886+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:39.189886+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42569728,"num_objects":10393,"num_object_clones":0,"num_object_copies":20786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7653,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6561},{"shards":"0,2","objects":3800}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10414","reported_seq":10590,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.255885+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:39.255885+0000","last_peered":"2024-09-10T15:56:39.255885+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:39.255885+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42205184,"num_objects":10304,"num_object_clones":0,"num_object_copies":20608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6832,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6393},{"shards":"0,2","objects":3866}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10431","reported_seq":10573,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.239141+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:39.239141+0000","last_peered":"2024-09-10T15:56:39.239141+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:39.239141+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42332160,"num_objects":10335,"num_object_clones":0,"num_object_copies":20670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3878,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7037,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6459},{"shards":"0,2","objects":3831}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10606","reported_seq":10747,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.189098+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:39.189098+0000","last_peered":"2024-09-10T15:56:39.189098+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:39.189098+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42975715,"num_objects":10493,"num_object_clones":0,"num_object_copies":20986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7425,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6491},{"shards":"0,2","objects":3952}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":675758587,"num_objects":164982,"num_object_clones":0,"num_object_copies":329964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38761,"num_objects_misplaced":2588,"num_objects_unfound":0,"num_objects_dirty":164982,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128068,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5296,"num_bytes_recovered":21602304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1374814208,"data_stored":1374722965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160647,"ondisk_log_size":160647,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626475,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570208,"kb_used_data":331728,"kb_used_omap":4,"kb_used_meta":238459,"kb_avail":93801632,"statfs":{"total":96636764160,"available":96052871168,"internally_reserved":0,"allocated":339689472,"data_stored":339347087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244182702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149758,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":900964,"kb_used_data":558356,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93470876,"statfs":{"total":96636764160,"available":95714177024,"internally_reserved":0,"allocated":571756544,"data_stored":571490491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350809773},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,8,1,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725428,"kb_used_data":455012,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93646412,"statfs":{"total":96636764160,"available":95893925888,"internally_reserved":0,"allocated":465932288,"data_stored":465662163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":570859520,"data_stored":570855907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339226624,"data_stored":339146679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":464728064,"data_stored":464720379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:40.563 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:56:40.563 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'10426", 'reported_seq': 10509, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:39.252086+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:56:39.252086+0000', 'last_peered': '2024-09-10T15:56:39.252086+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:56:39.252086+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42369024, 'num_objects': 10344, 'num_object_clones': 0, 'num_object_copies': 20688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3970, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10344, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10426, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6374}, {'shards': '0,2', 'objects': 3932}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:40.563 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:56:40.563 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'10372", 'reported_seq': 10502, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:39.825205+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:56:39.825205+0000', 'last_peered': '2024-09-10T15:56:39.825205+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:56:39.825205+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'300", 'ondisk_log_start': "37'300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42012696, 'num_objects': 10258, 'num_object_clones': 0, 'num_object_copies': 20516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3800, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10258, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8640, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3745}, {'shards': '2', 'objects': 6459}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:40.563 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:56:40.563 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'10475", 'reported_seq': 13170, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:56:39.229135+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:56:39.229135+0000', 'last_peered': '2024-09-10T15:56:39.229135+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:56:39.229135+0000', 'last_undegraded': '2024-09-10T15:56:39.229135+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42565632, 'num_objects': 10392, 'num_object_clones': 0, 'num_object_copies': 20784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2588, 'num_objects_unfound': 0, 'num_objects_dirty': 10392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7193, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1280, 'num_bytes_recovered': 5242880, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7804}, {'shards': '0,2', 'objects': 2588}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:40.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:56:40.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'10182", 'reported_seq': 10326, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:39.144570+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:56:39.144570+0000', 'last_peered': '2024-09-10T15:56:39.144570+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:56:39.144570+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'100", 'ondisk_log_start': "37'100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41385984, 'num_objects': 10104, 'num_object_clones': 0, 'num_object_copies': 20208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3834, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10104, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6832, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6272}, {'shards': '0,2', 'objects': 3798}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:40.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:56:40.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'10317", 'reported_seq': 10448, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:39.209146+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:56:39.209146+0000', 'last_peered': '2024-09-10T15:56:39.209146+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:56:39.209146+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'300", 'ondisk_log_start': "37'300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42029056, 'num_objects': 10261, 'num_object_clones': 0, 'num_object_copies': 20522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3863, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10261, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7181, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6400}, {'shards': '0,2', 'objects': 3839}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:40.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:56:40.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'10444", 'reported_seq': 10571, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:39.735990+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:56:39.735990+0000', 'last_peered': '2024-09-10T15:56:39.735990+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:56:39.735990+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42356736, 'num_objects': 10341, 'num_object_clones': 0, 'num_object_copies': 20682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3833, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10341, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8418, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3783}, {'shards': '2', 'objects': 6510}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:40.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:56:40.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'10289", 'reported_seq': 10415, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:39.839054+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:56:39.839054+0000', 'last_peered': '2024-09-10T15:56:39.839054+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:56:39.839054+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'200", 'ondisk_log_start': "37'200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41644032, 'num_objects': 10167, 'num_object_clones': 0, 'num_object_copies': 20334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10167, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8158, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3770}, {'shards': '2', 'objects': 6338}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:40.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:56:40.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'10461", 'reported_seq': 10590, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:39.189886+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:56:39.189886+0000', 'last_peered': '2024-09-10T15:56:39.189886+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:56:39.189886+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42569728, 'num_objects': 10393, 'num_object_clones': 0, 'num_object_copies': 20786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3833, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10393, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7653, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6561}, {'shards': '0,2', 'objects': 3800}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:40.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:56:40.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'10414", 'reported_seq': 10590, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:39.255885+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:56:39.255885+0000', 'last_peered': '2024-09-10T15:56:39.255885+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:56:39.255885+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42205184, 'num_objects': 10304, 'num_object_clones': 0, 'num_object_copies': 20608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3914, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10304, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6832, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6393}, {'shards': '0,2', 'objects': 3866}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:40.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:56:40.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'10431", 'reported_seq': 10573, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:39.239141+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:56:39.239141+0000', 'last_peered': '2024-09-10T15:56:39.239141+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:56:39.239141+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42332160, 'num_objects': 10335, 'num_object_clones': 0, 'num_object_copies': 20670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3878, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7037, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6459}, {'shards': '0,2', 'objects': 3831}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:40.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:56:40.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'10606", 'reported_seq': 10747, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:39.189098+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:56:39.189098+0000', 'last_peered': '2024-09-10T15:56:39.189098+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:56:39.189098+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'600", 'ondisk_log_start': "37'600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42975715, 'num_objects': 10493, 'num_object_clones': 0, 'num_object_copies': 20986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4005, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10493, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7425, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6491}, {'shards': '0,2', 'objects': 3952}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:40.565 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:56:41.213 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:41.214 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38761/329968 objects degraded (11.747%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":327,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":164984,"data_bytes":675758606,"bytes_used":2249318400,"bytes_avail":287660974080,"bytes_total":289910292480,"degraded_objects":38761,"degraded_total":329968,"degraded_ratio":0.11746896668767881,"misplaced_objects":2588,"misplaced_total":329968,"misplaced_ratio":0.0078431847936769623,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":82234,"recovering_keys_per_sec":0,"num_objects_recovered":261,"num_bytes_recovered":987136,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":2031733,"read_op_per_sec":0,"write_op_per_sec":557},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T15:56:24.496337+0000","services":{}},"progress_events":{}} 2024-09-10T15:56:41.235 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:56:41.236 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:41.764 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:41.764 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:41.784 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-10T15:56:40.501598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":675758606,"num_objects":164984,"num_object_clones":0,"num_object_copies":329968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38761,"num_objects_misplaced":2588,"num_objects_unfound":0,"num_objects_dirty":164984,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128070,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5298,"num_bytes_recovered":21602323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160650,"ondisk_log_size":160650,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2196600,"kb_used_data":1345096,"kb_used_omap":12,"kb_used_meta":851443,"kb_avail":280918920,"statfs":{"total":289910292480,"available":287660974080,"internally_reserved":0,"allocated":1377378304,"data_stored":1376499741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":871877637},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,7,4,9,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21348376,"num_objects":5213,"num_object_clones":0,"num_object_copies":10426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-20,"num_objects_misplaced":-241,"num_objects_unfound":0,"num_objects_dirty":5213,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6697,"num_write_kb":23817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":987136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003844"},"pg_stats":[{"pgid":"2.f","version":"46'10426","reported_seq":10509,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.252086+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:39.252086+0000","last_peered":"2024-09-10T15:56:39.252086+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:39.252086+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42369024,"num_objects":10344,"num_object_clones":0,"num_object_copies":20688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10426,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6374},{"shards":"0,2","objects":3932}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10598","reported_seq":10705,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:39.812695+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:39.812695+0000","last_peered":"2024-09-10T15:56:39.812695+0000","last_clean":"2024-09-10T15:56:39.812695+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:39.812695+0000","last_undegraded":"2024-09-10T15:56:39.812695+0000","last_fullsized":"2024-09-10T15:56:39.812695+0000","mapping_epoch":36,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42909696,"num_objects":10476,"num_object_clones":0,"num_object_copies":20952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8934,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10372","reported_seq":10502,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.825205+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:39.825205+0000","last_peered":"2024-09-10T15:56:39.825205+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:39.825205+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42012696,"num_objects":10258,"num_object_clones":0,"num_object_copies":20516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10258,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8640,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3745},{"shards":"2","objects":6459}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10417","reported_seq":10496,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:39.917519+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:39.917519+0000","last_peered":"2024-09-10T15:56:39.917519+0000","last_clean":"2024-09-10T15:56:39.917519+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:39.917519+0000","last_undegraded":"2024-09-10T15:56:39.917519+0000","last_fullsized":"2024-09-10T15:56:39.917519+0000","mapping_epoch":36,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42045440,"num_objects":10265,"num_object_clones":0,"num_object_copies":20530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10417,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10202","reported_seq":10329,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:39.231786+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:39.231786+0000","last_peered":"2024-09-10T15:56:39.231786+0000","last_clean":"2024-09-10T15:56:39.231786+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:39.231786+0000","last_undegraded":"2024-09-10T15:56:39.231786+0000","last_fullsized":"2024-09-10T15:56:39.231786+0000","mapping_epoch":35,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41488384,"num_objects":10129,"num_object_clones":0,"num_object_copies":20258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7273,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10475","reported_seq":13170,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:39.229135+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:39.229135+0000","last_peered":"2024-09-10T15:56:39.229135+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:39.229135+0000","last_undegraded":"2024-09-10T15:56:39.229135+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42565632,"num_objects":10392,"num_object_clones":0,"num_object_copies":20784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2588,"num_objects_unfound":0,"num_objects_dirty":10392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7193,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1280,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7804},{"shards":"0,2","objects":2588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10182","reported_seq":10326,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.144570+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:39.144570+0000","last_peered":"2024-09-10T15:56:39.144570+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:39.144570+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41385984,"num_objects":10104,"num_object_clones":0,"num_object_copies":20208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6832,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6272},{"shards":"0,2","objects":3798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10510","reported_seq":10641,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:39.168831+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:39.168831+0000","last_peered":"2024-09-10T15:56:39.168831+0000","last_clean":"2024-09-10T15:56:39.168831+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:39.168831+0000","last_undegraded":"2024-09-10T15:56:39.168831+0000","last_fullsized":"2024-09-10T15:56:39.168831+0000","mapping_epoch":35,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42614784,"num_objects":10404,"num_object_clones":0,"num_object_copies":20808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7269,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10403","reported_seq":18530,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:39.292307+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:39.292307+0000","last_peered":"2024-09-10T15:56:39.292307+0000","last_clean":"2024-09-10T15:56:39.292307+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:39.292307+0000","last_undegraded":"2024-09-10T15:56:39.292307+0000","last_fullsized":"2024-09-10T15:56:39.292307+0000","mapping_epoch":45,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42254336,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8380,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10317","reported_seq":10448,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.209146+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:39.209146+0000","last_peered":"2024-09-10T15:56:39.209146+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:39.209146+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42029056,"num_objects":10261,"num_object_clones":0,"num_object_copies":20522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3863,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7181,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6400},{"shards":"0,2","objects":3839}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10444","reported_seq":10571,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.735990+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:39.735990+0000","last_peered":"2024-09-10T15:56:39.735990+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:39.735990+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42356736,"num_objects":10341,"num_object_clones":0,"num_object_copies":20682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8418,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3783},{"shards":"2","objects":6510}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10289","reported_seq":10415,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.839054+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:39.839054+0000","last_peered":"2024-09-10T15:56:39.839054+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:39.839054+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41644032,"num_objects":10167,"num_object_clones":0,"num_object_copies":20334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8158,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3770},{"shards":"2","objects":6338}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10461","reported_seq":10590,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.189886+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:39.189886+0000","last_peered":"2024-09-10T15:56:39.189886+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:39.189886+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42569728,"num_objects":10393,"num_object_clones":0,"num_object_copies":20786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7653,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6561},{"shards":"0,2","objects":3800}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10414","reported_seq":10590,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.255885+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:39.255885+0000","last_peered":"2024-09-10T15:56:39.255885+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:39.255885+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42205184,"num_objects":10304,"num_object_clones":0,"num_object_copies":20608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6832,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6393},{"shards":"0,2","objects":3866}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10431","reported_seq":10573,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.239141+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:39.239141+0000","last_peered":"2024-09-10T15:56:39.239141+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:39.239141+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42332160,"num_objects":10335,"num_object_clones":0,"num_object_copies":20670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3878,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7037,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6459},{"shards":"0,2","objects":3831}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10606","reported_seq":10747,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.189098+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:39.189098+0000","last_peered":"2024-09-10T15:56:39.189098+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:39.189098+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42975715,"num_objects":10493,"num_object_clones":0,"num_object_copies":20986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7425,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6491},{"shards":"0,2","objects":3952}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":675758587,"num_objects":164982,"num_object_clones":0,"num_object_copies":329964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38761,"num_objects_misplaced":2588,"num_objects_unfound":0,"num_objects_dirty":164982,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128068,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5296,"num_bytes_recovered":21602304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1374814208,"data_stored":1374722965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160647,"ondisk_log_size":160647,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626475,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570208,"kb_used_data":331728,"kb_used_omap":4,"kb_used_meta":238459,"kb_avail":93801632,"statfs":{"total":96636764160,"available":96052871168,"internally_reserved":0,"allocated":339689472,"data_stored":339347087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244182702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149758,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":900964,"kb_used_data":558356,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93470876,"statfs":{"total":96636764160,"available":95714177024,"internally_reserved":0,"allocated":571756544,"data_stored":571490491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350809773},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,8,1,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725428,"kb_used_data":455012,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93646412,"statfs":{"total":96636764160,"available":95893925888,"internally_reserved":0,"allocated":465932288,"data_stored":465662163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":570859520,"data_stored":570855907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339226624,"data_stored":339146679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":464728064,"data_stored":464720379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:44.787 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:45.989 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:45.989 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:46.018 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-10T15:56:44.502739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669590030,"num_objects":163478,"num_object_clones":0,"num_object_copies":326956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38659,"num_objects_misplaced":2458,"num_objects_unfound":0,"num_objects_dirty":163478,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129576,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5530,"num_bytes_recovered":22134803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2203588,"kb_used_data":1338196,"kb_used_omap":12,"kb_used_meta":865331,"kb_avail":280911932,"statfs":{"total":289910292480,"available":287653818368,"internally_reserved":0,"allocated":1370312704,"data_stored":1369434141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":886098949},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,11,8,2,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5914648,"num_objects":1445,"num_object_clones":0,"num_object_copies":2890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-122,"num_objects_misplaced":-290,"num_objects_unfound":0,"num_objects_dirty":1445,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5941,"num_write_kb":14769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":1187840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003268"},"pg_stats":[{"pgid":"2.f","version":"46'10558","reported_seq":10673,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.317485+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:44.317485+0000","last_peered":"2024-09-10T15:56:44.317485+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:44.317485+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41828352,"num_objects":10212,"num_object_clones":0,"num_object_copies":20424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10558,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6258},{"shards":"0,2","objects":3800}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10598","reported_seq":10705,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:39.812695+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:39.812695+0000","last_peered":"2024-09-10T15:56:39.812695+0000","last_clean":"2024-09-10T15:56:39.812695+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:39.812695+0000","last_undegraded":"2024-09-10T15:56:39.812695+0000","last_fullsized":"2024-09-10T15:56:39.812695+0000","mapping_epoch":36,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42909696,"num_objects":10476,"num_object_clones":0,"num_object_copies":20952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8934,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10372","reported_seq":10502,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.825205+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:39.825205+0000","last_peered":"2024-09-10T15:56:39.825205+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:39.825205+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42012696,"num_objects":10258,"num_object_clones":0,"num_object_copies":20516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10258,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8640,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3745},{"shards":"2","objects":6459}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10417","reported_seq":10496,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:39.917519+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:39.917519+0000","last_peered":"2024-09-10T15:56:39.917519+0000","last_clean":"2024-09-10T15:56:39.917519+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:39.917519+0000","last_undegraded":"2024-09-10T15:56:39.917519+0000","last_fullsized":"2024-09-10T15:56:39.917519+0000","mapping_epoch":36,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42045440,"num_objects":10265,"num_object_clones":0,"num_object_copies":20530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10417,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10335","reported_seq":10462,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.296882+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:44.296882+0000","last_peered":"2024-09-10T15:56:44.296882+0000","last_clean":"2024-09-10T15:56:44.296882+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:44.296882+0000","last_undegraded":"2024-09-10T15:56:44.296882+0000","last_fullsized":"2024-09-10T15:56:44.296882+0000","mapping_epoch":35,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40943616,"num_objects":9996,"num_object_clones":0,"num_object_copies":19992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7406,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10588","reported_seq":13544,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:44.312190+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:44.312190+0000","last_peered":"2024-09-10T15:56:44.312190+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:44.312190+0000","last_undegraded":"2024-09-10T15:56:44.312190+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42102784,"num_objects":10279,"num_object_clones":0,"num_object_copies":20558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2458,"num_objects_unfound":0,"num_objects_dirty":10279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1410,"num_bytes_recovered":5775360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7821},{"shards":"0,2","objects":2458}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10308","reported_seq":10469,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.296681+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:44.296681+0000","last_peered":"2024-09-10T15:56:44.296681+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:44.296681+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40869888,"num_objects":9978,"num_object_clones":0,"num_object_copies":19956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6155},{"shards":"0,2","objects":3672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10653","reported_seq":10784,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.246243+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:44.246243+0000","last_peered":"2024-09-10T15:56:44.246243+0000","last_clean":"2024-09-10T15:56:44.246243+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:44.246243+0000","last_undegraded":"2024-09-10T15:56:44.246243+0000","last_fullsized":"2024-09-10T15:56:44.246243+0000","mapping_epoch":35,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42029056,"num_objects":10261,"num_object_clones":0,"num_object_copies":20522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7412,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10545","reported_seq":18672,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.271080+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:44.271080+0000","last_peered":"2024-09-10T15:56:44.271080+0000","last_clean":"2024-09-10T15:56:44.271080+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:44.271080+0000","last_undegraded":"2024-09-10T15:56:44.271080+0000","last_fullsized":"2024-09-10T15:56:44.271080+0000","mapping_epoch":45,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41672704,"num_objects":10174,"num_object_clones":0,"num_object_copies":20348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8522,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10461","reported_seq":10620,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.330380+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:44.330380+0000","last_peered":"2024-09-10T15:56:44.330380+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:44.330380+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41439232,"num_objects":10117,"num_object_clones":0,"num_object_copies":20234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3849,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7325,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6270},{"shards":"0,2","objects":3695}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10444","reported_seq":10571,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.735990+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:39.735990+0000","last_peered":"2024-09-10T15:56:39.735990+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:39.735990+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42356736,"num_objects":10341,"num_object_clones":0,"num_object_copies":20682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8418,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3783},{"shards":"2","objects":6510}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10289","reported_seq":10415,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:39.839054+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:39.839054+0000","last_peered":"2024-09-10T15:56:39.839054+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:39.839054+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41644032,"num_objects":10167,"num_object_clones":0,"num_object_copies":20334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8158,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3770},{"shards":"2","objects":6338}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10601","reported_seq":10760,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.322344+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:44.322344+0000","last_peered":"2024-09-10T15:56:44.322344+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:44.322344+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41996288,"num_objects":10253,"num_object_clones":0,"num_object_copies":20506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7793,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6436},{"shards":"0,2","objects":3660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10570","reported_seq":10772,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.288016+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:44.288016+0000","last_peered":"2024-09-10T15:56:44.288016+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:44.288016+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41566208,"num_objects":10148,"num_object_clones":0,"num_object_copies":20296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6250},{"shards":"0,2","objects":3710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10561","reported_seq":10726,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.304486+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:44.304486+0000","last_peered":"2024-09-10T15:56:44.304486+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:44.304486+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41799680,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3866,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7167,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6341},{"shards":"0,2","objects":3701}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10753","reported_seq":10940,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.327585+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:44.327585+0000","last_peered":"2024-09-10T15:56:44.327585+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:44.327585+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42373603,"num_objects":10346,"num_object_clones":0,"num_object_copies":20692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7572,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6367},{"shards":"0,2","objects":3805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":669590011,"num_objects":163476,"num_object_clones":0,"num_object_copies":326952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38659,"num_objects_misplaced":2458,"num_objects_unfound":0,"num_objects_dirty":163476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129574,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5528,"num_bytes_recovered":22134784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1367666688,"data_stored":1367575445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626476,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579504,"kb_used_data":331296,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93792336,"statfs":{"total":96636764160,"available":96043352064,"internally_reserved":0,"allocated":339247104,"data_stored":338904719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254144174},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149759,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":898656,"kb_used_data":551888,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93473184,"statfs":{"total":96636764160,"available":95716540416,"internally_reserved":0,"allocated":565133312,"data_stored":564867259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355069613},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,3,2,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725428,"kb_used_data":455012,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93646412,"statfs":{"total":96636764160,"available":95893925888,"internally_reserved":0,"allocated":465932288,"data_stored":465662163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":564215808,"data_stored":564212195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338722816,"data_stored":338642871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":464728064,"data_stored":464720379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:46.019 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:46.565 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:46.566 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:46.592 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-10T15:56:46.503401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666866190,"num_objects":162813,"num_object_clones":0,"num_object_copies":325626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38645,"num_objects_misplaced":2458,"num_objects_unfound":0,"num_objects_dirty":162813,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130241,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5544,"num_bytes_recovered":22134803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2203516,"kb_used_data":1333964,"kb_used_omap":12,"kb_used_meta":869491,"kb_avail":280912004,"statfs":{"total":289910292480,"available":287653892096,"internally_reserved":0,"allocated":1365979136,"data_stored":1365100573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":890358789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,10,6,7,0,0,5,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2994152,"num_objects":-730,"num_object_clones":0,"num_object_copies":-1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-136,"num_objects_misplaced":-218,"num_objects_unfound":0,"num_objects_dirty":-730,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5096,"num_write_kb":8729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":892928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003295"},"pg_stats":[{"pgid":"2.f","version":"46'10558","reported_seq":10673,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.317485+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:44.317485+0000","last_peered":"2024-09-10T15:56:44.317485+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:44.317485+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41828352,"num_objects":10212,"num_object_clones":0,"num_object_copies":20424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10558,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6258},{"shards":"0,2","objects":3800}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10723","reported_seq":10830,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.822662+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:44.822662+0000","last_peered":"2024-09-10T15:56:44.822662+0000","last_clean":"2024-09-10T15:56:44.822662+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:44.822662+0000","last_undegraded":"2024-09-10T15:56:44.822662+0000","last_fullsized":"2024-09-10T15:56:44.822662+0000","mapping_epoch":36,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42397696,"num_objects":10351,"num_object_clones":0,"num_object_copies":20702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9059,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10509","reported_seq":10653,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.804710+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:44.804710+0000","last_peered":"2024-09-10T15:56:44.804710+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:44.804710+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41451544,"num_objects":10121,"num_object_clones":0,"num_object_copies":20242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8777,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3608},{"shards":"2","objects":6329}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10558","reported_seq":10637,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.917990+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:44.917990+0000","last_peered":"2024-09-10T15:56:44.917990+0000","last_clean":"2024-09-10T15:56:44.917990+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:44.917990+0000","last_undegraded":"2024-09-10T15:56:44.917990+0000","last_fullsized":"2024-09-10T15:56:44.917990+0000","mapping_epoch":36,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41467904,"num_objects":10124,"num_object_clones":0,"num_object_copies":20248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10558,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10335","reported_seq":10462,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.296882+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:44.296882+0000","last_peered":"2024-09-10T15:56:44.296882+0000","last_clean":"2024-09-10T15:56:44.296882+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:44.296882+0000","last_undegraded":"2024-09-10T15:56:44.296882+0000","last_fullsized":"2024-09-10T15:56:44.296882+0000","mapping_epoch":35,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40943616,"num_objects":9996,"num_object_clones":0,"num_object_copies":19992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7406,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10588","reported_seq":13544,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:44.312190+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:44.312190+0000","last_peered":"2024-09-10T15:56:44.312190+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:44.312190+0000","last_undegraded":"2024-09-10T15:56:44.312190+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42102784,"num_objects":10279,"num_object_clones":0,"num_object_copies":20558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2458,"num_objects_unfound":0,"num_objects_dirty":10279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1410,"num_bytes_recovered":5775360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7821},{"shards":"0,2","objects":2458}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10308","reported_seq":10469,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.296681+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:44.296681+0000","last_peered":"2024-09-10T15:56:44.296681+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:44.296681+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40869888,"num_objects":9978,"num_object_clones":0,"num_object_copies":19956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6155},{"shards":"0,2","objects":3672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10653","reported_seq":10784,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.246243+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:44.246243+0000","last_peered":"2024-09-10T15:56:44.246243+0000","last_clean":"2024-09-10T15:56:44.246243+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:44.246243+0000","last_undegraded":"2024-09-10T15:56:44.246243+0000","last_fullsized":"2024-09-10T15:56:44.246243+0000","mapping_epoch":35,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42029056,"num_objects":10261,"num_object_clones":0,"num_object_copies":20522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7412,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10545","reported_seq":18672,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.271080+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:44.271080+0000","last_peered":"2024-09-10T15:56:44.271080+0000","last_clean":"2024-09-10T15:56:44.271080+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:44.271080+0000","last_undegraded":"2024-09-10T15:56:44.271080+0000","last_fullsized":"2024-09-10T15:56:44.271080+0000","mapping_epoch":45,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41672704,"num_objects":10174,"num_object_clones":0,"num_object_copies":20348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8522,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10461","reported_seq":10620,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.330380+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:44.330380+0000","last_peered":"2024-09-10T15:56:44.330380+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:44.330380+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41439232,"num_objects":10117,"num_object_clones":0,"num_object_copies":20234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3849,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7325,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6270},{"shards":"0,2","objects":3695}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10574","reported_seq":10707,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.763883+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:44.763883+0000","last_peered":"2024-09-10T15:56:44.763883+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:44.763883+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41824256,"num_objects":10211,"num_object_clones":0,"num_object_copies":20422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8548,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3653},{"shards":"2","objects":6383}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10421","reported_seq":10555,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.823171+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:44.823171+0000","last_peered":"2024-09-10T15:56:44.823171+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:44.823171+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41103360,"num_objects":10035,"num_object_clones":0,"num_object_copies":20070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8290,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3638},{"shards":"2","objects":6210}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10601","reported_seq":10760,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.322344+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:44.322344+0000","last_peered":"2024-09-10T15:56:44.322344+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:44.322344+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41996288,"num_objects":10253,"num_object_clones":0,"num_object_copies":20506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7793,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6436},{"shards":"0,2","objects":3660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10570","reported_seq":10772,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.288016+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:44.288016+0000","last_peered":"2024-09-10T15:56:44.288016+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:44.288016+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41566208,"num_objects":10148,"num_object_clones":0,"num_object_copies":20296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6250},{"shards":"0,2","objects":3710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10561","reported_seq":10726,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.304486+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:44.304486+0000","last_peered":"2024-09-10T15:56:44.304486+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:44.304486+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41799680,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3866,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7167,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6341},{"shards":"0,2","objects":3701}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10753","reported_seq":10940,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.327585+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:44.327585+0000","last_peered":"2024-09-10T15:56:44.327585+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:44.327585+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42373603,"num_objects":10346,"num_object_clones":0,"num_object_copies":20692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7572,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6367},{"shards":"0,2","objects":3805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":666866171,"num_objects":162811,"num_object_clones":0,"num_object_copies":325622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38645,"num_objects_misplaced":2458,"num_objects_unfound":0,"num_objects_dirty":162811,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130239,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5542,"num_bytes_recovered":22134784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1363251200,"data_stored":1363159957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626476,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579504,"kb_used_data":331296,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93792336,"statfs":{"total":96636764160,"available":96043352064,"internally_reserved":0,"allocated":339247104,"data_stored":338904719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254144174},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149759,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":898656,"kb_used_data":551888,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93473184,"statfs":{"total":96636764160,"available":95716540416,"internally_reserved":0,"allocated":565133312,"data_stored":564867259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355069613},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,3,2,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725356,"kb_used_data":450780,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93646484,"statfs":{"total":96636764160,"available":95893999616,"internally_reserved":0,"allocated":461598720,"data_stored":461328595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,5,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":564215808,"data_stored":564212195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338722816,"data_stored":338642871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":460312576,"data_stored":460304891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:46.593 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:56:46.593 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'10558", 'reported_seq': 10673, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:44.317485+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:56:44.317485+0000', 'last_peered': '2024-09-10T15:56:44.317485+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:56:44.317485+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41828352, 'num_objects': 10212, 'num_object_clones': 0, 'num_object_copies': 20424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3954, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10212, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10558, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 19, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6258}, {'shards': '0,2', 'objects': 3800}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:46.593 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:56:46.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'10509", 'reported_seq': 10653, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:44.804710+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:56:44.804710+0000', 'last_peered': '2024-09-10T15:56:44.804710+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:56:44.804710+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41451544, 'num_objects': 10121, 'num_object_clones': 0, 'num_object_copies': 20242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3793, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10121, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8777, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3608}, {'shards': '2', 'objects': 6329}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:46.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:56:46.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'10588", 'reported_seq': 13544, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:56:44.312190+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:56:44.312190+0000', 'last_peered': '2024-09-10T15:56:44.312190+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:56:44.312190+0000', 'last_undegraded': '2024-09-10T15:56:44.312190+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42102784, 'num_objects': 10279, 'num_object_clones': 0, 'num_object_copies': 20558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2458, 'num_objects_unfound': 0, 'num_objects_dirty': 10279, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7306, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1410, 'num_bytes_recovered': 5775360, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7821}, {'shards': '0,2', 'objects': 2458}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:46.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:56:46.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'10308", 'reported_seq': 10469, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:44.296681+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:56:44.296681+0000', 'last_peered': '2024-09-10T15:56:44.296681+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:56:44.296681+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'300", 'ondisk_log_start': "37'300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40869888, 'num_objects': 9978, 'num_object_clones': 0, 'num_object_copies': 19956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3825, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9978, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6958, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6155}, {'shards': '0,2', 'objects': 3672}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:46.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:56:46.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'10461", 'reported_seq': 10620, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:44.330380+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:56:44.330380+0000', 'last_peered': '2024-09-10T15:56:44.330380+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:56:44.330380+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41439232, 'num_objects': 10117, 'num_object_clones': 0, 'num_object_copies': 20234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3849, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7325, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6270}, {'shards': '0,2', 'objects': 3695}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:46.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:56:46.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'10574", 'reported_seq': 10707, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:44.763883+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:56:44.763883+0000', 'last_peered': '2024-09-10T15:56:44.763883+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:56:44.763883+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41824256, 'num_objects': 10211, 'num_object_clones': 0, 'num_object_copies': 20422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3830, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10211, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8548, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3653}, {'shards': '2', 'objects': 6383}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:46.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:56:46.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'10421", 'reported_seq': 10555, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:44.823171+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:56:44.823171+0000', 'last_peered': '2024-09-10T15:56:44.823171+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:56:44.823171+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41103360, 'num_objects': 10035, 'num_object_clones': 0, 'num_object_copies': 20070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3827, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10035, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8290, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3638}, {'shards': '2', 'objects': 6210}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:46.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:56:46.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'10601", 'reported_seq': 10760, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:44.322344+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:56:44.322344+0000', 'last_peered': '2024-09-10T15:56:44.322344+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:56:44.322344+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'600", 'ondisk_log_start': "37'600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41996288, 'num_objects': 10253, 'num_object_clones': 0, 'num_object_copies': 20506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3818, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10253, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7793, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6436}, {'shards': '0,2', 'objects': 3660}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:46.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:56:46.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'10570", 'reported_seq': 10772, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:44.288016+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:56:44.288016+0000', 'last_peered': '2024-09-10T15:56:44.288016+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:56:44.288016+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41566208, 'num_objects': 10148, 'num_object_clones': 0, 'num_object_copies': 20296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3901, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10148, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6988, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 19, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6250}, {'shards': '0,2', 'objects': 3710}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:46.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:56:46.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'10561", 'reported_seq': 10726, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:44.304486+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:56:44.304486+0000', 'last_peered': '2024-09-10T15:56:44.304486+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:56:44.304486+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41799680, 'num_objects': 10205, 'num_object_clones': 0, 'num_object_copies': 20410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3866, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7167, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 14, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6341}, {'shards': '0,2', 'objects': 3701}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:46.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:56:46.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'10753", 'reported_seq': 10940, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:44.327585+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:56:44.327585+0000', 'last_peered': '2024-09-10T15:56:44.327585+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:56:44.327585+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42373603, 'num_objects': 10346, 'num_object_clones': 0, 'num_object_copies': 20692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10346, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7572, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6367}, {'shards': '0,2', 'objects': 3805}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:46.596 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:56:47.270 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:47.271 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38645/325626 objects degraded (11.868%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":333,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":162813,"data_bytes":666866190,"bytes_used":2256400384,"bytes_avail":287653892096,"bytes_total":289910292480,"degraded_objects":38645,"degraded_total":325626,"degraded_ratio":0.11867909810641657,"misplaced_objects":2458,"misplaced_total":325626,"misplaced_ratio":0.0075485372789642104,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":74390,"recovering_keys_per_sec":0,"num_objects_recovered":354,"num_bytes_recovered":892928,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":744670,"read_op_per_sec":0,"write_op_per_sec":424},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T15:56:24.496337+0000","services":{}},"progress_events":{}} 2024-09-10T15:56:47.299 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:56:47.299 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:47.849 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:47.849 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:47.878 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-10T15:56:46.503401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666866190,"num_objects":162813,"num_object_clones":0,"num_object_copies":325626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38645,"num_objects_misplaced":2458,"num_objects_unfound":0,"num_objects_dirty":162813,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130241,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5544,"num_bytes_recovered":22134803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2203516,"kb_used_data":1333964,"kb_used_omap":12,"kb_used_meta":869491,"kb_avail":280912004,"statfs":{"total":289910292480,"available":287653892096,"internally_reserved":0,"allocated":1365979136,"data_stored":1365100573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":890358789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,10,6,7,0,0,5,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2994152,"num_objects":-730,"num_object_clones":0,"num_object_copies":-1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-136,"num_objects_misplaced":-218,"num_objects_unfound":0,"num_objects_dirty":-730,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5096,"num_write_kb":8729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":892928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003295"},"pg_stats":[{"pgid":"2.f","version":"46'10558","reported_seq":10673,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.317485+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:44.317485+0000","last_peered":"2024-09-10T15:56:44.317485+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:44.317485+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41828352,"num_objects":10212,"num_object_clones":0,"num_object_copies":20424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10558,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6258},{"shards":"0,2","objects":3800}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10723","reported_seq":10830,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.822662+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:44.822662+0000","last_peered":"2024-09-10T15:56:44.822662+0000","last_clean":"2024-09-10T15:56:44.822662+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:44.822662+0000","last_undegraded":"2024-09-10T15:56:44.822662+0000","last_fullsized":"2024-09-10T15:56:44.822662+0000","mapping_epoch":36,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42397696,"num_objects":10351,"num_object_clones":0,"num_object_copies":20702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9059,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10509","reported_seq":10653,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.804710+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:44.804710+0000","last_peered":"2024-09-10T15:56:44.804710+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:44.804710+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41451544,"num_objects":10121,"num_object_clones":0,"num_object_copies":20242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8777,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3608},{"shards":"2","objects":6329}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10558","reported_seq":10637,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.917990+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:44.917990+0000","last_peered":"2024-09-10T15:56:44.917990+0000","last_clean":"2024-09-10T15:56:44.917990+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:44.917990+0000","last_undegraded":"2024-09-10T15:56:44.917990+0000","last_fullsized":"2024-09-10T15:56:44.917990+0000","mapping_epoch":36,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41467904,"num_objects":10124,"num_object_clones":0,"num_object_copies":20248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10558,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10335","reported_seq":10462,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.296882+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:44.296882+0000","last_peered":"2024-09-10T15:56:44.296882+0000","last_clean":"2024-09-10T15:56:44.296882+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:44.296882+0000","last_undegraded":"2024-09-10T15:56:44.296882+0000","last_fullsized":"2024-09-10T15:56:44.296882+0000","mapping_epoch":35,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40943616,"num_objects":9996,"num_object_clones":0,"num_object_copies":19992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7406,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10588","reported_seq":13544,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:44.312190+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:44.312190+0000","last_peered":"2024-09-10T15:56:44.312190+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:44.312190+0000","last_undegraded":"2024-09-10T15:56:44.312190+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42102784,"num_objects":10279,"num_object_clones":0,"num_object_copies":20558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2458,"num_objects_unfound":0,"num_objects_dirty":10279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1410,"num_bytes_recovered":5775360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7821},{"shards":"0,2","objects":2458}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10308","reported_seq":10469,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.296681+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:44.296681+0000","last_peered":"2024-09-10T15:56:44.296681+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:44.296681+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40869888,"num_objects":9978,"num_object_clones":0,"num_object_copies":19956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6155},{"shards":"0,2","objects":3672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10653","reported_seq":10784,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.246243+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:44.246243+0000","last_peered":"2024-09-10T15:56:44.246243+0000","last_clean":"2024-09-10T15:56:44.246243+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:44.246243+0000","last_undegraded":"2024-09-10T15:56:44.246243+0000","last_fullsized":"2024-09-10T15:56:44.246243+0000","mapping_epoch":35,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42029056,"num_objects":10261,"num_object_clones":0,"num_object_copies":20522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7412,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10545","reported_seq":18672,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:44.271080+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:44.271080+0000","last_peered":"2024-09-10T15:56:44.271080+0000","last_clean":"2024-09-10T15:56:44.271080+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:44.271080+0000","last_undegraded":"2024-09-10T15:56:44.271080+0000","last_fullsized":"2024-09-10T15:56:44.271080+0000","mapping_epoch":45,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41672704,"num_objects":10174,"num_object_clones":0,"num_object_copies":20348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8522,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10461","reported_seq":10620,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.330380+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:44.330380+0000","last_peered":"2024-09-10T15:56:44.330380+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:44.330380+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41439232,"num_objects":10117,"num_object_clones":0,"num_object_copies":20234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3849,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7325,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6270},{"shards":"0,2","objects":3695}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10574","reported_seq":10707,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.763883+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:44.763883+0000","last_peered":"2024-09-10T15:56:44.763883+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:44.763883+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41824256,"num_objects":10211,"num_object_clones":0,"num_object_copies":20422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8548,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3653},{"shards":"2","objects":6383}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10421","reported_seq":10555,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.823171+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:44.823171+0000","last_peered":"2024-09-10T15:56:44.823171+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:44.823171+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41103360,"num_objects":10035,"num_object_clones":0,"num_object_copies":20070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8290,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3638},{"shards":"2","objects":6210}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10601","reported_seq":10760,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.322344+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:44.322344+0000","last_peered":"2024-09-10T15:56:44.322344+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:44.322344+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41996288,"num_objects":10253,"num_object_clones":0,"num_object_copies":20506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7793,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6436},{"shards":"0,2","objects":3660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10570","reported_seq":10772,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.288016+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:44.288016+0000","last_peered":"2024-09-10T15:56:44.288016+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:44.288016+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41566208,"num_objects":10148,"num_object_clones":0,"num_object_copies":20296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6250},{"shards":"0,2","objects":3710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10561","reported_seq":10726,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.304486+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:44.304486+0000","last_peered":"2024-09-10T15:56:44.304486+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:44.304486+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41799680,"num_objects":10205,"num_object_clones":0,"num_object_copies":20410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3866,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7167,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6341},{"shards":"0,2","objects":3701}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10753","reported_seq":10940,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:44.327585+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:44.327585+0000","last_peered":"2024-09-10T15:56:44.327585+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:44.327585+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42373603,"num_objects":10346,"num_object_clones":0,"num_object_copies":20692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7572,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6367},{"shards":"0,2","objects":3805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":666866171,"num_objects":162811,"num_object_clones":0,"num_object_copies":325622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38645,"num_objects_misplaced":2458,"num_objects_unfound":0,"num_objects_dirty":162811,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130239,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5542,"num_bytes_recovered":22134784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1363251200,"data_stored":1363159957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626476,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579504,"kb_used_data":331296,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93792336,"statfs":{"total":96636764160,"available":96043352064,"internally_reserved":0,"allocated":339247104,"data_stored":338904719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254144174},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149759,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":898656,"kb_used_data":551888,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93473184,"statfs":{"total":96636764160,"available":95716540416,"internally_reserved":0,"allocated":565133312,"data_stored":564867259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355069613},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,3,2,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725356,"kb_used_data":450780,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93646484,"statfs":{"total":96636764160,"available":95893999616,"internally_reserved":0,"allocated":461598720,"data_stored":461328595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,5,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":564215808,"data_stored":564212195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338722816,"data_stored":338642871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":460312576,"data_stored":460304891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:50.880 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:51.404 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:51.404 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:51.429 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-10T15:56:50.504750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658100750,"num_objects":160673,"num_object_clones":0,"num_object_copies":321346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38524,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":160673,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132381,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5799,"num_bytes_recovered":22683667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160761,"ondisk_log_size":160761,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2206360,"kb_used_data":1322920,"kb_used_omap":12,"kb_used_meta":883379,"kb_avail":280909160,"statfs":{"total":289910292480,"available":287650979840,"internally_reserved":0,"allocated":1354670080,"data_stored":1353791517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":904580101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,11,7,11,5,6,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14532584,"num_objects":-3547,"num_object_clones":0,"num_object_copies":-7094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-257,"num_objects_misplaced":-352,"num_objects_unfound":0,"num_objects_dirty":-3547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6559,"num_write_kb":6021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":1441792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003587"},"pg_stats":[{"pgid":"2.f","version":"46'10702","reported_seq":10844,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.377470+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:49.377470+0000","last_peered":"2024-09-10T15:56:49.377470+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:49.377470+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41238528,"num_objects":10068,"num_object_clones":0,"num_object_copies":20136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3940,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10702,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6128},{"shards":"0,2","objects":3656}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10843","reported_seq":10950,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.858669+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:49.858669+0000","last_peered":"2024-09-10T15:56:49.858669+0000","last_clean":"2024-09-10T15:56:49.858669+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:49.858669+0000","last_undegraded":"2024-09-10T15:56:49.858669+0000","last_fullsized":"2024-09-10T15:56:49.858669+0000","mapping_epoch":36,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906176,"num_objects":10231,"num_object_clones":0,"num_object_copies":20462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9179,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10638","reported_seq":10810,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.906074+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:49.906074+0000","last_peered":"2024-09-10T15:56:49.906074+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:49.906074+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923160,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3779,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8906,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3479},{"shards":"2","objects":6214}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10698","reported_seq":10777,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.919526+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:49.919526+0000","last_peered":"2024-09-10T15:56:49.919526+0000","last_clean":"2024-09-10T15:56:49.919526+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:49.919526+0000","last_undegraded":"2024-09-10T15:56:49.919526+0000","last_fullsized":"2024-09-10T15:56:49.919526+0000","mapping_epoch":36,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894464,"num_objects":9984,"num_object_clones":0,"num_object_copies":19968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10698,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10462","reported_seq":10588,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.377354+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:49.377354+0000","last_peered":"2024-09-10T15:56:49.377354+0000","last_clean":"2024-09-10T15:56:49.377354+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:49.377354+0000","last_undegraded":"2024-09-10T15:56:49.377354+0000","last_fullsized":"2024-09-10T15:56:49.377354+0000","mapping_epoch":35,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40423424,"num_objects":9869,"num_object_clones":0,"num_object_copies":19738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7533,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10707","reported_seq":13931,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:49.397718+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:49.397718+0000","last_peered":"2024-09-10T15:56:49.397718+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:49.397718+0000","last_undegraded":"2024-09-10T15:56:49.397718+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41615360,"num_objects":10160,"num_object_clones":0,"num_object_copies":20320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":10160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7425,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1544,"num_bytes_recovered":6324224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7836},{"shards":"0,2","objects":2324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10459","reported_seq":10633,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.366212+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:49.366212+0000","last_peered":"2024-09-10T15:56:49.366212+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:49.366212+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40251392,"num_objects":9827,"num_object_clones":0,"num_object_copies":19654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7109,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6010},{"shards":"0,2","objects":3521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10777","reported_seq":10908,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.291691+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:49.291691+0000","last_peered":"2024-09-10T15:56:49.291691+0000","last_clean":"2024-09-10T15:56:49.291691+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:49.291691+0000","last_undegraded":"2024-09-10T15:56:49.291691+0000","last_fullsized":"2024-09-10T15:56:49.291691+0000","mapping_epoch":35,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41521152,"num_objects":10137,"num_object_clones":0,"num_object_copies":20274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7536,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10673","reported_seq":18800,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.359952+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:49.359952+0000","last_peered":"2024-09-10T15:56:49.359952+0000","last_clean":"2024-09-10T15:56:49.359952+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:49.359952+0000","last_undegraded":"2024-09-10T15:56:49.359952+0000","last_fullsized":"2024-09-10T15:56:49.359952+0000","mapping_epoch":45,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41148416,"num_objects":10046,"num_object_clones":0,"num_object_copies":20092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8650,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10587","reported_seq":10788,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.355623+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:49.355623+0000","last_peered":"2024-09-10T15:56:49.355623+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:49.355623+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923136,"num_objects":9991,"num_object_clones":0,"num_object_copies":19982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3828,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7451,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6165},{"shards":"0,2","objects":3569}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10704","reported_seq":10851,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.863489+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:49.863489+0000","last_peered":"2024-09-10T15:56:49.863489+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:49.863489+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41291776,"num_objects":10081,"num_object_clones":0,"num_object_copies":20162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8678,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3523},{"shards":"2","objects":6260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10560","reported_seq":10708,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.904145+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:49.904145+0000","last_peered":"2024-09-10T15:56:49.904145+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:49.904145+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40534016,"num_objects":9896,"num_object_clones":0,"num_object_copies":19792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3820,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8429,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3499},{"shards":"2","objects":6078}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10745","reported_seq":10933,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.338027+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:49.338027+0000","last_peered":"2024-09-10T15:56:49.338027+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:49.338027+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41406464,"num_objects":10109,"num_object_clones":0,"num_object_copies":20218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7937,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6306},{"shards":"0,2","objects":3516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10711","reported_seq":10927,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.323638+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:49.323638+0000","last_peered":"2024-09-10T15:56:49.323638+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:49.323638+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40988672,"num_objects":10007,"num_object_clones":0,"num_object_copies":20014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3894,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7129,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6116},{"shards":"0,2","objects":3569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10706","reported_seq":10904,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.406200+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:49.406200+0000","last_peered":"2024-09-10T15:56:49.406200+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:49.406200+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41205760,"num_objects":10060,"num_object_clones":0,"num_object_copies":20120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3850,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7312,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6212},{"shards":"0,2","objects":3556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10886","reported_seq":11103,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.337675+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:49.337675+0000","last_peered":"2024-09-10T15:56:49.337675+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:49.337675+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41828835,"num_objects":10213,"num_object_clones":0,"num_object_copies":20426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7705,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6249},{"shards":"0,2","objects":3672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":658100731,"num_objects":160671,"num_object_clones":0,"num_object_copies":321342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38524,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":160671,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132379,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5797,"num_bytes_recovered":22683648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1351946240,"data_stored":1351854997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626477,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601188,"kb_used_data":330772,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93770652,"statfs":{"total":96636764160,"available":96021147648,"internally_reserved":0,"allocated":338710528,"data_stored":338368143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276885166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149760,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":883976,"kb_used_data":545528,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93487864,"statfs":{"total":96636764160,"available":95731572736,"internally_reserved":0,"allocated":558620672,"data_stored":558354619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346549933},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,1,1,3,3,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721196,"kb_used_data":446620,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93650644,"statfs":{"total":96636764160,"available":95898259456,"internally_reserved":0,"allocated":457338880,"data_stored":457068755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,5,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":557547520,"data_stored":557543907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338259968,"data_stored":338180023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456138752,"data_stored":456131067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:51.431 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:51.972 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:51.975 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:51.997 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-10T15:56:50.504750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658100750,"num_objects":160673,"num_object_clones":0,"num_object_copies":321346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38524,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":160673,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132381,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5799,"num_bytes_recovered":22683667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160761,"ondisk_log_size":160761,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2206360,"kb_used_data":1322920,"kb_used_omap":12,"kb_used_meta":883379,"kb_avail":280909160,"statfs":{"total":289910292480,"available":287650979840,"internally_reserved":0,"allocated":1354670080,"data_stored":1353791517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":904580101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,11,7,11,5,6,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14532584,"num_objects":-3547,"num_object_clones":0,"num_object_copies":-7094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-257,"num_objects_misplaced":-352,"num_objects_unfound":0,"num_objects_dirty":-3547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6559,"num_write_kb":6021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":1441792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003587"},"pg_stats":[{"pgid":"2.f","version":"46'10702","reported_seq":10844,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.377470+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:49.377470+0000","last_peered":"2024-09-10T15:56:49.377470+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:49.377470+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41238528,"num_objects":10068,"num_object_clones":0,"num_object_copies":20136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3940,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10702,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6128},{"shards":"0,2","objects":3656}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10843","reported_seq":10950,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.858669+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:49.858669+0000","last_peered":"2024-09-10T15:56:49.858669+0000","last_clean":"2024-09-10T15:56:49.858669+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:49.858669+0000","last_undegraded":"2024-09-10T15:56:49.858669+0000","last_fullsized":"2024-09-10T15:56:49.858669+0000","mapping_epoch":36,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906176,"num_objects":10231,"num_object_clones":0,"num_object_copies":20462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9179,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10638","reported_seq":10810,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.906074+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:49.906074+0000","last_peered":"2024-09-10T15:56:49.906074+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:49.906074+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923160,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3779,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8906,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3479},{"shards":"2","objects":6214}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10698","reported_seq":10777,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.919526+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:49.919526+0000","last_peered":"2024-09-10T15:56:49.919526+0000","last_clean":"2024-09-10T15:56:49.919526+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:49.919526+0000","last_undegraded":"2024-09-10T15:56:49.919526+0000","last_fullsized":"2024-09-10T15:56:49.919526+0000","mapping_epoch":36,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894464,"num_objects":9984,"num_object_clones":0,"num_object_copies":19968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10698,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10462","reported_seq":10588,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.377354+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:49.377354+0000","last_peered":"2024-09-10T15:56:49.377354+0000","last_clean":"2024-09-10T15:56:49.377354+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:49.377354+0000","last_undegraded":"2024-09-10T15:56:49.377354+0000","last_fullsized":"2024-09-10T15:56:49.377354+0000","mapping_epoch":35,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40423424,"num_objects":9869,"num_object_clones":0,"num_object_copies":19738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7533,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10707","reported_seq":13931,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:49.397718+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:49.397718+0000","last_peered":"2024-09-10T15:56:49.397718+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:49.397718+0000","last_undegraded":"2024-09-10T15:56:49.397718+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41615360,"num_objects":10160,"num_object_clones":0,"num_object_copies":20320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":10160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7425,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1544,"num_bytes_recovered":6324224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7836},{"shards":"0,2","objects":2324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10459","reported_seq":10633,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.366212+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:49.366212+0000","last_peered":"2024-09-10T15:56:49.366212+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:49.366212+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40251392,"num_objects":9827,"num_object_clones":0,"num_object_copies":19654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7109,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6010},{"shards":"0,2","objects":3521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10777","reported_seq":10908,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.291691+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:49.291691+0000","last_peered":"2024-09-10T15:56:49.291691+0000","last_clean":"2024-09-10T15:56:49.291691+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:49.291691+0000","last_undegraded":"2024-09-10T15:56:49.291691+0000","last_fullsized":"2024-09-10T15:56:49.291691+0000","mapping_epoch":35,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41521152,"num_objects":10137,"num_object_clones":0,"num_object_copies":20274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7536,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10673","reported_seq":18800,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.359952+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:49.359952+0000","last_peered":"2024-09-10T15:56:49.359952+0000","last_clean":"2024-09-10T15:56:49.359952+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:49.359952+0000","last_undegraded":"2024-09-10T15:56:49.359952+0000","last_fullsized":"2024-09-10T15:56:49.359952+0000","mapping_epoch":45,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41148416,"num_objects":10046,"num_object_clones":0,"num_object_copies":20092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8650,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10587","reported_seq":10788,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.355623+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:49.355623+0000","last_peered":"2024-09-10T15:56:49.355623+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:49.355623+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923136,"num_objects":9991,"num_object_clones":0,"num_object_copies":19982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3828,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7451,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6165},{"shards":"0,2","objects":3569}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10704","reported_seq":10851,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.863489+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:49.863489+0000","last_peered":"2024-09-10T15:56:49.863489+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:49.863489+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41291776,"num_objects":10081,"num_object_clones":0,"num_object_copies":20162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8678,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3523},{"shards":"2","objects":6260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10560","reported_seq":10708,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.904145+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:49.904145+0000","last_peered":"2024-09-10T15:56:49.904145+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:49.904145+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40534016,"num_objects":9896,"num_object_clones":0,"num_object_copies":19792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3820,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8429,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3499},{"shards":"2","objects":6078}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10745","reported_seq":10933,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.338027+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:49.338027+0000","last_peered":"2024-09-10T15:56:49.338027+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:49.338027+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41406464,"num_objects":10109,"num_object_clones":0,"num_object_copies":20218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7937,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6306},{"shards":"0,2","objects":3516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10711","reported_seq":10927,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.323638+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:49.323638+0000","last_peered":"2024-09-10T15:56:49.323638+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:49.323638+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40988672,"num_objects":10007,"num_object_clones":0,"num_object_copies":20014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3894,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7129,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6116},{"shards":"0,2","objects":3569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10706","reported_seq":10904,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.406200+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:49.406200+0000","last_peered":"2024-09-10T15:56:49.406200+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:49.406200+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41205760,"num_objects":10060,"num_object_clones":0,"num_object_copies":20120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3850,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7312,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6212},{"shards":"0,2","objects":3556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10886","reported_seq":11103,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.337675+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:49.337675+0000","last_peered":"2024-09-10T15:56:49.337675+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:49.337675+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41828835,"num_objects":10213,"num_object_clones":0,"num_object_copies":20426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7705,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6249},{"shards":"0,2","objects":3672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":658100731,"num_objects":160671,"num_object_clones":0,"num_object_copies":321342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38524,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":160671,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132379,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5797,"num_bytes_recovered":22683648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1351946240,"data_stored":1351854997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626477,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601188,"kb_used_data":330772,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93770652,"statfs":{"total":96636764160,"available":96021147648,"internally_reserved":0,"allocated":338710528,"data_stored":338368143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276885166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149760,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":883976,"kb_used_data":545528,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93487864,"statfs":{"total":96636764160,"available":95731572736,"internally_reserved":0,"allocated":558620672,"data_stored":558354619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346549933},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,1,1,3,3,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721196,"kb_used_data":446620,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93650644,"statfs":{"total":96636764160,"available":95898259456,"internally_reserved":0,"allocated":457338880,"data_stored":457068755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,5,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":557547520,"data_stored":557543907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338259968,"data_stored":338180023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456138752,"data_stored":456131067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:51.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:56:51.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'10702", 'reported_seq': 10844, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:49.377470+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:56:49.377470+0000', 'last_peered': '2024-09-10T15:56:49.377470+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:56:49.377470+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41238528, 'num_objects': 10068, 'num_object_clones': 0, 'num_object_copies': 20136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3940, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10068, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10702, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 33, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6128}, {'shards': '0,2', 'objects': 3656}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:51.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:56:51.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'10638", 'reported_seq': 10810, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:49.906074+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:56:49.906074+0000', 'last_peered': '2024-09-10T15:56:49.906074+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:56:49.906074+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'600", 'ondisk_log_start': "37'600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40923160, 'num_objects': 9992, 'num_object_clones': 0, 'num_object_copies': 19984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3779, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9992, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8906, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3479}, {'shards': '2', 'objects': 6214}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:51.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:56:51.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'10707", 'reported_seq': 13931, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:56:49.397718+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:56:49.397718+0000', 'last_peered': '2024-09-10T15:56:49.397718+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:56:49.397718+0000', 'last_undegraded': '2024-09-10T15:56:49.397718+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41615360, 'num_objects': 10160, 'num_object_clones': 0, 'num_object_copies': 20320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2324, 'num_objects_unfound': 0, 'num_objects_dirty': 10160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7425, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1544, 'num_bytes_recovered': 6324224, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7836}, {'shards': '0,2', 'objects': 2324}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:51.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:56:51.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'10459", 'reported_seq': 10633, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:49.366212+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:56:49.366212+0000', 'last_peered': '2024-09-10T15:56:49.366212+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:56:49.366212+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40251392, 'num_objects': 9827, 'num_object_clones': 0, 'num_object_copies': 19654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3819, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7109, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6010}, {'shards': '0,2', 'objects': 3521}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:51.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:56:51.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'10587", 'reported_seq': 10788, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:49.355623+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:56:49.355623+0000', 'last_peered': '2024-09-10T15:56:49.355623+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:56:49.355623+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40923136, 'num_objects': 9991, 'num_object_clones': 0, 'num_object_copies': 19982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3828, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9991, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7451, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6165}, {'shards': '0,2', 'objects': 3569}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:51.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:56:51.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'10704", 'reported_seq': 10851, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:49.863489+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:56:49.863489+0000', 'last_peered': '2024-09-10T15:56:49.863489+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:56:49.863489+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41291776, 'num_objects': 10081, 'num_object_clones': 0, 'num_object_copies': 20162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3823, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10081, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8678, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3523}, {'shards': '2', 'objects': 6260}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:51.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:56:51.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'10560", 'reported_seq': 10708, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:49.904145+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:56:49.904145+0000', 'last_peered': '2024-09-10T15:56:49.904145+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:56:49.904145+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40534016, 'num_objects': 9896, 'num_object_clones': 0, 'num_object_copies': 19792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3820, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9896, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8429, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3499}, {'shards': '2', 'objects': 6078}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:51.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:56:51.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'10745", 'reported_seq': 10933, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:49.338027+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:56:49.338027+0000', 'last_peered': '2024-09-10T15:56:49.338027+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:56:49.338027+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41406464, 'num_objects': 10109, 'num_object_clones': 0, 'num_object_copies': 20218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3804, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10109, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7937, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6306}, {'shards': '0,2', 'objects': 3516}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:51.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:56:52.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'10711", 'reported_seq': 10927, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:49.323638+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:56:49.323638+0000', 'last_peered': '2024-09-10T15:56:49.323638+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:56:49.323638+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40988672, 'num_objects': 10007, 'num_object_clones': 0, 'num_object_copies': 20014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3894, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10007, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7129, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6116}, {'shards': '0,2', 'objects': 3569}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:52.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:56:52.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'10706", 'reported_seq': 10904, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:49.406200+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:56:49.406200+0000', 'last_peered': '2024-09-10T15:56:49.406200+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:56:49.406200+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41205760, 'num_objects': 10060, 'num_object_clones': 0, 'num_object_copies': 20120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3850, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10060, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7312, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6212}, {'shards': '0,2', 'objects': 3556}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:52.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:56:52.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'10886", 'reported_seq': 11103, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:49.337675+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:56:49.337675+0000', 'last_peered': '2024-09-10T15:56:49.337675+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:56:49.337675+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41828835, 'num_objects': 10213, 'num_object_clones': 0, 'num_object_copies': 20426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3967, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10213, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7705, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6249}, {'shards': '0,2', 'objects': 3672}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:52.000 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:56:52.686 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:52.687 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38524/321346 objects degraded (11.988%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":338,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":160673,"data_bytes":658100750,"bytes_used":2259312640,"bytes_avail":287650979840,"bytes_total":289910292480,"degraded_objects":38524,"degraded_total":321346,"degraded_ratio":0.1198832411170514,"misplaced_objects":2324,"misplaced_total":321346,"misplaced_ratio":0.0072320800632340223,"recovering_objects_per_sec":50,"recovering_bytes_per_sec":120113,"recovering_keys_per_sec":0,"num_objects_recovered":609,"num_bytes_recovered":1441792,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":513638,"read_op_per_sec":0,"write_op_per_sec":546},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T15:56:24.496337+0000","services":{}},"progress_events":{}} 2024-09-10T15:56:52.715 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:56:52.715 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:53.258 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:53.259 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:53.279 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2024-09-10T15:56:52.505485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658100750,"num_objects":160673,"num_object_clones":0,"num_object_copies":321346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38524,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":160673,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132381,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5799,"num_bytes_recovered":22683667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160761,"ondisk_log_size":160761,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2206360,"kb_used_data":1322920,"kb_used_omap":12,"kb_used_meta":883379,"kb_avail":280909160,"statfs":{"total":289910292480,"available":287650979840,"internally_reserved":0,"allocated":1354670080,"data_stored":1353791517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":904580101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,11,7,11,5,6,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17657856,"num_objects":-4311,"num_object_clones":0,"num_object_copies":-8622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":-264,"num_objects_unfound":0,"num_objects_dirty":-4311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4311,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003886"},"pg_stats":[{"pgid":"2.f","version":"46'10702","reported_seq":10844,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.377470+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:49.377470+0000","last_peered":"2024-09-10T15:56:49.377470+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:49.377470+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41238528,"num_objects":10068,"num_object_clones":0,"num_object_copies":20136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3940,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10702,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6128},{"shards":"0,2","objects":3656}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10843","reported_seq":10950,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.858669+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:49.858669+0000","last_peered":"2024-09-10T15:56:49.858669+0000","last_clean":"2024-09-10T15:56:49.858669+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:49.858669+0000","last_undegraded":"2024-09-10T15:56:49.858669+0000","last_fullsized":"2024-09-10T15:56:49.858669+0000","mapping_epoch":36,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906176,"num_objects":10231,"num_object_clones":0,"num_object_copies":20462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9179,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10638","reported_seq":10810,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.906074+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:49.906074+0000","last_peered":"2024-09-10T15:56:49.906074+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:49.906074+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923160,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3779,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8906,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3479},{"shards":"2","objects":6214}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10698","reported_seq":10777,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.919526+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:49.919526+0000","last_peered":"2024-09-10T15:56:49.919526+0000","last_clean":"2024-09-10T15:56:49.919526+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:49.919526+0000","last_undegraded":"2024-09-10T15:56:49.919526+0000","last_fullsized":"2024-09-10T15:56:49.919526+0000","mapping_epoch":36,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894464,"num_objects":9984,"num_object_clones":0,"num_object_copies":19968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10698,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10462","reported_seq":10588,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.377354+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:49.377354+0000","last_peered":"2024-09-10T15:56:49.377354+0000","last_clean":"2024-09-10T15:56:49.377354+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:49.377354+0000","last_undegraded":"2024-09-10T15:56:49.377354+0000","last_fullsized":"2024-09-10T15:56:49.377354+0000","mapping_epoch":35,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40423424,"num_objects":9869,"num_object_clones":0,"num_object_copies":19738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7533,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10707","reported_seq":13931,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:49.397718+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:49.397718+0000","last_peered":"2024-09-10T15:56:49.397718+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:49.397718+0000","last_undegraded":"2024-09-10T15:56:49.397718+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41615360,"num_objects":10160,"num_object_clones":0,"num_object_copies":20320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":10160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7425,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1544,"num_bytes_recovered":6324224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7836},{"shards":"0,2","objects":2324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10459","reported_seq":10633,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.366212+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:49.366212+0000","last_peered":"2024-09-10T15:56:49.366212+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:49.366212+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40251392,"num_objects":9827,"num_object_clones":0,"num_object_copies":19654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7109,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6010},{"shards":"0,2","objects":3521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10777","reported_seq":10908,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.291691+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:49.291691+0000","last_peered":"2024-09-10T15:56:49.291691+0000","last_clean":"2024-09-10T15:56:49.291691+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:49.291691+0000","last_undegraded":"2024-09-10T15:56:49.291691+0000","last_fullsized":"2024-09-10T15:56:49.291691+0000","mapping_epoch":35,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41521152,"num_objects":10137,"num_object_clones":0,"num_object_copies":20274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7536,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10673","reported_seq":18800,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:49.359952+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:49.359952+0000","last_peered":"2024-09-10T15:56:49.359952+0000","last_clean":"2024-09-10T15:56:49.359952+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:49.359952+0000","last_undegraded":"2024-09-10T15:56:49.359952+0000","last_fullsized":"2024-09-10T15:56:49.359952+0000","mapping_epoch":45,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41148416,"num_objects":10046,"num_object_clones":0,"num_object_copies":20092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8650,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10587","reported_seq":10788,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.355623+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:49.355623+0000","last_peered":"2024-09-10T15:56:49.355623+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:49.355623+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40923136,"num_objects":9991,"num_object_clones":0,"num_object_copies":19982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3828,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7451,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6165},{"shards":"0,2","objects":3569}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10704","reported_seq":10851,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.863489+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:49.863489+0000","last_peered":"2024-09-10T15:56:49.863489+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:49.863489+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41291776,"num_objects":10081,"num_object_clones":0,"num_object_copies":20162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8678,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3523},{"shards":"2","objects":6260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10560","reported_seq":10708,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.904145+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:49.904145+0000","last_peered":"2024-09-10T15:56:49.904145+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:49.904145+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40534016,"num_objects":9896,"num_object_clones":0,"num_object_copies":19792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3820,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8429,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3499},{"shards":"2","objects":6078}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10745","reported_seq":10933,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.338027+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:49.338027+0000","last_peered":"2024-09-10T15:56:49.338027+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:49.338027+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41406464,"num_objects":10109,"num_object_clones":0,"num_object_copies":20218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7937,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6306},{"shards":"0,2","objects":3516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10711","reported_seq":10927,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.323638+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:49.323638+0000","last_peered":"2024-09-10T15:56:49.323638+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:49.323638+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40988672,"num_objects":10007,"num_object_clones":0,"num_object_copies":20014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3894,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7129,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6116},{"shards":"0,2","objects":3569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10706","reported_seq":10904,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.406200+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:49.406200+0000","last_peered":"2024-09-10T15:56:49.406200+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:49.406200+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41205760,"num_objects":10060,"num_object_clones":0,"num_object_copies":20120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3850,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7312,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6212},{"shards":"0,2","objects":3556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'10886","reported_seq":11103,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:49.337675+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:49.337675+0000","last_peered":"2024-09-10T15:56:49.337675+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:49.337675+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41828835,"num_objects":10213,"num_object_clones":0,"num_object_copies":20426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7705,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6249},{"shards":"0,2","objects":3672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":658100731,"num_objects":160671,"num_object_clones":0,"num_object_copies":321342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38524,"num_objects_misplaced":2324,"num_objects_unfound":0,"num_objects_dirty":160671,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132379,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5797,"num_bytes_recovered":22683648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1351946240,"data_stored":1351854997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626477,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601188,"kb_used_data":330772,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93770652,"statfs":{"total":96636764160,"available":96021147648,"internally_reserved":0,"allocated":338710528,"data_stored":338368143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276885166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149760,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":883976,"kb_used_data":545528,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93487864,"statfs":{"total":96636764160,"available":95731572736,"internally_reserved":0,"allocated":558620672,"data_stored":558354619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346549933},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,1,1,3,3,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721196,"kb_used_data":446620,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93650644,"statfs":{"total":96636764160,"available":95898259456,"internally_reserved":0,"allocated":457338880,"data_stored":457068755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,5,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":557547520,"data_stored":557543907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338259968,"data_stored":338180023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456138752,"data_stored":456131067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:56.281 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:56.814 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:56.816 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:56.839 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-10T15:56:56.506900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649155086,"num_objects":158489,"num_object_clones":0,"num_object_copies":316978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38441,"num_objects_misplaced":2209,"num_objects_unfound":0,"num_objects_dirty":158489,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134565,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5997,"num_bytes_recovered":23154707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160945,"ondisk_log_size":160945,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2194916,"kb_used_data":1311456,"kb_used_omap":12,"kb_used_meta":883379,"kb_avail":280920604,"statfs":{"total":289910292480,"available":287662698496,"internally_reserved":0,"allocated":1342930944,"data_stored":1342052381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":904580101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,2,7,8,5,0,0,3,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20434944,"num_objects":-4989,"num_object_clones":0,"num_object_copies":-9978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-218,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":-4989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":467,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004161"},"pg_stats":[{"pgid":"2.f","version":"46'10839","reported_seq":11000,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.431490+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:54.431490+0000","last_peered":"2024-09-10T15:56:54.431490+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:54.431490+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40677376,"num_objects":9931,"num_object_clones":0,"num_object_copies":19862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10839,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6000},{"shards":"0,2","objects":3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10972","reported_seq":11079,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.951778+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:54.951778+0000","last_peered":"2024-09-10T15:56:54.951778+0000","last_clean":"2024-09-10T15:56:54.951778+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:54.951778+0000","last_undegraded":"2024-09-10T15:56:54.951778+0000","last_fullsized":"2024-09-10T15:56:54.951778+0000","mapping_epoch":36,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41377792,"num_objects":10102,"num_object_clones":0,"num_object_copies":20204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10759","reported_seq":10945,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.841200+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:54.841200+0000","last_peered":"2024-09-10T15:56:54.841200+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:54.841200+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427544,"num_objects":9871,"num_object_clones":0,"num_object_copies":19742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3772,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9871,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9027,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3358},{"shards":"2","objects":6100}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10831","reported_seq":10911,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.949966+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:54.949966+0000","last_peered":"2024-09-10T15:56:54.949966+0000","last_clean":"2024-09-10T15:56:54.949966+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:54.949966+0000","last_undegraded":"2024-09-10T15:56:54.949966+0000","last_fullsized":"2024-09-10T15:56:54.949966+0000","mapping_epoch":36,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40349696,"num_objects":9851,"num_object_clones":0,"num_object_copies":19702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10831,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10594","reported_seq":10721,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.408319+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:54.408319+0000","last_peered":"2024-09-10T15:56:54.408319+0000","last_clean":"2024-09-10T15:56:54.408319+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:54.408319+0000","last_undegraded":"2024-09-10T15:56:54.408319+0000","last_fullsized":"2024-09-10T15:56:54.408319+0000","mapping_epoch":35,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39882752,"num_objects":9737,"num_object_clones":0,"num_object_copies":19474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7665,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10852","reported_seq":14306,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:54.403633+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:54.403633+0000","last_peered":"2024-09-10T15:56:54.403633+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:54.403633+0000","last_undegraded":"2024-09-10T15:56:54.403633+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41021440,"num_objects":10015,"num_object_clones":0,"num_object_copies":20030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2209,"num_objects_unfound":0,"num_objects_dirty":10015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1659,"num_bytes_recovered":6795264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7806},{"shards":"0,2","objects":2209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10591","reported_seq":10786,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.377951+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:54.377951+0000","last_peered":"2024-09-10T15:56:54.377951+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:54.377951+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39710720,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7241,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5888},{"shards":"0,2","objects":3389}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10916","reported_seq":11047,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.339634+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:54.339634+0000","last_peered":"2024-09-10T15:56:54.339634+0000","last_clean":"2024-09-10T15:56:54.339634+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:54.339634+0000","last_undegraded":"2024-09-10T15:56:54.339634+0000","last_fullsized":"2024-09-10T15:56:54.339634+0000","mapping_epoch":35,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40951808,"num_objects":9998,"num_object_clones":0,"num_object_copies":19996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7675,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10798","reported_seq":18925,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.386412+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:54.386412+0000","last_peered":"2024-09-10T15:56:54.386412+0000","last_clean":"2024-09-10T15:56:54.386412+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:54.386412+0000","last_undegraded":"2024-09-10T15:56:54.386412+0000","last_fullsized":"2024-09-10T15:56:54.386412+0000","mapping_epoch":45,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40636416,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8775,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10732","reported_seq":10953,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.370372+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:54.370372+0000","last_peered":"2024-09-10T15:56:54.370372+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:54.370372+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40329216,"num_objects":9846,"num_object_clones":0,"num_object_copies":19692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7596,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6030},{"shards":"0,2","objects":3424}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10847","reported_seq":11000,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.930274+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:54.930274+0000","last_peered":"2024-09-10T15:56:54.930274+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:54.930274+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40706048,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3820,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8821,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3380},{"shards":"2","objects":6120}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10688","reported_seq":10839,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.922858+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:54.922858+0000","last_peered":"2024-09-10T15:56:54.922858+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:54.922858+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40009728,"num_objects":9768,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8557,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3371},{"shards":"2","objects":5952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10888","reported_seq":11104,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.431538+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:54.431538+0000","last_peered":"2024-09-10T15:56:54.431538+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:54.431538+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40820736,"num_objects":9966,"num_object_clones":0,"num_object_copies":19932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3790,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6177},{"shards":"0,2","objects":3373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10868","reported_seq":11114,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.403795+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:54.403795+0000","last_peered":"2024-09-10T15:56:54.403795+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:54.403795+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40345600,"num_objects":9850,"num_object_clones":0,"num_object_copies":19700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5974},{"shards":"0,2","objects":3412}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10847","reported_seq":11061,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.402453+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:54.402453+0000","last_peered":"2024-09-10T15:56:54.402453+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:54.402453+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40628224,"num_objects":9919,"num_object_clones":0,"num_object_copies":19838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7453,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6079},{"shards":"0,2","objects":3415}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11020","reported_seq":11247,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.444361+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:54.444361+0000","last_peered":"2024-09-10T15:56:54.444361+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:54.444361+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41279971,"num_objects":10079,"num_object_clones":0,"num_object_copies":20158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7839,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6120},{"shards":"0,2","objects":3538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":649155067,"num_objects":158487,"num_object_clones":0,"num_object_copies":316974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38441,"num_objects_misplaced":2209,"num_objects_unfound":0,"num_objects_dirty":158487,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134563,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5995,"num_bytes_recovered":23154688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1340186624,"data_stored":1340095381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160942,"ondisk_log_size":160942,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626478,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608960,"kb_used_data":330224,"kb_used_omap":4,"kb_used_meta":278715,"kb_avail":93762880,"statfs":{"total":96636764160,"available":96013189120,"internally_reserved":0,"allocated":338149376,"data_stored":337806991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":285404846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149761,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":881480,"kb_used_data":538852,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93490360,"statfs":{"total":96636764160,"available":95734128640,"internally_reserved":0,"allocated":551784448,"data_stored":551518395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350809773},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,2,5,4,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704476,"kb_used_data":442380,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93667364,"statfs":{"total":96636764160,"available":95915380736,"internally_reserved":0,"allocated":452997120,"data_stored":452726995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,2,1,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":550719488,"data_stored":550715875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":337629184,"data_stored":337549239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":451837952,"data_stored":451830267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:56.841 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:57.421 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:57.422 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:57.444 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-10T15:56:56.506900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649155086,"num_objects":158489,"num_object_clones":0,"num_object_copies":316978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38441,"num_objects_misplaced":2209,"num_objects_unfound":0,"num_objects_dirty":158489,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134565,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5997,"num_bytes_recovered":23154707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160945,"ondisk_log_size":160945,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2194916,"kb_used_data":1311456,"kb_used_omap":12,"kb_used_meta":883379,"kb_avail":280920604,"statfs":{"total":289910292480,"available":287662698496,"internally_reserved":0,"allocated":1342930944,"data_stored":1342052381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":904580101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,2,7,8,5,0,0,3,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20434944,"num_objects":-4989,"num_object_clones":0,"num_object_copies":-9978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-218,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":-4989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4989,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":467,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004161"},"pg_stats":[{"pgid":"2.f","version":"46'10839","reported_seq":11000,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.431490+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:54.431490+0000","last_peered":"2024-09-10T15:56:54.431490+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:54.431490+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40677376,"num_objects":9931,"num_object_clones":0,"num_object_copies":19862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10839,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6000},{"shards":"0,2","objects":3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10972","reported_seq":11079,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.951778+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:54.951778+0000","last_peered":"2024-09-10T15:56:54.951778+0000","last_clean":"2024-09-10T15:56:54.951778+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:54.951778+0000","last_undegraded":"2024-09-10T15:56:54.951778+0000","last_fullsized":"2024-09-10T15:56:54.951778+0000","mapping_epoch":36,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41377792,"num_objects":10102,"num_object_clones":0,"num_object_copies":20204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10759","reported_seq":10945,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.841200+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:54.841200+0000","last_peered":"2024-09-10T15:56:54.841200+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:54.841200+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427544,"num_objects":9871,"num_object_clones":0,"num_object_copies":19742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3772,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9871,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9027,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3358},{"shards":"2","objects":6100}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10831","reported_seq":10911,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.949966+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:54.949966+0000","last_peered":"2024-09-10T15:56:54.949966+0000","last_clean":"2024-09-10T15:56:54.949966+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:54.949966+0000","last_undegraded":"2024-09-10T15:56:54.949966+0000","last_fullsized":"2024-09-10T15:56:54.949966+0000","mapping_epoch":36,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40349696,"num_objects":9851,"num_object_clones":0,"num_object_copies":19702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10831,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10594","reported_seq":10721,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.408319+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:54.408319+0000","last_peered":"2024-09-10T15:56:54.408319+0000","last_clean":"2024-09-10T15:56:54.408319+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:54.408319+0000","last_undegraded":"2024-09-10T15:56:54.408319+0000","last_fullsized":"2024-09-10T15:56:54.408319+0000","mapping_epoch":35,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39882752,"num_objects":9737,"num_object_clones":0,"num_object_copies":19474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7665,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10852","reported_seq":14306,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:54.403633+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:54.403633+0000","last_peered":"2024-09-10T15:56:54.403633+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:54.403633+0000","last_undegraded":"2024-09-10T15:56:54.403633+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41021440,"num_objects":10015,"num_object_clones":0,"num_object_copies":20030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2209,"num_objects_unfound":0,"num_objects_dirty":10015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1659,"num_bytes_recovered":6795264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7806},{"shards":"0,2","objects":2209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10591","reported_seq":10786,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.377951+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:54.377951+0000","last_peered":"2024-09-10T15:56:54.377951+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:54.377951+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39710720,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7241,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5888},{"shards":"0,2","objects":3389}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10916","reported_seq":11047,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.339634+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:54.339634+0000","last_peered":"2024-09-10T15:56:54.339634+0000","last_clean":"2024-09-10T15:56:54.339634+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:54.339634+0000","last_undegraded":"2024-09-10T15:56:54.339634+0000","last_fullsized":"2024-09-10T15:56:54.339634+0000","mapping_epoch":35,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40951808,"num_objects":9998,"num_object_clones":0,"num_object_copies":19996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7675,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10798","reported_seq":18925,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.386412+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:54.386412+0000","last_peered":"2024-09-10T15:56:54.386412+0000","last_clean":"2024-09-10T15:56:54.386412+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:54.386412+0000","last_undegraded":"2024-09-10T15:56:54.386412+0000","last_fullsized":"2024-09-10T15:56:54.386412+0000","mapping_epoch":45,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40636416,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8775,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10732","reported_seq":10953,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.370372+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:54.370372+0000","last_peered":"2024-09-10T15:56:54.370372+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:54.370372+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40329216,"num_objects":9846,"num_object_clones":0,"num_object_copies":19692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7596,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6030},{"shards":"0,2","objects":3424}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10847","reported_seq":11000,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.930274+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:54.930274+0000","last_peered":"2024-09-10T15:56:54.930274+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:54.930274+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40706048,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3820,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8821,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3380},{"shards":"2","objects":6120}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10688","reported_seq":10839,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.922858+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:54.922858+0000","last_peered":"2024-09-10T15:56:54.922858+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:54.922858+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40009728,"num_objects":9768,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8557,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3371},{"shards":"2","objects":5952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10888","reported_seq":11104,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.431538+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:54.431538+0000","last_peered":"2024-09-10T15:56:54.431538+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:54.431538+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40820736,"num_objects":9966,"num_object_clones":0,"num_object_copies":19932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3790,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6177},{"shards":"0,2","objects":3373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10868","reported_seq":11114,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.403795+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:54.403795+0000","last_peered":"2024-09-10T15:56:54.403795+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:54.403795+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40345600,"num_objects":9850,"num_object_clones":0,"num_object_copies":19700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5974},{"shards":"0,2","objects":3412}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10847","reported_seq":11061,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.402453+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:54.402453+0000","last_peered":"2024-09-10T15:56:54.402453+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:54.402453+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40628224,"num_objects":9919,"num_object_clones":0,"num_object_copies":19838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7453,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6079},{"shards":"0,2","objects":3415}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11020","reported_seq":11247,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.444361+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:54.444361+0000","last_peered":"2024-09-10T15:56:54.444361+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:54.444361+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41279971,"num_objects":10079,"num_object_clones":0,"num_object_copies":20158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7839,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6120},{"shards":"0,2","objects":3538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":649155067,"num_objects":158487,"num_object_clones":0,"num_object_copies":316974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38441,"num_objects_misplaced":2209,"num_objects_unfound":0,"num_objects_dirty":158487,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134563,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5995,"num_bytes_recovered":23154688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1340186624,"data_stored":1340095381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160942,"ondisk_log_size":160942,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626478,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608960,"kb_used_data":330224,"kb_used_omap":4,"kb_used_meta":278715,"kb_avail":93762880,"statfs":{"total":96636764160,"available":96013189120,"internally_reserved":0,"allocated":338149376,"data_stored":337806991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":285404846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149761,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":881480,"kb_used_data":538852,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93490360,"statfs":{"total":96636764160,"available":95734128640,"internally_reserved":0,"allocated":551784448,"data_stored":551518395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350809773},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,2,5,4,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704476,"kb_used_data":442380,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93667364,"statfs":{"total":96636764160,"available":95915380736,"internally_reserved":0,"allocated":452997120,"data_stored":452726995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,2,1,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":550719488,"data_stored":550715875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":337629184,"data_stored":337549239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":451837952,"data_stored":451830267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:56:57.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:56:57.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'10839", 'reported_seq': 11000, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:54.431490+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:56:54.431490+0000', 'last_peered': '2024-09-10T15:56:54.431490+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:56:54.431490+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40677376, 'num_objects': 9931, 'num_object_clones': 0, 'num_object_copies': 19862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3931, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9931, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10839, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6000}, {'shards': '0,2', 'objects': 3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:57.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:56:57.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'10759", 'reported_seq': 10945, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:54.841200+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:56:54.841200+0000', 'last_peered': '2024-09-10T15:56:54.841200+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:56:54.841200+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40427544, 'num_objects': 9871, 'num_object_clones': 0, 'num_object_copies': 19742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3772, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9871, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9027, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3358}, {'shards': '2', 'objects': 6100}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:57.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:56:57.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'10852", 'reported_seq': 14306, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:56:54.403633+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:56:54.403633+0000', 'last_peered': '2024-09-10T15:56:54.403633+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:56:54.403633+0000', 'last_undegraded': '2024-09-10T15:56:54.403633+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41021440, 'num_objects': 10015, 'num_object_clones': 0, 'num_object_copies': 20030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2209, 'num_objects_unfound': 0, 'num_objects_dirty': 10015, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7570, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1659, 'num_bytes_recovered': 6795264, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7806}, {'shards': '0,2', 'objects': 2209}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:57.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:56:57.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'10591", 'reported_seq': 10786, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:54.377951+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:56:54.377951+0000', 'last_peered': '2024-09-10T15:56:54.377951+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:56:54.377951+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39710720, 'num_objects': 9695, 'num_object_clones': 0, 'num_object_copies': 19390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3809, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7241, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5888}, {'shards': '0,2', 'objects': 3389}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:57.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:56:57.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'10732", 'reported_seq': 10953, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:54.370372+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:56:54.370372+0000', 'last_peered': '2024-09-10T15:56:54.370372+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:56:54.370372+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40329216, 'num_objects': 9846, 'num_object_clones': 0, 'num_object_copies': 19692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3818, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9846, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7596, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 47, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6030}, {'shards': '0,2', 'objects': 3424}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:57.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:56:57.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'10847", 'reported_seq': 11000, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:54.930274+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:56:54.930274+0000', 'last_peered': '2024-09-10T15:56:54.930274+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:56:54.930274+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40706048, 'num_objects': 9938, 'num_object_clones': 0, 'num_object_copies': 19876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3820, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9938, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8821, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 14, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3380}, {'shards': '2', 'objects': 6120}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:57.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:56:57.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'10688", 'reported_seq': 10839, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:54.922858+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:56:54.922858+0000', 'last_peered': '2024-09-10T15:56:54.922858+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:56:54.922858+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'600", 'ondisk_log_start': "37'600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40009728, 'num_objects': 9768, 'num_object_clones': 0, 'num_object_copies': 19536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3818, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8557, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 13, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3371}, {'shards': '2', 'objects': 5952}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:56:57.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:56:57.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'10888", 'reported_seq': 11104, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:54.431538+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:56:54.431538+0000', 'last_peered': '2024-09-10T15:56:54.431538+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:56:54.431538+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40820736, 'num_objects': 9966, 'num_object_clones': 0, 'num_object_copies': 19932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3790, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9966, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8080, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6177}, {'shards': '0,2', 'objects': 3373}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:57.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:56:57.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'10868", 'reported_seq': 11114, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:54.403795+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:56:54.403795+0000', 'last_peered': '2024-09-10T15:56:54.403795+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:56:54.403795+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40345600, 'num_objects': 9850, 'num_object_clones': 0, 'num_object_copies': 19700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3879, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9850, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7286, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5974}, {'shards': '0,2', 'objects': 3412}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:57.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:56:57.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'10847", 'reported_seq': 11061, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:54.402453+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:56:54.402453+0000', 'last_peered': '2024-09-10T15:56:54.402453+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:56:54.402453+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40628224, 'num_objects': 9919, 'num_object_clones': 0, 'num_object_copies': 19838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3842, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9919, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7453, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 38, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6079}, {'shards': '0,2', 'objects': 3415}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:57.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:56:57.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'11020", 'reported_seq': 11247, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:54.444361+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:56:54.444361+0000', 'last_peered': '2024-09-10T15:56:54.444361+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:56:54.444361+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'1000", 'ondisk_log_start': "37'1000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41279971, 'num_objects': 10079, 'num_object_clones': 0, 'num_object_copies': 20158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3962, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10079, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7839, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6120}, {'shards': '0,2', 'objects': 3538}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:56:57.448 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:56:58.034 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:58.034 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38441/316978 objects degraded (12.127%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":343,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158489,"data_bytes":649155086,"bytes_used":2247593984,"bytes_avail":287662698496,"bytes_total":289910292480,"degraded_objects":38441,"degraded_total":316978,"degraded_ratio":0.12127340067764955,"misplaced_objects":2209,"misplaced_total":316978,"misplaced_ratio":0.0069689379073626552,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":84962,"recovering_keys_per_sec":0,"num_objects_recovered":467,"num_bytes_recovered":1019904,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":415},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T15:56:24.496337+0000","services":{}},"progress_events":{}} 2024-09-10T15:56:58.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:56:58.056 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:56:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:56:58.583 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:56:58.600 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-10T15:56:58.507740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649155086,"num_objects":158489,"num_object_clones":0,"num_object_copies":316978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38441,"num_objects_misplaced":2209,"num_objects_unfound":0,"num_objects_dirty":158489,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134565,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5997,"num_bytes_recovered":23154707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160945,"ondisk_log_size":160945,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2194916,"kb_used_data":1311456,"kb_used_omap":12,"kb_used_meta":883379,"kb_avail":280920604,"statfs":{"total":289910292480,"available":287662698496,"internally_reserved":0,"allocated":1342930944,"data_stored":1342052381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":904580101},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,2,7,8,5,0,0,3,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17711104,"num_objects":-4324,"num_object_clones":0,"num_object_copies":-8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-204,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":-4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4324,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004338"},"pg_stats":[{"pgid":"2.f","version":"46'10839","reported_seq":11000,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.431490+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:54.431490+0000","last_peered":"2024-09-10T15:56:54.431490+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:54.431490+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40677376,"num_objects":9931,"num_object_clones":0,"num_object_copies":19862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10839,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6000},{"shards":"0,2","objects":3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'10972","reported_seq":11079,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.951778+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:54.951778+0000","last_peered":"2024-09-10T15:56:54.951778+0000","last_clean":"2024-09-10T15:56:54.951778+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:54.951778+0000","last_undegraded":"2024-09-10T15:56:54.951778+0000","last_fullsized":"2024-09-10T15:56:54.951778+0000","mapping_epoch":36,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41377792,"num_objects":10102,"num_object_clones":0,"num_object_copies":20204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10759","reported_seq":10945,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.841200+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:54.841200+0000","last_peered":"2024-09-10T15:56:54.841200+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:54.841200+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427544,"num_objects":9871,"num_object_clones":0,"num_object_copies":19742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3772,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9871,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9027,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3358},{"shards":"2","objects":6100}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10831","reported_seq":10911,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.949966+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:54.949966+0000","last_peered":"2024-09-10T15:56:54.949966+0000","last_clean":"2024-09-10T15:56:54.949966+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:54.949966+0000","last_undegraded":"2024-09-10T15:56:54.949966+0000","last_fullsized":"2024-09-10T15:56:54.949966+0000","mapping_epoch":36,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40349696,"num_objects":9851,"num_object_clones":0,"num_object_copies":19702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10831,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10594","reported_seq":10721,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.408319+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:54.408319+0000","last_peered":"2024-09-10T15:56:54.408319+0000","last_clean":"2024-09-10T15:56:54.408319+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:54.408319+0000","last_undegraded":"2024-09-10T15:56:54.408319+0000","last_fullsized":"2024-09-10T15:56:54.408319+0000","mapping_epoch":35,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39882752,"num_objects":9737,"num_object_clones":0,"num_object_copies":19474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7665,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10852","reported_seq":14306,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:54.403633+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:54.403633+0000","last_peered":"2024-09-10T15:56:54.403633+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:54.403633+0000","last_undegraded":"2024-09-10T15:56:54.403633+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41021440,"num_objects":10015,"num_object_clones":0,"num_object_copies":20030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2209,"num_objects_unfound":0,"num_objects_dirty":10015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1659,"num_bytes_recovered":6795264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7806},{"shards":"0,2","objects":2209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10591","reported_seq":10786,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.377951+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:54.377951+0000","last_peered":"2024-09-10T15:56:54.377951+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:54.377951+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39710720,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7241,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5888},{"shards":"0,2","objects":3389}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'10916","reported_seq":11047,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.339634+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:54.339634+0000","last_peered":"2024-09-10T15:56:54.339634+0000","last_clean":"2024-09-10T15:56:54.339634+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:54.339634+0000","last_undegraded":"2024-09-10T15:56:54.339634+0000","last_fullsized":"2024-09-10T15:56:54.339634+0000","mapping_epoch":35,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40951808,"num_objects":9998,"num_object_clones":0,"num_object_copies":19996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7675,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10798","reported_seq":18925,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:54.386412+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:54.386412+0000","last_peered":"2024-09-10T15:56:54.386412+0000","last_clean":"2024-09-10T15:56:54.386412+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:54.386412+0000","last_undegraded":"2024-09-10T15:56:54.386412+0000","last_fullsized":"2024-09-10T15:56:54.386412+0000","mapping_epoch":45,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40636416,"num_objects":9921,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8775,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10732","reported_seq":10953,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.370372+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:54.370372+0000","last_peered":"2024-09-10T15:56:54.370372+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:54.370372+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40329216,"num_objects":9846,"num_object_clones":0,"num_object_copies":19692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7596,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6030},{"shards":"0,2","objects":3424}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10847","reported_seq":11000,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.930274+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:54.930274+0000","last_peered":"2024-09-10T15:56:54.930274+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:54.930274+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40706048,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3820,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8821,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3380},{"shards":"2","objects":6120}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10688","reported_seq":10839,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.922858+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:54.922858+0000","last_peered":"2024-09-10T15:56:54.922858+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:54.922858+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40009728,"num_objects":9768,"num_object_clones":0,"num_object_copies":19536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8557,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3371},{"shards":"2","objects":5952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'10888","reported_seq":11104,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.431538+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:54.431538+0000","last_peered":"2024-09-10T15:56:54.431538+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:54.431538+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40820736,"num_objects":9966,"num_object_clones":0,"num_object_copies":19932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3790,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6177},{"shards":"0,2","objects":3373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'10868","reported_seq":11114,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.403795+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:54.403795+0000","last_peered":"2024-09-10T15:56:54.403795+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:54.403795+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40345600,"num_objects":9850,"num_object_clones":0,"num_object_copies":19700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3879,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5974},{"shards":"0,2","objects":3412}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10847","reported_seq":11061,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.402453+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:54.402453+0000","last_peered":"2024-09-10T15:56:54.402453+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:54.402453+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40628224,"num_objects":9919,"num_object_clones":0,"num_object_copies":19838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7453,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6079},{"shards":"0,2","objects":3415}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11020","reported_seq":11247,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:54.444361+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:54.444361+0000","last_peered":"2024-09-10T15:56:54.444361+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:54.444361+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41279971,"num_objects":10079,"num_object_clones":0,"num_object_copies":20158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7839,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6120},{"shards":"0,2","objects":3538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":649155067,"num_objects":158487,"num_object_clones":0,"num_object_copies":316974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38441,"num_objects_misplaced":2209,"num_objects_unfound":0,"num_objects_dirty":158487,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134563,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5995,"num_bytes_recovered":23154688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1340186624,"data_stored":1340095381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160942,"ondisk_log_size":160942,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626478,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608960,"kb_used_data":330224,"kb_used_omap":4,"kb_used_meta":278715,"kb_avail":93762880,"statfs":{"total":96636764160,"available":96013189120,"internally_reserved":0,"allocated":338149376,"data_stored":337806991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":285404846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149761,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":881480,"kb_used_data":538852,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93490360,"statfs":{"total":96636764160,"available":95734128640,"internally_reserved":0,"allocated":551784448,"data_stored":551518395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350809773},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,2,5,4,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704476,"kb_used_data":442380,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93667364,"statfs":{"total":96636764160,"available":95915380736,"internally_reserved":0,"allocated":452997120,"data_stored":452726995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,2,1,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":550719488,"data_stored":550715875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":337629184,"data_stored":337549239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":451837952,"data_stored":451830267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:01.602 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:02.129 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:02.132 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:02.158 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-10T15:57:00.508364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640320014,"num_objects":156332,"num_object_clones":0,"num_object_copies":312664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38351,"num_objects_misplaced":2081,"num_objects_unfound":0,"num_objects_dirty":156332,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136722,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6217,"num_bytes_recovered":23678995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160902,"ondisk_log_size":160902,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2197636,"kb_used_data":1300296,"kb_used_omap":12,"kb_used_meta":897267,"kb_avail":280917884,"statfs":{"total":289910292480,"available":287659913216,"internally_reserved":0,"allocated":1331503104,"data_stored":1330631767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":918801413},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,8,4,10,2,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26546176,"num_objects":-6481,"num_object_clones":0,"num_object_copies":-12962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-294,"num_objects_misplaced":-377,"num_objects_unfound":0,"num_objects_dirty":-6481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6481,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":1544192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004140"},"pg_stats":[{"pgid":"2.f","version":"46'10979","reported_seq":11156,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.459326+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:59.459326+0000","last_peered":"2024-09-10T15:56:59.459326+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:59.459326+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40103936,"num_objects":9791,"num_object_clones":0,"num_object_copies":19582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10979,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5868},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11128","reported_seq":11235,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.942348+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:59.942348+0000","last_peered":"2024-09-10T15:56:59.942348+0000","last_clean":"2024-09-10T15:56:59.942348+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:59.942348+0000","last_undegraded":"2024-09-10T15:56:59.942348+0000","last_fullsized":"2024-09-10T15:56:59.942348+0000","mapping_epoch":36,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40738816,"num_objects":9946,"num_object_clones":0,"num_object_copies":19892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9464,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10898","reported_seq":11088,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.785131+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:59.785131+0000","last_peered":"2024-09-10T15:56:59.785131+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:59.785131+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39858200,"num_objects":9732,"num_object_clones":0,"num_object_copies":19464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9732,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9166,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3219},{"shards":"2","objects":5963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10956","reported_seq":11036,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.943057+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:59.943057+0000","last_peered":"2024-09-10T15:56:59.943057+0000","last_clean":"2024-09-10T15:56:59.943057+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:59.943057+0000","last_undegraded":"2024-09-10T15:56:59.943057+0000","last_fullsized":"2024-09-10T15:56:59.943057+0000","mapping_epoch":36,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39837696,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10956,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10699","reported_seq":10826,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.316962+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:59.316962+0000","last_peered":"2024-09-10T15:56:59.316962+0000","last_clean":"2024-09-10T15:56:59.316962+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:59.316962+0000","last_undegraded":"2024-09-10T15:56:59.316962+0000","last_fullsized":"2024-09-10T15:56:59.316962+0000","mapping_epoch":35,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39452672,"num_objects":9632,"num_object_clones":0,"num_object_copies":19264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7770,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10986","reported_seq":14700,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:59.449829+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:59.449829+0000","last_peered":"2024-09-10T15:56:59.449829+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:59.449829+0000","last_undegraded":"2024-09-10T15:56:59.449829+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40472576,"num_objects":9881,"num_object_clones":0,"num_object_copies":19762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2081,"num_objects_unfound":0,"num_objects_dirty":9881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7704,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1789,"num_bytes_recovered":7319552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7800},{"shards":"0,2","objects":2081}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10706","reported_seq":10920,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.392489+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:59.392489+0000","last_peered":"2024-09-10T15:56:59.392489+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:59.392489+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39239680,"num_objects":9580,"num_object_clones":0,"num_object_copies":19160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3799,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7356,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5783},{"shards":"0,2","objects":3274}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11055","reported_seq":11186,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.362449+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:59.362449+0000","last_peered":"2024-09-10T15:56:59.362449+0000","last_clean":"2024-09-10T15:56:59.362449+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:59.362449+0000","last_undegraded":"2024-09-10T15:56:59.362449+0000","last_fullsized":"2024-09-10T15:56:59.362449+0000","mapping_epoch":35,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40382464,"num_objects":9859,"num_object_clones":0,"num_object_copies":19718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7814,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10962","reported_seq":19089,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.437796+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:59.437796+0000","last_peered":"2024-09-10T15:56:59.437796+0000","last_clean":"2024-09-10T15:56:59.437796+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:59.437796+0000","last_undegraded":"2024-09-10T15:56:59.437796+0000","last_fullsized":"2024-09-10T15:56:59.437796+0000","mapping_epoch":45,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39964672,"num_objects":9757,"num_object_clones":0,"num_object_copies":19514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8939,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10881","reported_seq":11122,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.418036+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:59.418036+0000","last_peered":"2024-09-10T15:56:59.418036+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:59.418036+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39718912,"num_objects":9697,"num_object_clones":0,"num_object_copies":19394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7745,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5891},{"shards":"0,2","objects":3275}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10966","reported_seq":11122,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.934312+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:59.934312+0000","last_peered":"2024-09-10T15:56:59.934312+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:59.934312+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40218624,"num_objects":9819,"num_object_clones":0,"num_object_copies":19638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8940,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3261},{"shards":"2","objects":6003}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10819","reported_seq":10976,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.919174+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:59.919174+0000","last_peered":"2024-09-10T15:56:59.919174+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:59.919174+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39473152,"num_objects":9637,"num_object_clones":0,"num_object_copies":19274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8688,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3240},{"shards":"2","objects":5823}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11030","reported_seq":11287,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.418722+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:59.418722+0000","last_peered":"2024-09-10T15:56:59.418722+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:59.418722+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40239104,"num_objects":9824,"num_object_clones":0,"num_object_copies":19648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3769,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8222,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6056},{"shards":"0,2","objects":3231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11013","reported_seq":11291,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.440354+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:59.440354+0000","last_peered":"2024-09-10T15:56:59.440354+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:59.440354+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39751680,"num_objects":9705,"num_object_clones":0,"num_object_copies":19410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3863,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7431,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5845},{"shards":"0,2","objects":3267}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10970","reported_seq":11190,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.472181+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:59.472181+0000","last_peered":"2024-09-10T15:56:59.472181+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:59.472181+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40124416,"num_objects":9796,"num_object_clones":0,"num_object_copies":19592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3838,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7576,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5960},{"shards":"0,2","objects":3292}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11151","reported_seq":11408,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.343891+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:59.343891+0000","last_peered":"2024-09-10T15:56:59.343891+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:59.343891+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40743395,"num_objects":9948,"num_object_clones":0,"num_object_copies":19896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3947,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7970,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6004},{"shards":"0,2","objects":3407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":640319995,"num_objects":156330,"num_object_clones":0,"num_object_copies":312660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38351,"num_objects_misplaced":2081,"num_objects_unfound":0,"num_objects_dirty":156330,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136720,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6215,"num_bytes_recovered":23678976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1328472064,"data_stored":1328388047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626479,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595864,"kb_used_data":329604,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93775976,"statfs":{"total":96636764160,"available":96026599424,"internally_reserved":0,"allocated":337514496,"data_stored":337179337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272625326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149762,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":879324,"kb_used_data":532548,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93492516,"statfs":{"total":96636764160,"available":95736336384,"internally_reserved":0,"allocated":545329152,"data_stored":545063099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355069613},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,6,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722448,"kb_used_data":438144,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93649392,"statfs":{"total":96636764160,"available":95896977408,"internally_reserved":0,"allocated":448659456,"data_stored":448389331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":544186368,"data_stored":544182755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":336920576,"data_stored":336847857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":447365120,"data_stored":447357435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:02.159 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:02.709 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:02.710 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:02.734 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-10T15:57:02.509260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640320014,"num_objects":156332,"num_object_clones":0,"num_object_copies":312664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38351,"num_objects_misplaced":2081,"num_objects_unfound":0,"num_objects_dirty":156332,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136722,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6217,"num_bytes_recovered":23678995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160902,"ondisk_log_size":160902,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2197636,"kb_used_data":1300296,"kb_used_omap":12,"kb_used_meta":897267,"kb_avail":280917884,"statfs":{"total":289910292480,"available":287659913216,"internally_reserved":0,"allocated":1331503104,"data_stored":1330631767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":918801413},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,8,4,10,2,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17780736,"num_objects":-4341,"num_object_clones":0,"num_object_copies":-8682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-173,"num_objects_misplaced":-243,"num_objects_unfound":0,"num_objects_dirty":-4341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4341,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004509"},"pg_stats":[{"pgid":"2.f","version":"46'10979","reported_seq":11156,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.459326+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:59.459326+0000","last_peered":"2024-09-10T15:56:59.459326+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:59.459326+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40103936,"num_objects":9791,"num_object_clones":0,"num_object_copies":19582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10979,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5868},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11128","reported_seq":11235,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.942348+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:59.942348+0000","last_peered":"2024-09-10T15:56:59.942348+0000","last_clean":"2024-09-10T15:56:59.942348+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:59.942348+0000","last_undegraded":"2024-09-10T15:56:59.942348+0000","last_fullsized":"2024-09-10T15:56:59.942348+0000","mapping_epoch":36,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40738816,"num_objects":9946,"num_object_clones":0,"num_object_copies":19892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9464,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10898","reported_seq":11088,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.785131+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:59.785131+0000","last_peered":"2024-09-10T15:56:59.785131+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:59.785131+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39858200,"num_objects":9732,"num_object_clones":0,"num_object_copies":19464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9732,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9166,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3219},{"shards":"2","objects":5963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10956","reported_seq":11036,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.943057+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:59.943057+0000","last_peered":"2024-09-10T15:56:59.943057+0000","last_clean":"2024-09-10T15:56:59.943057+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:59.943057+0000","last_undegraded":"2024-09-10T15:56:59.943057+0000","last_fullsized":"2024-09-10T15:56:59.943057+0000","mapping_epoch":36,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39837696,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10956,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10699","reported_seq":10826,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.316962+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:59.316962+0000","last_peered":"2024-09-10T15:56:59.316962+0000","last_clean":"2024-09-10T15:56:59.316962+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:59.316962+0000","last_undegraded":"2024-09-10T15:56:59.316962+0000","last_fullsized":"2024-09-10T15:56:59.316962+0000","mapping_epoch":35,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39452672,"num_objects":9632,"num_object_clones":0,"num_object_copies":19264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7770,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10986","reported_seq":14700,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:59.449829+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:59.449829+0000","last_peered":"2024-09-10T15:56:59.449829+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:59.449829+0000","last_undegraded":"2024-09-10T15:56:59.449829+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40472576,"num_objects":9881,"num_object_clones":0,"num_object_copies":19762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2081,"num_objects_unfound":0,"num_objects_dirty":9881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7704,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1789,"num_bytes_recovered":7319552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7800},{"shards":"0,2","objects":2081}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10706","reported_seq":10920,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.392489+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:59.392489+0000","last_peered":"2024-09-10T15:56:59.392489+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:59.392489+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39239680,"num_objects":9580,"num_object_clones":0,"num_object_copies":19160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3799,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7356,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5783},{"shards":"0,2","objects":3274}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11055","reported_seq":11186,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.362449+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:59.362449+0000","last_peered":"2024-09-10T15:56:59.362449+0000","last_clean":"2024-09-10T15:56:59.362449+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:59.362449+0000","last_undegraded":"2024-09-10T15:56:59.362449+0000","last_fullsized":"2024-09-10T15:56:59.362449+0000","mapping_epoch":35,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40382464,"num_objects":9859,"num_object_clones":0,"num_object_copies":19718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7814,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10962","reported_seq":19089,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.437796+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:59.437796+0000","last_peered":"2024-09-10T15:56:59.437796+0000","last_clean":"2024-09-10T15:56:59.437796+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:59.437796+0000","last_undegraded":"2024-09-10T15:56:59.437796+0000","last_fullsized":"2024-09-10T15:56:59.437796+0000","mapping_epoch":45,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39964672,"num_objects":9757,"num_object_clones":0,"num_object_copies":19514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8939,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10881","reported_seq":11122,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.418036+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:59.418036+0000","last_peered":"2024-09-10T15:56:59.418036+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:59.418036+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39718912,"num_objects":9697,"num_object_clones":0,"num_object_copies":19394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7745,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5891},{"shards":"0,2","objects":3275}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10966","reported_seq":11122,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.934312+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:59.934312+0000","last_peered":"2024-09-10T15:56:59.934312+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:59.934312+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40218624,"num_objects":9819,"num_object_clones":0,"num_object_copies":19638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8940,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3261},{"shards":"2","objects":6003}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10819","reported_seq":10976,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.919174+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:59.919174+0000","last_peered":"2024-09-10T15:56:59.919174+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:59.919174+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39473152,"num_objects":9637,"num_object_clones":0,"num_object_copies":19274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8688,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3240},{"shards":"2","objects":5823}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11030","reported_seq":11287,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.418722+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:59.418722+0000","last_peered":"2024-09-10T15:56:59.418722+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:59.418722+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40239104,"num_objects":9824,"num_object_clones":0,"num_object_copies":19648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3769,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8222,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6056},{"shards":"0,2","objects":3231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11013","reported_seq":11291,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.440354+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:59.440354+0000","last_peered":"2024-09-10T15:56:59.440354+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:59.440354+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39751680,"num_objects":9705,"num_object_clones":0,"num_object_copies":19410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3863,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7431,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5845},{"shards":"0,2","objects":3267}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10970","reported_seq":11190,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.472181+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:59.472181+0000","last_peered":"2024-09-10T15:56:59.472181+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:59.472181+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40124416,"num_objects":9796,"num_object_clones":0,"num_object_copies":19592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3838,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7576,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5960},{"shards":"0,2","objects":3292}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11151","reported_seq":11408,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.343891+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:59.343891+0000","last_peered":"2024-09-10T15:56:59.343891+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:59.343891+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40743395,"num_objects":9948,"num_object_clones":0,"num_object_copies":19896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3947,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7970,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6004},{"shards":"0,2","objects":3407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":640319995,"num_objects":156330,"num_object_clones":0,"num_object_copies":312660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38351,"num_objects_misplaced":2081,"num_objects_unfound":0,"num_objects_dirty":156330,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136720,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6215,"num_bytes_recovered":23678976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1328472064,"data_stored":1328388047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626479,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595864,"kb_used_data":329604,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93775976,"statfs":{"total":96636764160,"available":96026599424,"internally_reserved":0,"allocated":337514496,"data_stored":337179337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272625326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149762,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":879324,"kb_used_data":532548,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93492516,"statfs":{"total":96636764160,"available":95736336384,"internally_reserved":0,"allocated":545329152,"data_stored":545063099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355069613},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,6,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722448,"kb_used_data":438144,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93649392,"statfs":{"total":96636764160,"available":95896977408,"internally_reserved":0,"allocated":448659456,"data_stored":448389331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":544186368,"data_stored":544182755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":336920576,"data_stored":336847857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":447365120,"data_stored":447357435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:02.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:57:02.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'10979", 'reported_seq': 11156, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:59.459326+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:56:59.459326+0000', 'last_peered': '2024-09-10T15:56:59.459326+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:56:59.459326+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40103936, 'num_objects': 9791, 'num_object_clones': 0, 'num_object_copies': 19582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3923, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9791, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10979, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 50, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5868}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:02.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:57:02.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'10898", 'reported_seq': 11088, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:59.785131+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:56:59.785131+0000', 'last_peered': '2024-09-10T15:56:59.785131+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:56:59.785131+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39858200, 'num_objects': 9732, 'num_object_clones': 0, 'num_object_copies': 19464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3770, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9732, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9166, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 33, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3219}, {'shards': '2', 'objects': 5963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:02.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:57:02.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'10986", 'reported_seq': 14700, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:56:59.449829+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:56:59.449829+0000', 'last_peered': '2024-09-10T15:56:59.449829+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:56:59.449829+0000', 'last_undegraded': '2024-09-10T15:56:59.449829+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40472576, 'num_objects': 9881, 'num_object_clones': 0, 'num_object_copies': 19762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2081, 'num_objects_unfound': 0, 'num_objects_dirty': 9881, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7704, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1789, 'num_bytes_recovered': 7319552, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7800}, {'shards': '0,2', 'objects': 2081}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:02.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:57:02.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'10706", 'reported_seq': 10920, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:59.392489+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:56:59.392489+0000', 'last_peered': '2024-09-10T15:56:59.392489+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:56:59.392489+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39239680, 'num_objects': 9580, 'num_object_clones': 0, 'num_object_copies': 19160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3799, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9580, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7356, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 38, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5783}, {'shards': '0,2', 'objects': 3274}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:02.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:57:02.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'10881", 'reported_seq': 11122, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:59.418036+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:56:59.418036+0000', 'last_peered': '2024-09-10T15:56:59.418036+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:56:59.418036+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39718912, 'num_objects': 9697, 'num_object_clones': 0, 'num_object_copies': 19394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3808, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7745, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 57, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5891}, {'shards': '0,2', 'objects': 3275}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:02.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:57:02.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'10966", 'reported_seq': 11122, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:59.934312+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:56:59.934312+0000', 'last_peered': '2024-09-10T15:56:59.934312+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:56:59.934312+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40218624, 'num_objects': 9819, 'num_object_clones': 0, 'num_object_copies': 19638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3818, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9819, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8940, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3261}, {'shards': '2', 'objects': 6003}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:02.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:57:02.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'10819", 'reported_seq': 10976, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:59.919174+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:56:59.919174+0000', 'last_peered': '2024-09-10T15:56:59.919174+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:56:59.919174+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39473152, 'num_objects': 9637, 'num_object_clones': 0, 'num_object_copies': 19274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3816, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9637, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8688, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3240}, {'shards': '2', 'objects': 5823}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:02.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:57:02.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'11030", 'reported_seq': 11287, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:59.418722+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:56:59.418722+0000', 'last_peered': '2024-09-10T15:56:59.418722+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:56:59.418722+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'1000", 'ondisk_log_start': "37'1000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40239104, 'num_objects': 9824, 'num_object_clones': 0, 'num_object_copies': 19648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3769, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9824, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8222, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 65, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6056}, {'shards': '0,2', 'objects': 3231}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:02.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:57:02.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'11013", 'reported_seq': 11291, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:59.440354+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:56:59.440354+0000', 'last_peered': '2024-09-10T15:56:59.440354+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:56:59.440354+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'1000", 'ondisk_log_start': "37'1000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39751680, 'num_objects': 9705, 'num_object_clones': 0, 'num_object_copies': 19410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3863, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9705, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7431, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 57, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5845}, {'shards': '0,2', 'objects': 3267}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:02.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:57:02.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'10970", 'reported_seq': 11190, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:59.472181+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:56:59.472181+0000', 'last_peered': '2024-09-10T15:56:59.472181+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:56:59.472181+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40124416, 'num_objects': 9796, 'num_object_clones': 0, 'num_object_copies': 19592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3838, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9796, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7576, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5960}, {'shards': '0,2', 'objects': 3292}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:02.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:57:02.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'11151", 'reported_seq': 11408, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:56:59.343891+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:56:59.343891+0000', 'last_peered': '2024-09-10T15:56:59.343891+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:56:59.343891+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40743395, 'num_objects': 9948, 'num_object_clones': 0, 'num_object_copies': 19896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3947, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9948, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7970, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6004}, {'shards': '0,2', 'objects': 3407}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:02.738 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:57:03.437 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:03.437 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38351/312664 objects degraded (12.266%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":349,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156332,"data_bytes":640320014,"bytes_used":2250379264,"bytes_avail":287659913216,"bytes_total":289910292480,"degraded_objects":38351,"degraded_total":312664,"degraded_ratio":0.1226588286467262,"misplaced_objects":2081,"misplaced_total":312664,"misplaced_ratio":0.0066557070849218329,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":82912,"recovering_keys_per_sec":0,"num_objects_recovered":418,"num_bytes_recovered":995328,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":361},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T15:56:24.496337+0000","services":{}},"progress_events":{}} 2024-09-10T15:57:03.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:57:03.458 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:03.881 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:03.884 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:03.906 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-10T15:57:02.509260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640320014,"num_objects":156332,"num_object_clones":0,"num_object_copies":312664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38351,"num_objects_misplaced":2081,"num_objects_unfound":0,"num_objects_dirty":156332,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136722,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6217,"num_bytes_recovered":23678995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160902,"ondisk_log_size":160902,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2197636,"kb_used_data":1300296,"kb_used_omap":12,"kb_used_meta":897267,"kb_avail":280917884,"statfs":{"total":289910292480,"available":287659913216,"internally_reserved":0,"allocated":1331503104,"data_stored":1330631767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":918801413},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,8,4,10,2,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17780736,"num_objects":-4341,"num_object_clones":0,"num_object_copies":-8682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-173,"num_objects_misplaced":-243,"num_objects_unfound":0,"num_objects_dirty":-4341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4341,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004509"},"pg_stats":[{"pgid":"2.f","version":"46'10979","reported_seq":11156,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.459326+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:56:59.459326+0000","last_peered":"2024-09-10T15:56:59.459326+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:56:59.459326+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40103936,"num_objects":9791,"num_object_clones":0,"num_object_copies":19582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10979,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5868},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11128","reported_seq":11235,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.942348+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:56:59.942348+0000","last_peered":"2024-09-10T15:56:59.942348+0000","last_clean":"2024-09-10T15:56:59.942348+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:56:59.942348+0000","last_undegraded":"2024-09-10T15:56:59.942348+0000","last_fullsized":"2024-09-10T15:56:59.942348+0000","mapping_epoch":36,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40738816,"num_objects":9946,"num_object_clones":0,"num_object_copies":19892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9464,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'10898","reported_seq":11088,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.785131+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:56:59.785131+0000","last_peered":"2024-09-10T15:56:59.785131+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:56:59.785131+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39858200,"num_objects":9732,"num_object_clones":0,"num_object_copies":19464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9732,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9166,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3219},{"shards":"2","objects":5963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'10956","reported_seq":11036,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.943057+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:56:59.943057+0000","last_peered":"2024-09-10T15:56:59.943057+0000","last_clean":"2024-09-10T15:56:59.943057+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:56:59.943057+0000","last_undegraded":"2024-09-10T15:56:59.943057+0000","last_fullsized":"2024-09-10T15:56:59.943057+0000","mapping_epoch":36,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39837696,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10956,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10699","reported_seq":10826,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.316962+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:56:59.316962+0000","last_peered":"2024-09-10T15:56:59.316962+0000","last_clean":"2024-09-10T15:56:59.316962+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:56:59.316962+0000","last_undegraded":"2024-09-10T15:56:59.316962+0000","last_fullsized":"2024-09-10T15:56:59.316962+0000","mapping_epoch":35,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39452672,"num_objects":9632,"num_object_clones":0,"num_object_copies":19264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7770,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'10986","reported_seq":14700,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:56:59.449829+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:56:59.449829+0000","last_peered":"2024-09-10T15:56:59.449829+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:56:59.449829+0000","last_undegraded":"2024-09-10T15:56:59.449829+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40472576,"num_objects":9881,"num_object_clones":0,"num_object_copies":19762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2081,"num_objects_unfound":0,"num_objects_dirty":9881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7704,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1789,"num_bytes_recovered":7319552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7800},{"shards":"0,2","objects":2081}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10706","reported_seq":10920,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.392489+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:56:59.392489+0000","last_peered":"2024-09-10T15:56:59.392489+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:56:59.392489+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39239680,"num_objects":9580,"num_object_clones":0,"num_object_copies":19160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3799,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7356,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5783},{"shards":"0,2","objects":3274}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11055","reported_seq":11186,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.362449+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:56:59.362449+0000","last_peered":"2024-09-10T15:56:59.362449+0000","last_clean":"2024-09-10T15:56:59.362449+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:56:59.362449+0000","last_undegraded":"2024-09-10T15:56:59.362449+0000","last_fullsized":"2024-09-10T15:56:59.362449+0000","mapping_epoch":35,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40382464,"num_objects":9859,"num_object_clones":0,"num_object_copies":19718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7814,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'10962","reported_seq":19089,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:56:59.437796+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:56:59.437796+0000","last_peered":"2024-09-10T15:56:59.437796+0000","last_clean":"2024-09-10T15:56:59.437796+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:56:59.437796+0000","last_undegraded":"2024-09-10T15:56:59.437796+0000","last_fullsized":"2024-09-10T15:56:59.437796+0000","mapping_epoch":45,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39964672,"num_objects":9757,"num_object_clones":0,"num_object_copies":19514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8939,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'10881","reported_seq":11122,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.418036+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:56:59.418036+0000","last_peered":"2024-09-10T15:56:59.418036+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:56:59.418036+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39718912,"num_objects":9697,"num_object_clones":0,"num_object_copies":19394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7745,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5891},{"shards":"0,2","objects":3275}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'10966","reported_seq":11122,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.934312+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:56:59.934312+0000","last_peered":"2024-09-10T15:56:59.934312+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:56:59.934312+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40218624,"num_objects":9819,"num_object_clones":0,"num_object_copies":19638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8940,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3261},{"shards":"2","objects":6003}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10819","reported_seq":10976,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.919174+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:56:59.919174+0000","last_peered":"2024-09-10T15:56:59.919174+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:56:59.919174+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39473152,"num_objects":9637,"num_object_clones":0,"num_object_copies":19274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8688,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3240},{"shards":"2","objects":5823}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11030","reported_seq":11287,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.418722+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:56:59.418722+0000","last_peered":"2024-09-10T15:56:59.418722+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:56:59.418722+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40239104,"num_objects":9824,"num_object_clones":0,"num_object_copies":19648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3769,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8222,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6056},{"shards":"0,2","objects":3231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11013","reported_seq":11291,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.440354+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:56:59.440354+0000","last_peered":"2024-09-10T15:56:59.440354+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:56:59.440354+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39751680,"num_objects":9705,"num_object_clones":0,"num_object_copies":19410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3863,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7431,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5845},{"shards":"0,2","objects":3267}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'10970","reported_seq":11190,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.472181+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:56:59.472181+0000","last_peered":"2024-09-10T15:56:59.472181+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:56:59.472181+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40124416,"num_objects":9796,"num_object_clones":0,"num_object_copies":19592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3838,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7576,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5960},{"shards":"0,2","objects":3292}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11151","reported_seq":11408,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:56:59.343891+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:56:59.343891+0000","last_peered":"2024-09-10T15:56:59.343891+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:56:59.343891+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40743395,"num_objects":9948,"num_object_clones":0,"num_object_copies":19896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3947,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7970,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6004},{"shards":"0,2","objects":3407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":640319995,"num_objects":156330,"num_object_clones":0,"num_object_copies":312660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38351,"num_objects_misplaced":2081,"num_objects_unfound":0,"num_objects_dirty":156330,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136720,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6215,"num_bytes_recovered":23678976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1328472064,"data_stored":1328388047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626479,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595864,"kb_used_data":329604,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93775976,"statfs":{"total":96636764160,"available":96026599424,"internally_reserved":0,"allocated":337514496,"data_stored":337179337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272625326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149762,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":879324,"kb_used_data":532548,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93492516,"statfs":{"total":96636764160,"available":95736336384,"internally_reserved":0,"allocated":545329152,"data_stored":545063099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355069613},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,6,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722448,"kb_used_data":438144,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93649392,"statfs":{"total":96636764160,"available":95896977408,"internally_reserved":0,"allocated":448659456,"data_stored":448389331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":544186368,"data_stored":544182755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":336920576,"data_stored":336847857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":447365120,"data_stored":447357435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:06.909 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:07.431 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:07.432 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:07.455 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-10T15:57:06.510681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631656974,"num_objects":154217,"num_object_clones":0,"num_object_copies":308434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38232,"num_objects_misplaced":1934,"num_objects_unfound":0,"num_objects_dirty":154217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138837,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6483,"num_bytes_recovered":24281107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160917,"ondisk_log_size":160917,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2186352,"kb_used_data":1289012,"kb_used_omap":12,"kb_used_meta":897267,"kb_avail":280929168,"statfs":{"total":289910292480,"available":287671468032,"internally_reserved":0,"allocated":1319948288,"data_stored":1319076951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":918801413},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3,6,0,0,5,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20176896,"num_objects":-4926,"num_object_clones":0,"num_object_copies":-9852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-221,"num_objects_misplaced":-275,"num_objects_unfound":0,"num_objects_dirty":-4926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4926,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":1126400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004538"},"pg_stats":[{"pgid":"2.f","version":"46'11080","reported_seq":11289,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.538359+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:04.538359+0000","last_peered":"2024-09-10T15:57:04.538359+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:04.538359+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39690240,"num_objects":9690,"num_object_clones":0,"num_object_copies":19380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3907,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11080,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5783},{"shards":"0,2","objects":3278}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11271","reported_seq":11378,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.954946+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:04.954946+0000","last_peered":"2024-09-10T15:57:04.954946+0000","last_clean":"2024-09-10T15:57:04.954946+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:04.954946+0000","last_undegraded":"2024-09-10T15:57:04.954946+0000","last_fullsized":"2024-09-10T15:57:04.954946+0000","mapping_epoch":36,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40153088,"num_objects":9803,"num_object_clones":0,"num_object_copies":19606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9607,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11034","reported_seq":11232,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.971185+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:04.971185+0000","last_peered":"2024-09-10T15:57:04.971185+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:04.971185+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39301144,"num_objects":9596,"num_object_clones":0,"num_object_copies":19192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3766,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9302,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3083},{"shards":"2","objects":5831}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11088","reported_seq":11168,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.954117+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:04.954117+0000","last_peered":"2024-09-10T15:57:04.954117+0000","last_clean":"2024-09-10T15:57:04.954117+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:04.954117+0000","last_undegraded":"2024-09-10T15:57:04.954117+0000","last_fullsized":"2024-09-10T15:57:04.954117+0000","mapping_epoch":36,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39297024,"num_objects":9594,"num_object_clones":0,"num_object_copies":19188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11088,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10848","reported_seq":10975,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.469102+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:04.469102+0000","last_peered":"2024-09-10T15:57:04.469102+0000","last_clean":"2024-09-10T15:57:04.469102+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:04.469102+0000","last_undegraded":"2024-09-10T15:57:04.469102+0000","last_fullsized":"2024-09-10T15:57:04.469102+0000","mapping_epoch":35,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11130","reported_seq":15138,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:04.499378+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:04.499378+0000","last_peered":"2024-09-10T15:57:04.499378+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:04.499378+0000","last_undegraded":"2024-09-10T15:57:04.499378+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39882752,"num_objects":9737,"num_object_clones":0,"num_object_copies":19474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1934,"num_objects_unfound":0,"num_objects_dirty":9737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7848,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1936,"num_bytes_recovered":7921664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7803},{"shards":"0,2","objects":1934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10832","reported_seq":11067,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.448114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:04.448114+0000","last_peered":"2024-09-10T15:57:04.448114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:04.448114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38723584,"num_objects":9454,"num_object_clones":0,"num_object_copies":18908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7482,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":3148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11179","reported_seq":11310,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.430478+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:04.430478+0000","last_peered":"2024-09-10T15:57:04.430478+0000","last_clean":"2024-09-10T15:57:04.430478+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:04.430478+0000","last_undegraded":"2024-09-10T15:57:04.430478+0000","last_fullsized":"2024-09-10T15:57:04.430478+0000","mapping_epoch":35,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39874560,"num_objects":9735,"num_object_clones":0,"num_object_copies":19470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7938,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11094","reported_seq":19221,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.438240+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:04.438240+0000","last_peered":"2024-09-10T15:57:04.438240+0000","last_clean":"2024-09-10T15:57:04.438240+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:04.438240+0000","last_undegraded":"2024-09-10T15:57:04.438240+0000","last_fullsized":"2024-09-10T15:57:04.438240+0000","mapping_epoch":45,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39424000,"num_objects":9625,"num_object_clones":0,"num_object_copies":19250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9071,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11002","reported_seq":11267,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.461083+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:04.461083+0000","last_peered":"2024-09-10T15:57:04.461083+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:04.461083+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39223296,"num_objects":9576,"num_object_clones":0,"num_object_copies":19152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5782},{"shards":"0,2","objects":3154}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11111","reported_seq":11288,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.927587+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:04.927587+0000","last_peered":"2024-09-10T15:57:04.927587+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:04.927587+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39624704,"num_objects":9674,"num_object_clones":0,"num_object_copies":19348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9085,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3116},{"shards":"2","objects":5868}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10965","reported_seq":11136,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.928402+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:04.928402+0000","last_peered":"2024-09-10T15:57:04.928402+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:04.928402+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38875136,"num_objects":9491,"num_object_clones":0,"num_object_copies":18982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8834,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3094},{"shards":"2","objects":5684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11168","reported_seq":11466,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.498724+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:04.498724+0000","last_peered":"2024-09-10T15:57:04.498724+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:04.498724+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39673856,"num_objects":9686,"num_object_clones":0,"num_object_copies":19372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3749,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8360,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5938},{"shards":"0,2","objects":3093}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11135","reported_seq":11433,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.445772+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:04.445772+0000","last_peered":"2024-09-10T15:57:04.445772+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:04.445772+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39251968,"num_objects":9583,"num_object_clones":0,"num_object_copies":19166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3853,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7553,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5733},{"shards":"0,2","objects":3145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11098","reported_seq":11354,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.499716+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:04.499716+0000","last_peered":"2024-09-10T15:57:04.499716+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:04.499716+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39600128,"num_objects":9668,"num_object_clones":0,"num_object_copies":19336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7704,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5849},{"shards":"0,2","objects":3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11279","reported_seq":11562,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.404088+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:04.404088+0000","last_peered":"2024-09-10T15:57:04.404088+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:04.404088+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40219107,"num_objects":9820,"num_object_clones":0,"num_object_copies":19640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3934,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8098,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5889},{"shards":"0,2","objects":3279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":631656955,"num_objects":154215,"num_object_clones":0,"num_object_copies":308430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38232,"num_objects_misplaced":1934,"num_objects_unfound":0,"num_objects_dirty":154215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138835,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6481,"num_bytes_recovered":24281088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1317064704,"data_stored":1316980687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599340,"kb_used_data":328916,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93772500,"statfs":{"total":96636764160,"available":96023040000,"internally_reserved":0,"allocated":336809984,"data_stored":336474825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276885166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149763,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864872,"kb_used_data":526424,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93506968,"statfs":{"total":96636764160,"available":95751135232,"internally_reserved":0,"allocated":539058176,"data_stored":538792123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346549933},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,2,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722140,"kb_used_data":433672,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93649700,"statfs":{"total":96636764160,"available":95897292800,"internally_reserved":0,"allocated":444080128,"data_stored":443810003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,4,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":537800704,"data_stored":537797091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":336392192,"data_stored":336319473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":442871808,"data_stored":442864123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:07.456 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:08.012 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:08.012 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:08.034 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-10T15:57:06.510681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631656974,"num_objects":154217,"num_object_clones":0,"num_object_copies":308434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38232,"num_objects_misplaced":1934,"num_objects_unfound":0,"num_objects_dirty":154217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138837,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6483,"num_bytes_recovered":24281107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160917,"ondisk_log_size":160917,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2186352,"kb_used_data":1289012,"kb_used_omap":12,"kb_used_meta":897267,"kb_avail":280929168,"statfs":{"total":289910292480,"available":287671468032,"internally_reserved":0,"allocated":1319948288,"data_stored":1319076951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":918801413},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3,6,0,0,5,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20176896,"num_objects":-4926,"num_object_clones":0,"num_object_copies":-9852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-221,"num_objects_misplaced":-275,"num_objects_unfound":0,"num_objects_dirty":-4926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4926,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":1126400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004538"},"pg_stats":[{"pgid":"2.f","version":"46'11080","reported_seq":11289,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.538359+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:04.538359+0000","last_peered":"2024-09-10T15:57:04.538359+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:04.538359+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39690240,"num_objects":9690,"num_object_clones":0,"num_object_copies":19380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3907,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11080,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5783},{"shards":"0,2","objects":3278}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11271","reported_seq":11378,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.954946+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:04.954946+0000","last_peered":"2024-09-10T15:57:04.954946+0000","last_clean":"2024-09-10T15:57:04.954946+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:04.954946+0000","last_undegraded":"2024-09-10T15:57:04.954946+0000","last_fullsized":"2024-09-10T15:57:04.954946+0000","mapping_epoch":36,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40153088,"num_objects":9803,"num_object_clones":0,"num_object_copies":19606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9607,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11034","reported_seq":11232,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.971185+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:04.971185+0000","last_peered":"2024-09-10T15:57:04.971185+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:04.971185+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39301144,"num_objects":9596,"num_object_clones":0,"num_object_copies":19192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3766,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9302,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3083},{"shards":"2","objects":5831}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11088","reported_seq":11168,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.954117+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:04.954117+0000","last_peered":"2024-09-10T15:57:04.954117+0000","last_clean":"2024-09-10T15:57:04.954117+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:04.954117+0000","last_undegraded":"2024-09-10T15:57:04.954117+0000","last_fullsized":"2024-09-10T15:57:04.954117+0000","mapping_epoch":36,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39297024,"num_objects":9594,"num_object_clones":0,"num_object_copies":19188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11088,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10848","reported_seq":10975,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.469102+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:04.469102+0000","last_peered":"2024-09-10T15:57:04.469102+0000","last_clean":"2024-09-10T15:57:04.469102+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:04.469102+0000","last_undegraded":"2024-09-10T15:57:04.469102+0000","last_fullsized":"2024-09-10T15:57:04.469102+0000","mapping_epoch":35,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11130","reported_seq":15138,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:04.499378+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:04.499378+0000","last_peered":"2024-09-10T15:57:04.499378+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:04.499378+0000","last_undegraded":"2024-09-10T15:57:04.499378+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39882752,"num_objects":9737,"num_object_clones":0,"num_object_copies":19474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1934,"num_objects_unfound":0,"num_objects_dirty":9737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7848,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1936,"num_bytes_recovered":7921664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7803},{"shards":"0,2","objects":1934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10832","reported_seq":11067,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.448114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:04.448114+0000","last_peered":"2024-09-10T15:57:04.448114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:04.448114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38723584,"num_objects":9454,"num_object_clones":0,"num_object_copies":18908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7482,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":3148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11179","reported_seq":11310,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.430478+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:04.430478+0000","last_peered":"2024-09-10T15:57:04.430478+0000","last_clean":"2024-09-10T15:57:04.430478+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:04.430478+0000","last_undegraded":"2024-09-10T15:57:04.430478+0000","last_fullsized":"2024-09-10T15:57:04.430478+0000","mapping_epoch":35,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39874560,"num_objects":9735,"num_object_clones":0,"num_object_copies":19470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7938,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11094","reported_seq":19221,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.438240+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:04.438240+0000","last_peered":"2024-09-10T15:57:04.438240+0000","last_clean":"2024-09-10T15:57:04.438240+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:04.438240+0000","last_undegraded":"2024-09-10T15:57:04.438240+0000","last_fullsized":"2024-09-10T15:57:04.438240+0000","mapping_epoch":45,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39424000,"num_objects":9625,"num_object_clones":0,"num_object_copies":19250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9071,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11002","reported_seq":11267,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.461083+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:04.461083+0000","last_peered":"2024-09-10T15:57:04.461083+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:04.461083+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39223296,"num_objects":9576,"num_object_clones":0,"num_object_copies":19152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5782},{"shards":"0,2","objects":3154}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11111","reported_seq":11288,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.927587+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:04.927587+0000","last_peered":"2024-09-10T15:57:04.927587+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:04.927587+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39624704,"num_objects":9674,"num_object_clones":0,"num_object_copies":19348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9085,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3116},{"shards":"2","objects":5868}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10965","reported_seq":11136,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.928402+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:04.928402+0000","last_peered":"2024-09-10T15:57:04.928402+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:04.928402+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38875136,"num_objects":9491,"num_object_clones":0,"num_object_copies":18982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8834,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3094},{"shards":"2","objects":5684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11168","reported_seq":11466,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.498724+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:04.498724+0000","last_peered":"2024-09-10T15:57:04.498724+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:04.498724+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39673856,"num_objects":9686,"num_object_clones":0,"num_object_copies":19372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3749,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8360,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5938},{"shards":"0,2","objects":3093}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11135","reported_seq":11433,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.445772+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:04.445772+0000","last_peered":"2024-09-10T15:57:04.445772+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:04.445772+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39251968,"num_objects":9583,"num_object_clones":0,"num_object_copies":19166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3853,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7553,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5733},{"shards":"0,2","objects":3145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11098","reported_seq":11354,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.499716+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:04.499716+0000","last_peered":"2024-09-10T15:57:04.499716+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:04.499716+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39600128,"num_objects":9668,"num_object_clones":0,"num_object_copies":19336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7704,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5849},{"shards":"0,2","objects":3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11279","reported_seq":11562,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.404088+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:04.404088+0000","last_peered":"2024-09-10T15:57:04.404088+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:04.404088+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40219107,"num_objects":9820,"num_object_clones":0,"num_object_copies":19640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3934,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8098,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5889},{"shards":"0,2","objects":3279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":631656955,"num_objects":154215,"num_object_clones":0,"num_object_copies":308430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38232,"num_objects_misplaced":1934,"num_objects_unfound":0,"num_objects_dirty":154215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138835,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6481,"num_bytes_recovered":24281088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1317064704,"data_stored":1316980687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599340,"kb_used_data":328916,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93772500,"statfs":{"total":96636764160,"available":96023040000,"internally_reserved":0,"allocated":336809984,"data_stored":336474825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276885166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149763,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864872,"kb_used_data":526424,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93506968,"statfs":{"total":96636764160,"available":95751135232,"internally_reserved":0,"allocated":539058176,"data_stored":538792123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346549933},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,2,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722140,"kb_used_data":433672,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93649700,"statfs":{"total":96636764160,"available":95897292800,"internally_reserved":0,"allocated":444080128,"data_stored":443810003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,4,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":537800704,"data_stored":537797091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":336392192,"data_stored":336319473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":442871808,"data_stored":442864123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:08.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:57:08.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'11080", 'reported_seq': 11289, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:04.538359+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:57:04.538359+0000', 'last_peered': '2024-09-10T15:57:04.538359+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:57:04.538359+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'1000", 'ondisk_log_start': "37'1000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39690240, 'num_objects': 9690, 'num_object_clones': 0, 'num_object_copies': 19380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3907, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11080, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5783}, {'shards': '0,2', 'objects': 3278}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:08.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:57:08.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'11034", 'reported_seq': 11232, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:04.971185+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:57:04.971185+0000', 'last_peered': '2024-09-10T15:57:04.971185+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:57:04.971185+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'1000", 'ondisk_log_start': "37'1000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39301144, 'num_objects': 9596, 'num_object_clones': 0, 'num_object_copies': 19192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3766, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9596, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9302, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3083}, {'shards': '2', 'objects': 5831}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:08.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:57:08.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'11130", 'reported_seq': 15138, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:57:04.499378+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:57:04.499378+0000', 'last_peered': '2024-09-10T15:57:04.499378+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:57:04.499378+0000', 'last_undegraded': '2024-09-10T15:57:04.499378+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39882752, 'num_objects': 9737, 'num_object_clones': 0, 'num_object_copies': 19474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1934, 'num_objects_unfound': 0, 'num_objects_dirty': 9737, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7848, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1936, 'num_bytes_recovered': 7921664, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7803}, {'shards': '0,2', 'objects': 1934}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:08.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:57:08.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'10832", 'reported_seq': 11067, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:04.448114+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:57:04.448114+0000', 'last_peered': '2024-09-10T15:57:04.448114+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:57:04.448114+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38723584, 'num_objects': 9454, 'num_object_clones': 0, 'num_object_copies': 18908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3789, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9454, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7482, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5667}, {'shards': '0,2', 'objects': 3148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:08.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:57:08.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'11002", 'reported_seq': 11267, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:04.461083+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:57:04.461083+0000', 'last_peered': '2024-09-10T15:57:04.461083+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:57:04.461083+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'1000", 'ondisk_log_start': "37'1000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39223296, 'num_objects': 9576, 'num_object_clones': 0, 'num_object_copies': 19152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3796, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9576, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7866, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5782}, {'shards': '0,2', 'objects': 3154}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:08.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:57:08.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'11111", 'reported_seq': 11288, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:04.927587+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:57:04.927587+0000', 'last_peered': '2024-09-10T15:57:04.927587+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:57:04.927587+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39624704, 'num_objects': 9674, 'num_object_clones': 0, 'num_object_copies': 19348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3808, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9674, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9085, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3116}, {'shards': '2', 'objects': 5868}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:08.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:57:08.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'10965", 'reported_seq': 11136, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:04.928402+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:57:04.928402+0000', 'last_peered': '2024-09-10T15:57:04.928402+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:57:04.928402+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38875136, 'num_objects': 9491, 'num_object_clones': 0, 'num_object_copies': 18982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3809, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9491, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8834, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3094}, {'shards': '2', 'objects': 5684}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:08.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:57:08.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'11168", 'reported_seq': 11466, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:04.498724+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:57:04.498724+0000', 'last_peered': '2024-09-10T15:57:04.498724+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:57:04.498724+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39673856, 'num_objects': 9686, 'num_object_clones': 0, 'num_object_copies': 19372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3749, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8360, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5938}, {'shards': '0,2', 'objects': 3093}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:08.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:57:08.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'11135", 'reported_seq': 11433, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:04.445772+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:57:04.445772+0000', 'last_peered': '2024-09-10T15:57:04.445772+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:57:04.445772+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39251968, 'num_objects': 9583, 'num_object_clones': 0, 'num_object_copies': 19166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3853, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7553, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5733}, {'shards': '0,2', 'objects': 3145}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:08.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:57:08.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'11098", 'reported_seq': 11354, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:04.499716+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:57:04.499716+0000', 'last_peered': '2024-09-10T15:57:04.499716+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:57:04.499716+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'1000", 'ondisk_log_start': "37'1000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39600128, 'num_objects': 9668, 'num_object_clones': 0, 'num_object_copies': 19336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3821, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9668, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7704, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5849}, {'shards': '0,2', 'objects': 3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:08.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:57:08.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'11279", 'reported_seq': 11562, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:04.404088+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:57:04.404088+0000', 'last_peered': '2024-09-10T15:57:04.404088+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:57:04.404088+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40219107, 'num_objects': 9820, 'num_object_clones': 0, 'num_object_copies': 19640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3934, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9820, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8098, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 72, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5889}, {'shards': '0,2', 'objects': 3279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:08.038 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:57:08.687 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:08.687 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38232/308434 objects degraded (12.396%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":354,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":154217,"data_bytes":631656974,"bytes_used":2238824448,"bytes_avail":287671468032,"bytes_total":289910292480,"degraded_objects":38232,"degraded_total":308434,"degraded_ratio":0.12395520597599488,"misplaced_objects":1934,"misplaced_total":308434,"misplaced_ratio":0.0062703852363876871,"recovering_objects_per_sec":41,"recovering_bytes_per_sec":93831,"recovering_keys_per_sec":0,"num_objects_recovered":498,"num_bytes_recovered":1126400,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":410},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T15:56:24.496337+0000","services":{}},"progress_events":{}} 2024-09-10T15:57:08.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:57:08.711 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:09.253 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:09.254 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:09.278 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-10T15:57:08.511353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631656974,"num_objects":154217,"num_object_clones":0,"num_object_copies":308434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38232,"num_objects_misplaced":1934,"num_objects_unfound":0,"num_objects_dirty":154217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138837,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6483,"num_bytes_recovered":24281107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160917,"ondisk_log_size":160917,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2186352,"kb_used_data":1289012,"kb_used_omap":12,"kb_used_meta":897267,"kb_avail":280929168,"statfs":{"total":289910292480,"available":287671468032,"internally_reserved":0,"allocated":1319948288,"data_stored":1319076951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":918801413},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3,6,0,0,5,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17498112,"num_objects":-4272,"num_object_clones":0,"num_object_copies":-8544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":-275,"num_objects_unfound":0,"num_objects_dirty":-4272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4272,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":1126400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004453"},"pg_stats":[{"pgid":"2.f","version":"46'11080","reported_seq":11289,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.538359+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:04.538359+0000","last_peered":"2024-09-10T15:57:04.538359+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:04.538359+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39690240,"num_objects":9690,"num_object_clones":0,"num_object_copies":19380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3907,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11080,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5783},{"shards":"0,2","objects":3278}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11271","reported_seq":11378,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.954946+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:04.954946+0000","last_peered":"2024-09-10T15:57:04.954946+0000","last_clean":"2024-09-10T15:57:04.954946+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:04.954946+0000","last_undegraded":"2024-09-10T15:57:04.954946+0000","last_fullsized":"2024-09-10T15:57:04.954946+0000","mapping_epoch":36,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40153088,"num_objects":9803,"num_object_clones":0,"num_object_copies":19606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9607,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11034","reported_seq":11232,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.971185+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:04.971185+0000","last_peered":"2024-09-10T15:57:04.971185+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:04.971185+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39301144,"num_objects":9596,"num_object_clones":0,"num_object_copies":19192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3766,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9302,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3083},{"shards":"2","objects":5831}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11088","reported_seq":11168,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.954117+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:04.954117+0000","last_peered":"2024-09-10T15:57:04.954117+0000","last_clean":"2024-09-10T15:57:04.954117+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:04.954117+0000","last_undegraded":"2024-09-10T15:57:04.954117+0000","last_fullsized":"2024-09-10T15:57:04.954117+0000","mapping_epoch":36,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39297024,"num_objects":9594,"num_object_clones":0,"num_object_copies":19188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11088,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10848","reported_seq":10975,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.469102+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:04.469102+0000","last_peered":"2024-09-10T15:57:04.469102+0000","last_clean":"2024-09-10T15:57:04.469102+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:04.469102+0000","last_undegraded":"2024-09-10T15:57:04.469102+0000","last_fullsized":"2024-09-10T15:57:04.469102+0000","mapping_epoch":35,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11130","reported_seq":15138,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:04.499378+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:04.499378+0000","last_peered":"2024-09-10T15:57:04.499378+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:04.499378+0000","last_undegraded":"2024-09-10T15:57:04.499378+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39882752,"num_objects":9737,"num_object_clones":0,"num_object_copies":19474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1934,"num_objects_unfound":0,"num_objects_dirty":9737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7848,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1936,"num_bytes_recovered":7921664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7803},{"shards":"0,2","objects":1934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10832","reported_seq":11067,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.448114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:04.448114+0000","last_peered":"2024-09-10T15:57:04.448114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:04.448114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38723584,"num_objects":9454,"num_object_clones":0,"num_object_copies":18908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7482,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":3148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11179","reported_seq":11310,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.430478+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:04.430478+0000","last_peered":"2024-09-10T15:57:04.430478+0000","last_clean":"2024-09-10T15:57:04.430478+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:04.430478+0000","last_undegraded":"2024-09-10T15:57:04.430478+0000","last_fullsized":"2024-09-10T15:57:04.430478+0000","mapping_epoch":35,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39874560,"num_objects":9735,"num_object_clones":0,"num_object_copies":19470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7938,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11094","reported_seq":19221,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:04.438240+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:04.438240+0000","last_peered":"2024-09-10T15:57:04.438240+0000","last_clean":"2024-09-10T15:57:04.438240+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:04.438240+0000","last_undegraded":"2024-09-10T15:57:04.438240+0000","last_fullsized":"2024-09-10T15:57:04.438240+0000","mapping_epoch":45,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39424000,"num_objects":9625,"num_object_clones":0,"num_object_copies":19250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9071,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11002","reported_seq":11267,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.461083+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:04.461083+0000","last_peered":"2024-09-10T15:57:04.461083+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:04.461083+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39223296,"num_objects":9576,"num_object_clones":0,"num_object_copies":19152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5782},{"shards":"0,2","objects":3154}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11111","reported_seq":11288,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.927587+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:04.927587+0000","last_peered":"2024-09-10T15:57:04.927587+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:04.927587+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39624704,"num_objects":9674,"num_object_clones":0,"num_object_copies":19348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9085,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3116},{"shards":"2","objects":5868}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'10965","reported_seq":11136,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.928402+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:04.928402+0000","last_peered":"2024-09-10T15:57:04.928402+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:04.928402+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38875136,"num_objects":9491,"num_object_clones":0,"num_object_copies":18982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8834,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3094},{"shards":"2","objects":5684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11168","reported_seq":11466,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.498724+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:04.498724+0000","last_peered":"2024-09-10T15:57:04.498724+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:04.498724+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39673856,"num_objects":9686,"num_object_clones":0,"num_object_copies":19372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3749,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8360,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5938},{"shards":"0,2","objects":3093}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11135","reported_seq":11433,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.445772+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:04.445772+0000","last_peered":"2024-09-10T15:57:04.445772+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:04.445772+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39251968,"num_objects":9583,"num_object_clones":0,"num_object_copies":19166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3853,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7553,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5733},{"shards":"0,2","objects":3145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11098","reported_seq":11354,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.499716+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:04.499716+0000","last_peered":"2024-09-10T15:57:04.499716+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:04.499716+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39600128,"num_objects":9668,"num_object_clones":0,"num_object_copies":19336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7704,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5849},{"shards":"0,2","objects":3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11279","reported_seq":11562,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:04.404088+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:04.404088+0000","last_peered":"2024-09-10T15:57:04.404088+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:04.404088+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40219107,"num_objects":9820,"num_object_clones":0,"num_object_copies":19640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3934,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8098,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5889},{"shards":"0,2","objects":3279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":631656955,"num_objects":154215,"num_object_clones":0,"num_object_copies":308430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38232,"num_objects_misplaced":1934,"num_objects_unfound":0,"num_objects_dirty":154215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138835,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6481,"num_bytes_recovered":24281088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1317064704,"data_stored":1316980687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599340,"kb_used_data":328916,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93772500,"statfs":{"total":96636764160,"available":96023040000,"internally_reserved":0,"allocated":336809984,"data_stored":336474825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276885166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149763,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864872,"kb_used_data":526424,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93506968,"statfs":{"total":96636764160,"available":95751135232,"internally_reserved":0,"allocated":539058176,"data_stored":538792123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346549933},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,2,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722140,"kb_used_data":433672,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93649700,"statfs":{"total":96636764160,"available":95897292800,"internally_reserved":0,"allocated":444080128,"data_stored":443810003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,4,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":537800704,"data_stored":537797091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":336392192,"data_stored":336319473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":442871808,"data_stored":442864123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:12.280 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:12.847 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:12.848 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:12.873 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-10T15:57:12.512744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622658062,"num_objects":152020,"num_object_clones":0,"num_object_copies":304040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38156,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":152020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":141034,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6687,"num_bytes_recovered":24805395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174672,"kb_used_data":1277340,"kb_used_omap":12,"kb_used_meta":897267,"kb_avail":280940848,"statfs":{"total":289910292480,"available":287683428352,"internally_reserved":0,"allocated":1307996160,"data_stored":1307124823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":918801413},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,7,0,11,5,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17661952,"num_objects":-4312,"num_object_clones":0,"num_object_copies":-8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-195,"num_objects_misplaced":-275,"num_objects_unfound":0,"num_objects_dirty":-4312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4312,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":470,"num_bytes_recovered":1126400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004380"},"pg_stats":[{"pgid":"2.f","version":"46'11221","reported_seq":11458,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.504641+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:09.504641+0000","last_peered":"2024-09-10T15:57:09.504641+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:09.504641+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11221,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5656},{"shards":"0,2","objects":3137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11426","reported_seq":11533,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:09.947516+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:09.947516+0000","last_peered":"2024-09-10T15:57:09.947516+0000","last_clean":"2024-09-10T15:57:09.947516+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:09.947516+0000","last_undegraded":"2024-09-10T15:57:09.947516+0000","last_fullsized":"2024-09-10T15:57:09.947516+0000","mapping_epoch":36,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39518208,"num_objects":9648,"num_object_clones":0,"num_object_copies":19296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9762,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11171","reported_seq":11376,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.893548+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:09.893548+0000","last_peered":"2024-09-10T15:57:09.893548+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:09.893548+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38739992,"num_objects":9459,"num_object_clones":0,"num_object_copies":18918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3763,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9439,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2946},{"shards":"2","objects":5697}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11217","reported_seq":11297,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:10.013602+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:10.013602+0000","last_peered":"2024-09-10T15:57:10.013602+0000","last_clean":"2024-09-10T15:57:10.013602+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:10.013602+0000","last_undegraded":"2024-09-10T15:57:10.013602+0000","last_fullsized":"2024-09-10T15:57:10.013602+0000","mapping_epoch":36,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38768640,"num_objects":9465,"num_object_clones":0,"num_object_copies":18930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11217,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10991","reported_seq":11118,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:09.461002+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:09.461002+0000","last_peered":"2024-09-10T15:57:09.461002+0000","last_clean":"2024-09-10T15:57:09.461002+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:09.461002+0000","last_undegraded":"2024-09-10T15:57:09.461002+0000","last_fullsized":"2024-09-10T15:57:09.461002+0000","mapping_epoch":35,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38256640,"num_objects":9340,"num_object_clones":0,"num_object_copies":18680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8062,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11256","reported_seq":15519,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:09.501167+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:09.501167+0000","last_peered":"2024-09-10T15:57:09.501167+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:09.501167+0000","last_undegraded":"2024-09-10T15:57:09.501167+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39366656,"num_objects":9611,"num_object_clones":0,"num_object_copies":19222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":9611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7974,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2064,"num_bytes_recovered":8445952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7805},{"shards":"0,2","objects":1806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10966","reported_seq":11217,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.502887+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:09.502887+0000","last_peered":"2024-09-10T15:57:09.502887+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:09.502887+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38174720,"num_objects":9320,"num_object_clones":0,"num_object_copies":18640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3781,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7616,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5541},{"shards":"0,2","objects":3014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11321","reported_seq":11452,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:09.538733+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:09.538733+0000","last_peered":"2024-09-10T15:57:09.538733+0000","last_clean":"2024-09-10T15:57:09.538733+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:09.538733+0000","last_undegraded":"2024-09-10T15:57:09.538733+0000","last_fullsized":"2024-09-10T15:57:09.538733+0000","mapping_epoch":35,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39292928,"num_objects":9593,"num_object_clones":0,"num_object_copies":19186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11252","reported_seq":19379,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:09.484329+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:09.484329+0000","last_peered":"2024-09-10T15:57:09.484329+0000","last_clean":"2024-09-10T15:57:09.484329+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:09.484329+0000","last_undegraded":"2024-09-10T15:57:09.484329+0000","last_fullsized":"2024-09-10T15:57:09.484329+0000","mapping_epoch":45,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38776832,"num_objects":9467,"num_object_clones":0,"num_object_copies":18934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9229,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11134","reported_seq":11413,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.465762+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:09.465762+0000","last_peered":"2024-09-10T15:57:09.465762+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:09.465762+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38682624,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7998,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5657},{"shards":"0,2","objects":3022}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11256","reported_seq":11441,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:10.002077+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:10.002077+0000","last_peered":"2024-09-10T15:57:10.002077+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:10.002077+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39030784,"num_objects":9529,"num_object_clones":0,"num_object_copies":19058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9230,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2971},{"shards":"2","objects":5727}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11105","reported_seq":11290,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.889208+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:09.889208+0000","last_peered":"2024-09-10T15:57:09.889208+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:09.889208+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38301696,"num_objects":9351,"num_object_clones":0,"num_object_copies":18702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3802,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8974,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2954},{"shards":"2","objects":5551}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11283","reported_seq":11598,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.542402+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:09.542402+0000","last_peered":"2024-09-10T15:57:09.542402+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:09.542402+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39202816,"num_objects":9571,"num_object_clones":0,"num_object_copies":19142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8475,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5832},{"shards":"0,2","objects":2978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11282","reported_seq":11602,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.521185+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:09.521185+0000","last_peered":"2024-09-10T15:57:09.521185+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:09.521185+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38649856,"num_objects":9436,"num_object_clones":0,"num_object_copies":18872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7700,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5597},{"shards":"0,2","objects":2998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11217","reported_seq":11495,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.397994+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:09.397994+0000","last_peered":"2024-09-10T15:57:09.397994+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:09.397994+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3810,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7823,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5741},{"shards":"0,2","objects":3045}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11413","reported_seq":11700,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.485044+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:09.485044+0000","last_peered":"2024-09-10T15:57:09.485044+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:09.485044+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39670243,"num_objects":9686,"num_object_clones":0,"num_object_copies":19372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3932,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8232,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5757},{"shards":"0,2","objects":3145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622658043,"num_objects":152018,"num_object_clones":0,"num_object_copies":304036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38156,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":152018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":141032,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6685,"num_bytes_recovered":24805376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1305096192,"data_stored":1305012175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626481,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590516,"kb_used_data":328420,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93781324,"statfs":{"total":96636764160,"available":96032075776,"internally_reserved":0,"allocated":336302080,"data_stored":335966921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268365486},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149764,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":862492,"kb_used_data":519880,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93509348,"statfs":{"total":96636764160,"available":95753572352,"internally_reserved":0,"allocated":532357120,"data_stored":532091067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350809773},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,0,4,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721664,"kb_used_data":429040,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93650176,"statfs":{"total":96636764160,"available":95897780224,"internally_reserved":0,"allocated":439336960,"data_stored":439066835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":531136512,"data_stored":531132899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335745024,"data_stored":335672305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":438214656,"data_stored":438206971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:12.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:13.427 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:13.429 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:13.448 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-10T15:57:12.512744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622658062,"num_objects":152020,"num_object_clones":0,"num_object_copies":304040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38156,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":152020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":141034,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6687,"num_bytes_recovered":24805395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174672,"kb_used_data":1277340,"kb_used_omap":12,"kb_used_meta":897267,"kb_avail":280940848,"statfs":{"total":289910292480,"available":287683428352,"internally_reserved":0,"allocated":1307996160,"data_stored":1307124823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":918801413},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,7,0,11,5,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17661952,"num_objects":-4312,"num_object_clones":0,"num_object_copies":-8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-195,"num_objects_misplaced":-275,"num_objects_unfound":0,"num_objects_dirty":-4312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4312,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":470,"num_bytes_recovered":1126400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004380"},"pg_stats":[{"pgid":"2.f","version":"46'11221","reported_seq":11458,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.504641+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:09.504641+0000","last_peered":"2024-09-10T15:57:09.504641+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:09.504641+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11221,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5656},{"shards":"0,2","objects":3137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11426","reported_seq":11533,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:09.947516+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:09.947516+0000","last_peered":"2024-09-10T15:57:09.947516+0000","last_clean":"2024-09-10T15:57:09.947516+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:09.947516+0000","last_undegraded":"2024-09-10T15:57:09.947516+0000","last_fullsized":"2024-09-10T15:57:09.947516+0000","mapping_epoch":36,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39518208,"num_objects":9648,"num_object_clones":0,"num_object_copies":19296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9762,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11171","reported_seq":11376,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.893548+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:09.893548+0000","last_peered":"2024-09-10T15:57:09.893548+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:09.893548+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38739992,"num_objects":9459,"num_object_clones":0,"num_object_copies":18918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3763,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9439,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2946},{"shards":"2","objects":5697}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11217","reported_seq":11297,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:10.013602+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:10.013602+0000","last_peered":"2024-09-10T15:57:10.013602+0000","last_clean":"2024-09-10T15:57:10.013602+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:10.013602+0000","last_undegraded":"2024-09-10T15:57:10.013602+0000","last_fullsized":"2024-09-10T15:57:10.013602+0000","mapping_epoch":36,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38768640,"num_objects":9465,"num_object_clones":0,"num_object_copies":18930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11217,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10991","reported_seq":11118,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:09.461002+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:09.461002+0000","last_peered":"2024-09-10T15:57:09.461002+0000","last_clean":"2024-09-10T15:57:09.461002+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:09.461002+0000","last_undegraded":"2024-09-10T15:57:09.461002+0000","last_fullsized":"2024-09-10T15:57:09.461002+0000","mapping_epoch":35,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38256640,"num_objects":9340,"num_object_clones":0,"num_object_copies":18680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8062,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11256","reported_seq":15519,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:09.501167+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:09.501167+0000","last_peered":"2024-09-10T15:57:09.501167+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:09.501167+0000","last_undegraded":"2024-09-10T15:57:09.501167+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39366656,"num_objects":9611,"num_object_clones":0,"num_object_copies":19222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":9611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7974,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2064,"num_bytes_recovered":8445952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7805},{"shards":"0,2","objects":1806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10966","reported_seq":11217,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.502887+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:09.502887+0000","last_peered":"2024-09-10T15:57:09.502887+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:09.502887+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38174720,"num_objects":9320,"num_object_clones":0,"num_object_copies":18640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3781,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7616,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5541},{"shards":"0,2","objects":3014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11321","reported_seq":11452,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:09.538733+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:09.538733+0000","last_peered":"2024-09-10T15:57:09.538733+0000","last_clean":"2024-09-10T15:57:09.538733+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:09.538733+0000","last_undegraded":"2024-09-10T15:57:09.538733+0000","last_fullsized":"2024-09-10T15:57:09.538733+0000","mapping_epoch":35,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39292928,"num_objects":9593,"num_object_clones":0,"num_object_copies":19186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11252","reported_seq":19379,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:09.484329+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:09.484329+0000","last_peered":"2024-09-10T15:57:09.484329+0000","last_clean":"2024-09-10T15:57:09.484329+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:09.484329+0000","last_undegraded":"2024-09-10T15:57:09.484329+0000","last_fullsized":"2024-09-10T15:57:09.484329+0000","mapping_epoch":45,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38776832,"num_objects":9467,"num_object_clones":0,"num_object_copies":18934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9229,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11134","reported_seq":11413,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.465762+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:09.465762+0000","last_peered":"2024-09-10T15:57:09.465762+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:09.465762+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38682624,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7998,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5657},{"shards":"0,2","objects":3022}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11256","reported_seq":11441,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:10.002077+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:10.002077+0000","last_peered":"2024-09-10T15:57:10.002077+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:10.002077+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39030784,"num_objects":9529,"num_object_clones":0,"num_object_copies":19058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9230,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2971},{"shards":"2","objects":5727}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11105","reported_seq":11290,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.889208+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:09.889208+0000","last_peered":"2024-09-10T15:57:09.889208+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:09.889208+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38301696,"num_objects":9351,"num_object_clones":0,"num_object_copies":18702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3802,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8974,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2954},{"shards":"2","objects":5551}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11283","reported_seq":11598,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.542402+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:09.542402+0000","last_peered":"2024-09-10T15:57:09.542402+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:09.542402+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39202816,"num_objects":9571,"num_object_clones":0,"num_object_copies":19142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8475,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5832},{"shards":"0,2","objects":2978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11282","reported_seq":11602,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.521185+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:09.521185+0000","last_peered":"2024-09-10T15:57:09.521185+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:09.521185+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38649856,"num_objects":9436,"num_object_clones":0,"num_object_copies":18872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7700,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5597},{"shards":"0,2","objects":2998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11217","reported_seq":11495,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.397994+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:09.397994+0000","last_peered":"2024-09-10T15:57:09.397994+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:09.397994+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3810,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7823,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5741},{"shards":"0,2","objects":3045}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11413","reported_seq":11700,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.485044+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:09.485044+0000","last_peered":"2024-09-10T15:57:09.485044+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:09.485044+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39670243,"num_objects":9686,"num_object_clones":0,"num_object_copies":19372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3932,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8232,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5757},{"shards":"0,2","objects":3145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622658043,"num_objects":152018,"num_object_clones":0,"num_object_copies":304036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38156,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":152018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":141032,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6685,"num_bytes_recovered":24805376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1305096192,"data_stored":1305012175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626481,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590516,"kb_used_data":328420,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93781324,"statfs":{"total":96636764160,"available":96032075776,"internally_reserved":0,"allocated":336302080,"data_stored":335966921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268365486},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149764,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":862492,"kb_used_data":519880,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93509348,"statfs":{"total":96636764160,"available":95753572352,"internally_reserved":0,"allocated":532357120,"data_stored":532091067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350809773},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,0,4,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721664,"kb_used_data":429040,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93650176,"statfs":{"total":96636764160,"available":95897780224,"internally_reserved":0,"allocated":439336960,"data_stored":439066835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":531136512,"data_stored":531132899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335745024,"data_stored":335672305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":438214656,"data_stored":438206971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:13.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:57:13.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'11221", 'reported_seq': 11458, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:09.504641+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:57:09.504641+0000', 'last_peered': '2024-09-10T15:57:09.504641+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:57:09.504641+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39112704, 'num_objects': 9549, 'num_object_clones': 0, 'num_object_copies': 19098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3893, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11221, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 80, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5656}, {'shards': '0,2', 'objects': 3137}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:13.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:57:13.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'11171", 'reported_seq': 11376, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:09.893548+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:57:09.893548+0000', 'last_peered': '2024-09-10T15:57:09.893548+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:57:09.893548+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38739992, 'num_objects': 9459, 'num_object_clones': 0, 'num_object_copies': 18918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3763, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9459, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9439, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2946}, {'shards': '2', 'objects': 5697}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:13.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:57:13.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'11256", 'reported_seq': 15519, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:57:09.501167+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:57:09.501167+0000', 'last_peered': '2024-09-10T15:57:09.501167+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:57:09.501167+0000', 'last_undegraded': '2024-09-10T15:57:09.501167+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39366656, 'num_objects': 9611, 'num_object_clones': 0, 'num_object_copies': 19222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1806, 'num_objects_unfound': 0, 'num_objects_dirty': 9611, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7974, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2064, 'num_bytes_recovered': 8445952, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7805}, {'shards': '0,2', 'objects': 1806}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:13.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:57:13.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'10966", 'reported_seq': 11217, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:09.502887+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:57:09.502887+0000', 'last_peered': '2024-09-10T15:57:09.502887+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:57:09.502887+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38174720, 'num_objects': 9320, 'num_object_clones': 0, 'num_object_copies': 18640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3781, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9320, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7616, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5541}, {'shards': '0,2', 'objects': 3014}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:13.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:57:13.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'11134", 'reported_seq': 11413, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:09.465762+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:57:09.465762+0000', 'last_peered': '2024-09-10T15:57:09.465762+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:57:09.465762+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38682624, 'num_objects': 9444, 'num_object_clones': 0, 'num_object_copies': 18888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3789, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9444, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7998, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 76, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5657}, {'shards': '0,2', 'objects': 3022}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:13.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:57:13.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'11256", 'reported_seq': 11441, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:10.002077+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:57:10.002077+0000', 'last_peered': '2024-09-10T15:57:10.002077+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:57:10.002077+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39030784, 'num_objects': 9529, 'num_object_clones': 0, 'num_object_copies': 19058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3804, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9529, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9230, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2971}, {'shards': '2', 'objects': 5727}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:13.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:57:13.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'11105", 'reported_seq': 11290, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:09.889208+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:57:09.889208+0000', 'last_peered': '2024-09-10T15:57:09.889208+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:57:09.889208+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38301696, 'num_objects': 9351, 'num_object_clones': 0, 'num_object_copies': 18702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3802, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9351, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8974, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2954}, {'shards': '2', 'objects': 5551}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:13.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:57:13.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'11283", 'reported_seq': 11598, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:09.542402+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:57:09.542402+0000', 'last_peered': '2024-09-10T15:57:09.542402+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:57:09.542402+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39202816, 'num_objects': 9571, 'num_object_clones': 0, 'num_object_copies': 19142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3740, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9571, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8475, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 94, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5832}, {'shards': '0,2', 'objects': 2978}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:13.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:57:13.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'11282", 'reported_seq': 11602, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:09.521185+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:57:09.521185+0000', 'last_peered': '2024-09-10T15:57:09.521185+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:57:09.521185+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38649856, 'num_objects': 9436, 'num_object_clones': 0, 'num_object_copies': 18872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3842, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9436, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7700, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5597}, {'shards': '0,2', 'objects': 2998}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:13.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:57:13.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'11217", 'reported_seq': 11495, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:09.397994+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:57:09.397994+0000', 'last_peered': '2024-09-10T15:57:09.397994+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:57:09.397994+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39112704, 'num_objects': 9549, 'num_object_clones': 0, 'num_object_copies': 19098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3810, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7823, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 70, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5741}, {'shards': '0,2', 'objects': 3045}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:13.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:57:13.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'11413", 'reported_seq': 11700, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:09.485044+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:57:09.485044+0000', 'last_peered': '2024-09-10T15:57:09.485044+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:57:09.485044+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39670243, 'num_objects': 9686, 'num_object_clones': 0, 'num_object_copies': 19372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3932, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8232, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 74, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5757}, {'shards': '0,2', 'objects': 3145}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:13.452 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:57:14.135 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:14.135 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38156/304040 objects degraded (12.550%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":360,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152020,"data_bytes":622658062,"bytes_used":2226864128,"bytes_avail":287683428352,"bytes_total":289910292480,"degraded_objects":38156,"degraded_total":304040,"degraded_ratio":0.12549664517826603,"misplaced_objects":1806,"misplaced_total":304040,"misplaced_ratio":0.0059400078936981985,"recovering_objects_per_sec":39,"recovering_bytes_per_sec":93832,"recovering_keys_per_sec":0,"num_objects_recovered":470,"num_bytes_recovered":1126400,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":359},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T15:56:24.496337+0000","services":{}},"progress_events":{}} 2024-09-10T15:57:14.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:57:14.161 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:14.731 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:14.733 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:14.753 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-10T15:57:14.513327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622658062,"num_objects":152020,"num_object_clones":0,"num_object_copies":304040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38156,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":152020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":141034,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6687,"num_bytes_recovered":24805395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174672,"kb_used_data":1277340,"kb_used_omap":12,"kb_used_meta":897267,"kb_avail":280940848,"statfs":{"total":289910292480,"available":287683428352,"internally_reserved":0,"allocated":1307996160,"data_stored":1307124823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":918801413},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,7,0,11,5,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17661952,"num_objects":-4312,"num_object_clones":0,"num_object_copies":-8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-195,"num_objects_misplaced":-275,"num_objects_unfound":0,"num_objects_dirty":-4312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4312,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":470,"num_bytes_recovered":1126400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004067"},"pg_stats":[{"pgid":"2.f","version":"46'11221","reported_seq":11458,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.504641+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:09.504641+0000","last_peered":"2024-09-10T15:57:09.504641+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:09.504641+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11221,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5656},{"shards":"0,2","objects":3137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11426","reported_seq":11533,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:09.947516+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:09.947516+0000","last_peered":"2024-09-10T15:57:09.947516+0000","last_clean":"2024-09-10T15:57:09.947516+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:09.947516+0000","last_undegraded":"2024-09-10T15:57:09.947516+0000","last_fullsized":"2024-09-10T15:57:09.947516+0000","mapping_epoch":36,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39518208,"num_objects":9648,"num_object_clones":0,"num_object_copies":19296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9762,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11171","reported_seq":11376,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.893548+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:09.893548+0000","last_peered":"2024-09-10T15:57:09.893548+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:09.893548+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38739992,"num_objects":9459,"num_object_clones":0,"num_object_copies":18918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3763,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9439,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2946},{"shards":"2","objects":5697}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11217","reported_seq":11297,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:10.013602+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:10.013602+0000","last_peered":"2024-09-10T15:57:10.013602+0000","last_clean":"2024-09-10T15:57:10.013602+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:10.013602+0000","last_undegraded":"2024-09-10T15:57:10.013602+0000","last_fullsized":"2024-09-10T15:57:10.013602+0000","mapping_epoch":36,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38768640,"num_objects":9465,"num_object_clones":0,"num_object_copies":18930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11217,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'10991","reported_seq":11118,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:09.461002+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:09.461002+0000","last_peered":"2024-09-10T15:57:09.461002+0000","last_clean":"2024-09-10T15:57:09.461002+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:09.461002+0000","last_undegraded":"2024-09-10T15:57:09.461002+0000","last_fullsized":"2024-09-10T15:57:09.461002+0000","mapping_epoch":35,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38256640,"num_objects":9340,"num_object_clones":0,"num_object_copies":18680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8062,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11256","reported_seq":15519,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:09.501167+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:09.501167+0000","last_peered":"2024-09-10T15:57:09.501167+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:09.501167+0000","last_undegraded":"2024-09-10T15:57:09.501167+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39366656,"num_objects":9611,"num_object_clones":0,"num_object_copies":19222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":9611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7974,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2064,"num_bytes_recovered":8445952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7805},{"shards":"0,2","objects":1806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'10966","reported_seq":11217,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.502887+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:09.502887+0000","last_peered":"2024-09-10T15:57:09.502887+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:09.502887+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38174720,"num_objects":9320,"num_object_clones":0,"num_object_copies":18640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3781,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7616,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5541},{"shards":"0,2","objects":3014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11321","reported_seq":11452,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:09.538733+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:09.538733+0000","last_peered":"2024-09-10T15:57:09.538733+0000","last_clean":"2024-09-10T15:57:09.538733+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:09.538733+0000","last_undegraded":"2024-09-10T15:57:09.538733+0000","last_fullsized":"2024-09-10T15:57:09.538733+0000","mapping_epoch":35,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39292928,"num_objects":9593,"num_object_clones":0,"num_object_copies":19186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11252","reported_seq":19379,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:09.484329+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:09.484329+0000","last_peered":"2024-09-10T15:57:09.484329+0000","last_clean":"2024-09-10T15:57:09.484329+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:09.484329+0000","last_undegraded":"2024-09-10T15:57:09.484329+0000","last_fullsized":"2024-09-10T15:57:09.484329+0000","mapping_epoch":45,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38776832,"num_objects":9467,"num_object_clones":0,"num_object_copies":18934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9229,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11134","reported_seq":11413,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.465762+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:09.465762+0000","last_peered":"2024-09-10T15:57:09.465762+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:09.465762+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38682624,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7998,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5657},{"shards":"0,2","objects":3022}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11256","reported_seq":11441,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:10.002077+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:10.002077+0000","last_peered":"2024-09-10T15:57:10.002077+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:10.002077+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39030784,"num_objects":9529,"num_object_clones":0,"num_object_copies":19058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9230,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2971},{"shards":"2","objects":5727}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11105","reported_seq":11290,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.889208+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:09.889208+0000","last_peered":"2024-09-10T15:57:09.889208+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:09.889208+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38301696,"num_objects":9351,"num_object_clones":0,"num_object_copies":18702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3802,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8974,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2954},{"shards":"2","objects":5551}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11283","reported_seq":11598,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.542402+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:09.542402+0000","last_peered":"2024-09-10T15:57:09.542402+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:09.542402+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39202816,"num_objects":9571,"num_object_clones":0,"num_object_copies":19142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8475,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5832},{"shards":"0,2","objects":2978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11282","reported_seq":11602,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.521185+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:09.521185+0000","last_peered":"2024-09-10T15:57:09.521185+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:09.521185+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38649856,"num_objects":9436,"num_object_clones":0,"num_object_copies":18872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7700,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5597},{"shards":"0,2","objects":2998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11217","reported_seq":11495,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.397994+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:09.397994+0000","last_peered":"2024-09-10T15:57:09.397994+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:09.397994+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3810,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7823,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5741},{"shards":"0,2","objects":3045}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11413","reported_seq":11700,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:09.485044+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:09.485044+0000","last_peered":"2024-09-10T15:57:09.485044+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:09.485044+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39670243,"num_objects":9686,"num_object_clones":0,"num_object_copies":19372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3932,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8232,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5757},{"shards":"0,2","objects":3145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622658043,"num_objects":152018,"num_object_clones":0,"num_object_copies":304036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38156,"num_objects_misplaced":1806,"num_objects_unfound":0,"num_objects_dirty":152018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":141032,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6685,"num_bytes_recovered":24805376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1305096192,"data_stored":1305012175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626481,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590516,"kb_used_data":328420,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93781324,"statfs":{"total":96636764160,"available":96032075776,"internally_reserved":0,"allocated":336302080,"data_stored":335966921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268365486},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149764,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":862492,"kb_used_data":519880,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93509348,"statfs":{"total":96636764160,"available":95753572352,"internally_reserved":0,"allocated":532357120,"data_stored":532091067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350809773},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,0,4,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721664,"kb_used_data":429040,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93650176,"statfs":{"total":96636764160,"available":95897780224,"internally_reserved":0,"allocated":439336960,"data_stored":439066835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":531136512,"data_stored":531132899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335745024,"data_stored":335672305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":438214656,"data_stored":438206971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:17.756 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:18.326 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:18.328 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:18.351 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-10T15:57:16.514107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613782030,"num_objects":149853,"num_object_clones":0,"num_object_copies":299706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38035,"num_objects_misplaced":1689,"num_objects_unfound":0,"num_objects_dirty":149853,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143201,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6925,"num_bytes_recovered":25284627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2158924,"kb_used_data":1265736,"kb_used_omap":12,"kb_used_meta":893107,"kb_avail":280956596,"statfs":{"total":289910292480,"available":287699554304,"internally_reserved":0,"allocated":1296113664,"data_stored":1295242327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":914541573},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,2,7,8,3,0,0,11],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25997312,"num_objects":-6347,"num_object_clones":0,"num_object_copies":-12694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-316,"num_objects_misplaced":-392,"num_objects_unfound":0,"num_objects_dirty":-6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6347,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":708,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004069"},"pg_stats":[{"pgid":"2.f","version":"46'11350","reported_seq":11629,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.504191+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:14.504191+0000","last_peered":"2024-09-10T15:57:14.504191+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:14.504191+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38584320,"num_objects":9420,"num_object_clones":0,"num_object_copies":18840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3872,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11350,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5548},{"shards":"0,2","objects":3008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11567","reported_seq":11674,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.943139+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:14.943139+0000","last_peered":"2024-09-10T15:57:14.943139+0000","last_clean":"2024-09-10T15:57:14.943139+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:14.943139+0000","last_undegraded":"2024-09-10T15:57:14.943139+0000","last_fullsized":"2024-09-10T15:57:14.943139+0000","mapping_epoch":36,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38940672,"num_objects":9507,"num_object_clones":0,"num_object_copies":19014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9903,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11291","reported_seq":11498,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.867033+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:14.867033+0000","last_peered":"2024-09-10T15:57:14.867033+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:14.867033+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248472,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9559,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2826},{"shards":"2","objects":5578}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11350","reported_seq":11430,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.945904+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:14.945904+0000","last_peered":"2024-09-10T15:57:14.945904+0000","last_clean":"2024-09-10T15:57:14.945904+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:14.945904+0000","last_undegraded":"2024-09-10T15:57:14.945904+0000","last_fullsized":"2024-09-10T15:57:14.945904+0000","mapping_epoch":36,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38223872,"num_objects":9332,"num_object_clones":0,"num_object_copies":18664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11350,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11121","reported_seq":11248,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.555308+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:14.555308+0000","last_peered":"2024-09-10T15:57:14.555308+0000","last_clean":"2024-09-10T15:57:14.555308+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:14.555308+0000","last_undegraded":"2024-09-10T15:57:14.555308+0000","last_fullsized":"2024-09-10T15:57:14.555308+0000","mapping_epoch":35,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37724160,"num_objects":9210,"num_object_clones":0,"num_object_copies":18420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8192,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11408","reported_seq":15906,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:14.532897+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:14.532897+0000","last_peered":"2024-09-10T15:57:14.532897+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:14.532897+0000","last_undegraded":"2024-09-10T15:57:14.532897+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38744064,"num_objects":9459,"num_object_clones":0,"num_object_copies":18918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1689,"num_objects_unfound":0,"num_objects_dirty":9459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8126,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2181,"num_bytes_recovered":8925184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7770},{"shards":"0,2","objects":1689}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11132","reported_seq":11417,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.521056+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:14.521056+0000","last_peered":"2024-09-10T15:57:14.521056+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:14.521056+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37494784,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7782,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5392},{"shards":"0,2","objects":2848}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11459","reported_seq":11590,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.546385+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:14.546385+0000","last_peered":"2024-09-10T15:57:14.546385+0000","last_clean":"2024-09-10T15:57:14.546385+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:14.546385+0000","last_undegraded":"2024-09-10T15:57:14.546385+0000","last_fullsized":"2024-09-10T15:57:14.546385+0000","mapping_epoch":35,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38727680,"num_objects":9455,"num_object_clones":0,"num_object_copies":18910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8218,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11383","reported_seq":19508,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.581172+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:14.581172+0000","last_peered":"2024-09-10T15:57:14.581172+0000","last_clean":"2024-09-10T15:57:14.581172+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:14.581172+0000","last_undegraded":"2024-09-10T15:57:14.581172+0000","last_fullsized":"2024-09-10T15:57:14.581172+0000","mapping_epoch":45,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38240256,"num_objects":9336,"num_object_clones":0,"num_object_copies":18672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9360,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11269","reported_seq":11582,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.520197+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:14.520197+0000","last_peered":"2024-09-10T15:57:14.520197+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:14.520197+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38129664,"num_objects":9309,"num_object_clones":0,"num_object_copies":18618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3772,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8133,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5539},{"shards":"0,2","objects":2887}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11382","reported_seq":11577,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.940136+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:14.940136+0000","last_peered":"2024-09-10T15:57:14.940136+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:14.940136+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514688,"num_objects":9403,"num_object_clones":0,"num_object_copies":18806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3799,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9356,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2845},{"shards":"2","objects":5606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11235","reported_seq":11428,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:15.026297+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:15.026297+0000","last_peered":"2024-09-10T15:57:15.026297+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:15.026297+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37769216,"num_objects":9221,"num_object_clones":0,"num_object_copies":18442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3798,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9104,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2824},{"shards":"2","objects":5425}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11419","reported_seq":11779,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.524350+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:14.524350+0000","last_peered":"2024-09-10T15:57:14.524350+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:14.524350+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38645760,"num_objects":9435,"num_object_clones":0,"num_object_copies":18870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8611,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5718},{"shards":"0,2","objects":2842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11415","reported_seq":11757,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.423473+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:14.423473+0000","last_peered":"2024-09-10T15:57:14.423473+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:14.423473+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38105088,"num_objects":9303,"num_object_clones":0,"num_object_copies":18606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7833,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5475},{"shards":"0,2","objects":2865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11341","reported_seq":11645,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.522084+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:14.522084+0000","last_peered":"2024-09-10T15:57:14.522084+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:14.522084+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38604800,"num_objects":9425,"num_object_clones":0,"num_object_copies":18850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3797,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7947,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5630},{"shards":"0,2","objects":2921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11556","reported_seq":11863,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.473201+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:14.473201+0000","last_peered":"2024-09-10T15:57:14.473201+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:14.473201+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39084515,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8375,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5624},{"shards":"0,2","objects":3002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":613782011,"num_objects":149851,"num_object_clones":0,"num_object_copies":299702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38035,"num_objects_misplaced":1689,"num_objects_unfound":0,"num_objects_dirty":149851,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143199,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6923,"num_bytes_recovered":25284608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1293316096,"data_stored":1293232079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626482,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594012,"kb_used_data":327752,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93777828,"statfs":{"total":96636764160,"available":96028495872,"internally_reserved":0,"allocated":335618048,"data_stored":335282889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272625326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149765,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859928,"kb_used_data":513152,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93511912,"statfs":{"total":96636764160,"available":95756197888,"internally_reserved":0,"allocated":525467648,"data_stored":525201595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355069613},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,0,2,2,0,0,0,11],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704984,"kb_used_data":424832,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93666856,"statfs":{"total":96636764160,"available":95914860544,"internally_reserved":0,"allocated":435027968,"data_stored":434757843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":286846634},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":524357632,"data_stored":524354019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335077376,"data_stored":335004657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":433881088,"data_stored":433873403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:18.353 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:18.942 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:18.943 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:18.966 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-10T15:57:18.514786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613782030,"num_objects":149853,"num_object_clones":0,"num_object_copies":299706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38035,"num_objects_misplaced":1689,"num_objects_unfound":0,"num_objects_dirty":149853,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143201,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6925,"num_bytes_recovered":25284627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2158924,"kb_used_data":1265736,"kb_used_omap":12,"kb_used_meta":893107,"kb_avail":280956596,"statfs":{"total":289910292480,"available":287699554304,"internally_reserved":0,"allocated":1296113664,"data_stored":1295242327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":914541573},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,2,7,8,3,0,0,11],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17874944,"num_objects":-4364,"num_object_clones":0,"num_object_copies":-8728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":-245,"num_objects_unfound":0,"num_objects_dirty":-4364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4364,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004104"},"pg_stats":[{"pgid":"2.f","version":"46'11350","reported_seq":11629,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.504191+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:14.504191+0000","last_peered":"2024-09-10T15:57:14.504191+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:14.504191+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38584320,"num_objects":9420,"num_object_clones":0,"num_object_copies":18840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3872,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11350,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5548},{"shards":"0,2","objects":3008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11567","reported_seq":11674,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.943139+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:14.943139+0000","last_peered":"2024-09-10T15:57:14.943139+0000","last_clean":"2024-09-10T15:57:14.943139+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:14.943139+0000","last_undegraded":"2024-09-10T15:57:14.943139+0000","last_fullsized":"2024-09-10T15:57:14.943139+0000","mapping_epoch":36,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38940672,"num_objects":9507,"num_object_clones":0,"num_object_copies":19014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9903,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11291","reported_seq":11498,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.867033+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:14.867033+0000","last_peered":"2024-09-10T15:57:14.867033+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:14.867033+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248472,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9559,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2826},{"shards":"2","objects":5578}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11350","reported_seq":11430,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.945904+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:14.945904+0000","last_peered":"2024-09-10T15:57:14.945904+0000","last_clean":"2024-09-10T15:57:14.945904+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:14.945904+0000","last_undegraded":"2024-09-10T15:57:14.945904+0000","last_fullsized":"2024-09-10T15:57:14.945904+0000","mapping_epoch":36,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38223872,"num_objects":9332,"num_object_clones":0,"num_object_copies":18664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11350,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11121","reported_seq":11248,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.555308+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:14.555308+0000","last_peered":"2024-09-10T15:57:14.555308+0000","last_clean":"2024-09-10T15:57:14.555308+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:14.555308+0000","last_undegraded":"2024-09-10T15:57:14.555308+0000","last_fullsized":"2024-09-10T15:57:14.555308+0000","mapping_epoch":35,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37724160,"num_objects":9210,"num_object_clones":0,"num_object_copies":18420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8192,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11408","reported_seq":15906,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:14.532897+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:14.532897+0000","last_peered":"2024-09-10T15:57:14.532897+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:14.532897+0000","last_undegraded":"2024-09-10T15:57:14.532897+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38744064,"num_objects":9459,"num_object_clones":0,"num_object_copies":18918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1689,"num_objects_unfound":0,"num_objects_dirty":9459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8126,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2181,"num_bytes_recovered":8925184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7770},{"shards":"0,2","objects":1689}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11132","reported_seq":11417,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.521056+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:14.521056+0000","last_peered":"2024-09-10T15:57:14.521056+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:14.521056+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37494784,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7782,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5392},{"shards":"0,2","objects":2848}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11459","reported_seq":11590,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.546385+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:14.546385+0000","last_peered":"2024-09-10T15:57:14.546385+0000","last_clean":"2024-09-10T15:57:14.546385+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:14.546385+0000","last_undegraded":"2024-09-10T15:57:14.546385+0000","last_fullsized":"2024-09-10T15:57:14.546385+0000","mapping_epoch":35,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38727680,"num_objects":9455,"num_object_clones":0,"num_object_copies":18910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8218,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11383","reported_seq":19508,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.581172+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:14.581172+0000","last_peered":"2024-09-10T15:57:14.581172+0000","last_clean":"2024-09-10T15:57:14.581172+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:14.581172+0000","last_undegraded":"2024-09-10T15:57:14.581172+0000","last_fullsized":"2024-09-10T15:57:14.581172+0000","mapping_epoch":45,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38240256,"num_objects":9336,"num_object_clones":0,"num_object_copies":18672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9360,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11269","reported_seq":11582,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.520197+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:14.520197+0000","last_peered":"2024-09-10T15:57:14.520197+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:14.520197+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38129664,"num_objects":9309,"num_object_clones":0,"num_object_copies":18618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3772,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8133,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5539},{"shards":"0,2","objects":2887}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11382","reported_seq":11577,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.940136+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:14.940136+0000","last_peered":"2024-09-10T15:57:14.940136+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:14.940136+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514688,"num_objects":9403,"num_object_clones":0,"num_object_copies":18806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3799,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9356,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2845},{"shards":"2","objects":5606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11235","reported_seq":11428,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:15.026297+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:15.026297+0000","last_peered":"2024-09-10T15:57:15.026297+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:15.026297+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37769216,"num_objects":9221,"num_object_clones":0,"num_object_copies":18442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3798,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9104,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2824},{"shards":"2","objects":5425}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11419","reported_seq":11779,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.524350+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:14.524350+0000","last_peered":"2024-09-10T15:57:14.524350+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:14.524350+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38645760,"num_objects":9435,"num_object_clones":0,"num_object_copies":18870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8611,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5718},{"shards":"0,2","objects":2842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11415","reported_seq":11757,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.423473+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:14.423473+0000","last_peered":"2024-09-10T15:57:14.423473+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:14.423473+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38105088,"num_objects":9303,"num_object_clones":0,"num_object_copies":18606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7833,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5475},{"shards":"0,2","objects":2865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11341","reported_seq":11645,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.522084+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:14.522084+0000","last_peered":"2024-09-10T15:57:14.522084+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:14.522084+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38604800,"num_objects":9425,"num_object_clones":0,"num_object_copies":18850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3797,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7947,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5630},{"shards":"0,2","objects":2921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11556","reported_seq":11863,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.473201+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:14.473201+0000","last_peered":"2024-09-10T15:57:14.473201+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:14.473201+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39084515,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8375,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5624},{"shards":"0,2","objects":3002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":613782011,"num_objects":149851,"num_object_clones":0,"num_object_copies":299702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38035,"num_objects_misplaced":1689,"num_objects_unfound":0,"num_objects_dirty":149851,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143199,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6923,"num_bytes_recovered":25284608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1293316096,"data_stored":1293232079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626482,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594012,"kb_used_data":327752,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93777828,"statfs":{"total":96636764160,"available":96028495872,"internally_reserved":0,"allocated":335618048,"data_stored":335282889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272625326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149765,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859928,"kb_used_data":513152,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93511912,"statfs":{"total":96636764160,"available":95756197888,"internally_reserved":0,"allocated":525467648,"data_stored":525201595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355069613},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,0,2,2,0,0,0,11],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704984,"kb_used_data":424832,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93666856,"statfs":{"total":96636764160,"available":95914860544,"internally_reserved":0,"allocated":435027968,"data_stored":434757843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":286846634},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":524357632,"data_stored":524354019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335077376,"data_stored":335004657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":433881088,"data_stored":433873403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:18.969 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:57:18.969 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'11350", 'reported_seq': 11629, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:14.504191+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:57:14.504191+0000', 'last_peered': '2024-09-10T15:57:14.504191+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:57:14.504191+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'1300", 'ondisk_log_start': "37'1300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38584320, 'num_objects': 9420, 'num_object_clones': 0, 'num_object_copies': 18840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3872, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11350, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 101, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5548}, {'shards': '0,2', 'objects': 3008}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:18.969 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:57:18.969 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'11291", 'reported_seq': 11498, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:14.867033+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:57:14.867033+0000', 'last_peered': '2024-09-10T15:57:14.867033+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:57:14.867033+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38248472, 'num_objects': 9339, 'num_object_clones': 0, 'num_object_copies': 18678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3762, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9339, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9559, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2826}, {'shards': '2', 'objects': 5578}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:18.969 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:57:18.970 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'11408", 'reported_seq': 15906, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:57:14.532897+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:57:14.532897+0000', 'last_peered': '2024-09-10T15:57:14.532897+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:57:14.532897+0000', 'last_undegraded': '2024-09-10T15:57:14.532897+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38744064, 'num_objects': 9459, 'num_object_clones': 0, 'num_object_copies': 18918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1689, 'num_objects_unfound': 0, 'num_objects_dirty': 9459, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8126, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2181, 'num_bytes_recovered': 8925184, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7770}, {'shards': '0,2', 'objects': 1689}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:18.970 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:57:18.970 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'11132", 'reported_seq': 11417, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:14.521056+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:57:14.521056+0000', 'last_peered': '2024-09-10T15:57:14.521056+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:57:14.521056+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37494784, 'num_objects': 9154, 'num_object_clones': 0, 'num_object_copies': 18308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3764, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7782, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 73, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5392}, {'shards': '0,2', 'objects': 2848}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:18.970 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:57:18.971 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'11269", 'reported_seq': 11582, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:14.520197+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:57:14.520197+0000', 'last_peered': '2024-09-10T15:57:14.520197+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:57:14.520197+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38129664, 'num_objects': 9309, 'num_object_clones': 0, 'num_object_copies': 18618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3772, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9309, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8133, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 93, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5539}, {'shards': '0,2', 'objects': 2887}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:18.971 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:57:18.971 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'11382", 'reported_seq': 11577, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:14.940136+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:57:14.940136+0000', 'last_peered': '2024-09-10T15:57:14.940136+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:57:14.940136+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'1300", 'ondisk_log_start': "37'1300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38514688, 'num_objects': 9403, 'num_object_clones': 0, 'num_object_copies': 18806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3799, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9403, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9356, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 35, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2845}, {'shards': '2', 'objects': 5606}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:18.971 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:57:18.971 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'11235", 'reported_seq': 11428, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:15.026297+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:57:15.026297+0000', 'last_peered': '2024-09-10T15:57:15.026297+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:57:15.026297+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37769216, 'num_objects': 9221, 'num_object_clones': 0, 'num_object_copies': 18442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3798, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9221, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9104, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 33, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2824}, {'shards': '2', 'objects': 5425}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:18.971 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:57:18.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'11419", 'reported_seq': 11779, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:14.524350+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:57:14.524350+0000', 'last_peered': '2024-09-10T15:57:14.524350+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:57:14.524350+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38645760, 'num_objects': 9435, 'num_object_clones': 0, 'num_object_copies': 18870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3718, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9435, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8611, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 116, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5718}, {'shards': '0,2', 'objects': 2842}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:18.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:57:18.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'11415", 'reported_seq': 11757, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:14.423473+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:57:14.423473+0000', 'last_peered': '2024-09-10T15:57:14.423473+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:57:14.423473+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38105088, 'num_objects': 9303, 'num_object_clones': 0, 'num_object_copies': 18606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3831, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9303, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7833, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 89, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5475}, {'shards': '0,2', 'objects': 2865}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:18.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:57:18.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'11341", 'reported_seq': 11645, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:14.522084+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:57:14.522084+0000', 'last_peered': '2024-09-10T15:57:14.522084+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:57:14.522084+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'1300", 'ondisk_log_start': "37'1300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38604800, 'num_objects': 9425, 'num_object_clones': 0, 'num_object_copies': 18850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3797, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9425, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7947, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5630}, {'shards': '0,2', 'objects': 2921}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:18.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:57:18.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'11556", 'reported_seq': 11863, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:14.473201+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:57:14.473201+0000', 'last_peered': '2024-09-10T15:57:14.473201+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:57:14.473201+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39084515, 'num_objects': 9543, 'num_object_clones': 0, 'num_object_copies': 19086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3922, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9543, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8375, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 84, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5624}, {'shards': '0,2', 'objects': 3002}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:18.973 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:57:19.689 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:19.689 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38035/299706 objects degraded (12.691%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":365,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":149853,"data_bytes":613782030,"bytes_used":2210738176,"bytes_avail":287699554304,"bytes_total":289910292480,"degraded_objects":38035,"degraded_total":299706,"degraded_ratio":0.12690770288215786,"misplaced_objects":1689,"misplaced_total":299706,"misplaced_ratio":0.0056355228123561082,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":83598,"recovering_keys_per_sec":0,"num_objects_recovered":442,"num_bytes_recovered":1003520,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":363},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T15:56:24.496337+0000","services":{}},"progress_events":{}} 2024-09-10T15:57:19.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:57:19.718 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:20.269 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:20.270 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:20.294 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-10T15:57:18.514786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613782030,"num_objects":149853,"num_object_clones":0,"num_object_copies":299706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38035,"num_objects_misplaced":1689,"num_objects_unfound":0,"num_objects_dirty":149853,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143201,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6925,"num_bytes_recovered":25284627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2158924,"kb_used_data":1265736,"kb_used_omap":12,"kb_used_meta":893107,"kb_avail":280956596,"statfs":{"total":289910292480,"available":287699554304,"internally_reserved":0,"allocated":1296113664,"data_stored":1295242327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":914541573},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,2,7,8,3,0,0,11],"upper_bound":1024},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17874944,"num_objects":-4364,"num_object_clones":0,"num_object_copies":-8728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":-245,"num_objects_unfound":0,"num_objects_dirty":-4364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4364,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004104"},"pg_stats":[{"pgid":"2.f","version":"46'11350","reported_seq":11629,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.504191+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:14.504191+0000","last_peered":"2024-09-10T15:57:14.504191+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:14.504191+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38584320,"num_objects":9420,"num_object_clones":0,"num_object_copies":18840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3872,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11350,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5548},{"shards":"0,2","objects":3008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11567","reported_seq":11674,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.943139+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:14.943139+0000","last_peered":"2024-09-10T15:57:14.943139+0000","last_clean":"2024-09-10T15:57:14.943139+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:14.943139+0000","last_undegraded":"2024-09-10T15:57:14.943139+0000","last_fullsized":"2024-09-10T15:57:14.943139+0000","mapping_epoch":36,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38940672,"num_objects":9507,"num_object_clones":0,"num_object_copies":19014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9903,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11291","reported_seq":11498,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.867033+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:14.867033+0000","last_peered":"2024-09-10T15:57:14.867033+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:14.867033+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248472,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9559,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2826},{"shards":"2","objects":5578}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11350","reported_seq":11430,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.945904+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:14.945904+0000","last_peered":"2024-09-10T15:57:14.945904+0000","last_clean":"2024-09-10T15:57:14.945904+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:14.945904+0000","last_undegraded":"2024-09-10T15:57:14.945904+0000","last_fullsized":"2024-09-10T15:57:14.945904+0000","mapping_epoch":36,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38223872,"num_objects":9332,"num_object_clones":0,"num_object_copies":18664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11350,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11121","reported_seq":11248,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.555308+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:14.555308+0000","last_peered":"2024-09-10T15:57:14.555308+0000","last_clean":"2024-09-10T15:57:14.555308+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:14.555308+0000","last_undegraded":"2024-09-10T15:57:14.555308+0000","last_fullsized":"2024-09-10T15:57:14.555308+0000","mapping_epoch":35,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37724160,"num_objects":9210,"num_object_clones":0,"num_object_copies":18420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8192,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11408","reported_seq":15906,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:14.532897+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:14.532897+0000","last_peered":"2024-09-10T15:57:14.532897+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:14.532897+0000","last_undegraded":"2024-09-10T15:57:14.532897+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38744064,"num_objects":9459,"num_object_clones":0,"num_object_copies":18918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1689,"num_objects_unfound":0,"num_objects_dirty":9459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8126,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2181,"num_bytes_recovered":8925184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7770},{"shards":"0,2","objects":1689}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11132","reported_seq":11417,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.521056+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:14.521056+0000","last_peered":"2024-09-10T15:57:14.521056+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:14.521056+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37494784,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7782,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5392},{"shards":"0,2","objects":2848}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11459","reported_seq":11590,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.546385+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:14.546385+0000","last_peered":"2024-09-10T15:57:14.546385+0000","last_clean":"2024-09-10T15:57:14.546385+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:14.546385+0000","last_undegraded":"2024-09-10T15:57:14.546385+0000","last_fullsized":"2024-09-10T15:57:14.546385+0000","mapping_epoch":35,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38727680,"num_objects":9455,"num_object_clones":0,"num_object_copies":18910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8218,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671948+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671948+0000","last_peered":"2024-09-10T15:55:20.671948+0000","last_clean":"2024-09-10T15:55:20.671948+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671948+0000","last_undegraded":"2024-09-10T15:55:20.671948+0000","last_fullsized":"2024-09-10T15:55:20.671948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11383","reported_seq":19508,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:14.581172+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:14.581172+0000","last_peered":"2024-09-10T15:57:14.581172+0000","last_clean":"2024-09-10T15:57:14.581172+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:14.581172+0000","last_undegraded":"2024-09-10T15:57:14.581172+0000","last_fullsized":"2024-09-10T15:57:14.581172+0000","mapping_epoch":45,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38240256,"num_objects":9336,"num_object_clones":0,"num_object_copies":18672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9360,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11269","reported_seq":11582,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.520197+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:14.520197+0000","last_peered":"2024-09-10T15:57:14.520197+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:14.520197+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38129664,"num_objects":9309,"num_object_clones":0,"num_object_copies":18618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3772,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8133,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5539},{"shards":"0,2","objects":2887}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11382","reported_seq":11577,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.940136+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:14.940136+0000","last_peered":"2024-09-10T15:57:14.940136+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:14.940136+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514688,"num_objects":9403,"num_object_clones":0,"num_object_copies":18806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3799,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9356,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2845},{"shards":"2","objects":5606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11235","reported_seq":11428,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:15.026297+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:15.026297+0000","last_peered":"2024-09-10T15:57:15.026297+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:15.026297+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37769216,"num_objects":9221,"num_object_clones":0,"num_object_copies":18442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3798,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9104,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2824},{"shards":"2","objects":5425}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11419","reported_seq":11779,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.524350+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:14.524350+0000","last_peered":"2024-09-10T15:57:14.524350+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:14.524350+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38645760,"num_objects":9435,"num_object_clones":0,"num_object_copies":18870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8611,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5718},{"shards":"0,2","objects":2842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11415","reported_seq":11757,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.423473+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:14.423473+0000","last_peered":"2024-09-10T15:57:14.423473+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:14.423473+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38105088,"num_objects":9303,"num_object_clones":0,"num_object_copies":18606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3831,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7833,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5475},{"shards":"0,2","objects":2865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11341","reported_seq":11645,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.522084+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:14.522084+0000","last_peered":"2024-09-10T15:57:14.522084+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:14.522084+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38604800,"num_objects":9425,"num_object_clones":0,"num_object_copies":18850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3797,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7947,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5630},{"shards":"0,2","objects":2921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11556","reported_seq":11863,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:14.473201+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:14.473201+0000","last_peered":"2024-09-10T15:57:14.473201+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:14.473201+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39084515,"num_objects":9543,"num_object_clones":0,"num_object_copies":19086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8375,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5624},{"shards":"0,2","objects":3002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":613782011,"num_objects":149851,"num_object_clones":0,"num_object_copies":299702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38035,"num_objects_misplaced":1689,"num_objects_unfound":0,"num_objects_dirty":149851,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143199,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6923,"num_bytes_recovered":25284608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1293316096,"data_stored":1293232079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626482,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594012,"kb_used_data":327752,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93777828,"statfs":{"total":96636764160,"available":96028495872,"internally_reserved":0,"allocated":335618048,"data_stored":335282889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272625326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149765,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859928,"kb_used_data":513152,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93511912,"statfs":{"total":96636764160,"available":95756197888,"internally_reserved":0,"allocated":525467648,"data_stored":525201595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355069613},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,0,2,2,0,0,0,11],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704984,"kb_used_data":424832,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93666856,"statfs":{"total":96636764160,"available":95914860544,"internally_reserved":0,"allocated":435027968,"data_stored":434757843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":286846634},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":524357632,"data_stored":524354019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":335077376,"data_stored":335004657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":433881088,"data_stored":433873403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:23.296 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:23.839 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:23.862 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-10T15:57:22.515744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604672526,"num_objects":147629,"num_object_clones":0,"num_object_copies":295258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37934,"num_objects_misplaced":1564,"num_objects_unfound":0,"num_objects_dirty":147629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145425,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7151,"num_bytes_recovered":25796627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2147080,"kb_used_data":1253908,"kb_used_omap":12,"kb_used_meta":893107,"kb_avail":280968440,"statfs":{"total":289910292480,"available":287711682560,"internally_reserved":0,"allocated":1284001792,"data_stored":1283130455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":914541573},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,5,6,8,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17985536,"num_objects":-4391,"num_object_clones":0,"num_object_copies":-8782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-222,"num_objects_misplaced":-242,"num_objects_unfound":0,"num_objects_dirty":-4391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4391,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":464,"num_bytes_recovered":991232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003714"},"pg_stats":[{"pgid":"2.f","version":"46'11485","reported_seq":11784,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.524572+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:19.524572+0000","last_peered":"2024-09-10T15:57:19.524572+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:19.524572+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38031360,"num_objects":9285,"num_object_clones":0,"num_object_copies":18570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11485,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5423},{"shards":"0,2","objects":2873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11713","reported_seq":11821,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:19.989047+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:19.989047+0000","last_peered":"2024-09-10T15:57:19.989047+0000","last_clean":"2024-09-10T15:57:19.989047+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:19.989047+0000","last_undegraded":"2024-09-10T15:57:19.989047+0000","last_fullsized":"2024-09-10T15:57:19.989047+0000","mapping_epoch":36,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38342656,"num_objects":9361,"num_object_clones":0,"num_object_copies":18722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10049,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11434","reported_seq":11647,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.899513+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:19.899513+0000","last_peered":"2024-09-10T15:57:19.899513+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:19.899513+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37662744,"num_objects":9196,"num_object_clones":0,"num_object_copies":18392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3759,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9196,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9702,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2683},{"shards":"2","objects":5438}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11501","reported_seq":11581,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:20.011056+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:20.011056+0000","last_peered":"2024-09-10T15:57:20.011056+0000","last_clean":"2024-09-10T15:57:20.011056+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:20.011056+0000","last_undegraded":"2024-09-10T15:57:20.011056+0000","last_fullsized":"2024-09-10T15:57:20.011056+0000","mapping_epoch":36,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37605376,"num_objects":9181,"num_object_clones":0,"num_object_copies":18362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11501,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11282","reported_seq":11409,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:19.594610+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:19.594610+0000","last_peered":"2024-09-10T15:57:19.594610+0000","last_clean":"2024-09-10T15:57:19.594610+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:19.594610+0000","last_undegraded":"2024-09-10T15:57:19.594610+0000","last_fullsized":"2024-09-10T15:57:19.594610+0000","mapping_epoch":35,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8353,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11534","reported_seq":16282,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:19.562394+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:19.562394+0000","last_peered":"2024-09-10T15:57:19.562394+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:19.562394+0000","last_undegraded":"2024-09-10T15:57:19.562394+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38227968,"num_objects":9333,"num_object_clones":0,"num_object_copies":18666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1564,"num_objects_unfound":0,"num_objects_dirty":9333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8252,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2306,"num_bytes_recovered":9437184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7769},{"shards":"0,2","objects":1564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11263","reported_seq":11572,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.426220+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:19.426220+0000","last_peered":"2024-09-10T15:57:19.426220+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:19.426220+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36958208,"num_objects":9023,"num_object_clones":0,"num_object_copies":18046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3752,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7913,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5273},{"shards":"0,2","objects":2717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11575","reported_seq":11706,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:19.309004+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:19.309004+0000","last_peered":"2024-09-10T15:57:19.309004+0000","last_clean":"2024-09-10T15:57:19.309004+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:19.309004+0000","last_undegraded":"2024-09-10T15:57:19.309004+0000","last_fullsized":"2024-09-10T15:57:19.309004+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38252544,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8334,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11552","reported_seq":19679,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:19.578533+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:19.578533+0000","last_peered":"2024-09-10T15:57:19.578533+0000","last_clean":"2024-09-10T15:57:19.578533+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:19.578533+0000","last_undegraded":"2024-09-10T15:57:19.578533+0000","last_fullsized":"2024-09-10T15:57:19.578533+0000","mapping_epoch":45,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37548032,"num_objects":9167,"num_object_clones":0,"num_object_copies":18334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9529,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11422","reported_seq":11778,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.552831+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:19.552831+0000","last_peered":"2024-09-10T15:57:19.552831+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:19.552831+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37502976,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8286,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5408},{"shards":"0,2","objects":2734}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11512","reported_seq":11719,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:20.000117+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:20.000117+0000","last_peered":"2024-09-10T15:57:20.000117+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:20.000117+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37982208,"num_objects":9273,"num_object_clones":0,"num_object_copies":18546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9486,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2715},{"shards":"2","objects":5482}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11371","reported_seq":11569,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.975424+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:19.975424+0000","last_peered":"2024-09-10T15:57:19.975424+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:19.975424+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37212160,"num_objects":9085,"num_object_clones":0,"num_object_copies":18170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9240,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2688},{"shards":"2","objects":5292}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11553","reported_seq":11933,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.590666+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:19.590666+0000","last_peered":"2024-09-10T15:57:19.590666+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:19.590666+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38096896,"num_objects":9301,"num_object_clones":0,"num_object_copies":18602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5594},{"shards":"0,2","objects":2708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11540","reported_seq":11908,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.544934+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:19.544934+0000","last_peered":"2024-09-10T15:57:19.544934+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:19.544934+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37593088,"num_objects":9178,"num_object_clones":0,"num_object_copies":18356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7958,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5363},{"shards":"0,2","objects":2740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11476","reported_seq":11804,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.485323+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:19.485323+0000","last_peered":"2024-09-10T15:57:19.485323+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:19.485323+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38051840,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8082,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5507},{"shards":"0,2","objects":2786}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11689","reported_seq":12015,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.408098+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:19.408098+0000","last_peered":"2024-09-10T15:57:19.408098+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:19.408098+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38539747,"num_objects":9410,"num_object_clones":0,"num_object_copies":18820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8508,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5501},{"shards":"0,2","objects":2869}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604672507,"num_objects":147627,"num_object_clones":0,"num_object_copies":295254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37934,"num_objects_misplaced":1564,"num_objects_unfound":0,"num_objects_dirty":147627,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145423,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7149,"num_bytes_recovered":25796608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1281134592,"data_stored":1281050575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626483,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601632,"kb_used_data":327056,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93770208,"statfs":{"total":96636764160,"available":96020692992,"internally_reserved":0,"allocated":334905344,"data_stored":334570185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":281145006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149766,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":840900,"kb_used_data":506608,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93530940,"statfs":{"total":96636764160,"available":95775682560,"internally_reserved":0,"allocated":518766592,"data_stored":518500539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342290093},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,2,4,3,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704548,"kb_used_data":420244,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93667292,"statfs":{"total":96636764160,"available":95915307008,"internally_reserved":0,"allocated":430329856,"data_stored":430059731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":517627904,"data_stored":517624291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334376960,"data_stored":334304241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":429129728,"data_stored":429122043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:23.863 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:24.415 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:24.416 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:24.440 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-10T15:57:22.515744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604672526,"num_objects":147629,"num_object_clones":0,"num_object_copies":295258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37934,"num_objects_misplaced":1564,"num_objects_unfound":0,"num_objects_dirty":147629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145425,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7151,"num_bytes_recovered":25796627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2147080,"kb_used_data":1253908,"kb_used_omap":12,"kb_used_meta":893107,"kb_avail":280968440,"statfs":{"total":289910292480,"available":287711682560,"internally_reserved":0,"allocated":1284001792,"data_stored":1283130455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":914541573},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,5,6,8,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17985536,"num_objects":-4391,"num_object_clones":0,"num_object_copies":-8782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-222,"num_objects_misplaced":-242,"num_objects_unfound":0,"num_objects_dirty":-4391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4391,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":464,"num_bytes_recovered":991232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003714"},"pg_stats":[{"pgid":"2.f","version":"46'11485","reported_seq":11784,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.524572+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:19.524572+0000","last_peered":"2024-09-10T15:57:19.524572+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:19.524572+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38031360,"num_objects":9285,"num_object_clones":0,"num_object_copies":18570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11485,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5423},{"shards":"0,2","objects":2873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11713","reported_seq":11821,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:19.989047+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:19.989047+0000","last_peered":"2024-09-10T15:57:19.989047+0000","last_clean":"2024-09-10T15:57:19.989047+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:19.989047+0000","last_undegraded":"2024-09-10T15:57:19.989047+0000","last_fullsized":"2024-09-10T15:57:19.989047+0000","mapping_epoch":36,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38342656,"num_objects":9361,"num_object_clones":0,"num_object_copies":18722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10049,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11434","reported_seq":11647,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.899513+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:19.899513+0000","last_peered":"2024-09-10T15:57:19.899513+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:19.899513+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37662744,"num_objects":9196,"num_object_clones":0,"num_object_copies":18392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3759,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9196,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9702,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2683},{"shards":"2","objects":5438}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11501","reported_seq":11581,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:20.011056+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:20.011056+0000","last_peered":"2024-09-10T15:57:20.011056+0000","last_clean":"2024-09-10T15:57:20.011056+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:20.011056+0000","last_undegraded":"2024-09-10T15:57:20.011056+0000","last_fullsized":"2024-09-10T15:57:20.011056+0000","mapping_epoch":36,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37605376,"num_objects":9181,"num_object_clones":0,"num_object_copies":18362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11501,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11282","reported_seq":11409,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:19.594610+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:19.594610+0000","last_peered":"2024-09-10T15:57:19.594610+0000","last_clean":"2024-09-10T15:57:19.594610+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:19.594610+0000","last_undegraded":"2024-09-10T15:57:19.594610+0000","last_fullsized":"2024-09-10T15:57:19.594610+0000","mapping_epoch":35,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8353,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11534","reported_seq":16282,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:19.562394+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:19.562394+0000","last_peered":"2024-09-10T15:57:19.562394+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:19.562394+0000","last_undegraded":"2024-09-10T15:57:19.562394+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38227968,"num_objects":9333,"num_object_clones":0,"num_object_copies":18666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1564,"num_objects_unfound":0,"num_objects_dirty":9333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8252,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2306,"num_bytes_recovered":9437184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7769},{"shards":"0,2","objects":1564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11263","reported_seq":11572,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.426220+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:19.426220+0000","last_peered":"2024-09-10T15:57:19.426220+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:19.426220+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36958208,"num_objects":9023,"num_object_clones":0,"num_object_copies":18046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3752,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7913,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5273},{"shards":"0,2","objects":2717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11575","reported_seq":11706,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:19.309004+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:19.309004+0000","last_peered":"2024-09-10T15:57:19.309004+0000","last_clean":"2024-09-10T15:57:19.309004+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:19.309004+0000","last_undegraded":"2024-09-10T15:57:19.309004+0000","last_fullsized":"2024-09-10T15:57:19.309004+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38252544,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8334,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11552","reported_seq":19679,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:19.578533+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:19.578533+0000","last_peered":"2024-09-10T15:57:19.578533+0000","last_clean":"2024-09-10T15:57:19.578533+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:19.578533+0000","last_undegraded":"2024-09-10T15:57:19.578533+0000","last_fullsized":"2024-09-10T15:57:19.578533+0000","mapping_epoch":45,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37548032,"num_objects":9167,"num_object_clones":0,"num_object_copies":18334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9529,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11422","reported_seq":11778,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.552831+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:19.552831+0000","last_peered":"2024-09-10T15:57:19.552831+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:19.552831+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37502976,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8286,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5408},{"shards":"0,2","objects":2734}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11512","reported_seq":11719,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:20.000117+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:20.000117+0000","last_peered":"2024-09-10T15:57:20.000117+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:20.000117+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37982208,"num_objects":9273,"num_object_clones":0,"num_object_copies":18546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9486,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2715},{"shards":"2","objects":5482}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11371","reported_seq":11569,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.975424+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:19.975424+0000","last_peered":"2024-09-10T15:57:19.975424+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:19.975424+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37212160,"num_objects":9085,"num_object_clones":0,"num_object_copies":18170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9240,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2688},{"shards":"2","objects":5292}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11553","reported_seq":11933,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.590666+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:19.590666+0000","last_peered":"2024-09-10T15:57:19.590666+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:19.590666+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38096896,"num_objects":9301,"num_object_clones":0,"num_object_copies":18602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5594},{"shards":"0,2","objects":2708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11540","reported_seq":11908,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.544934+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:19.544934+0000","last_peered":"2024-09-10T15:57:19.544934+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:19.544934+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37593088,"num_objects":9178,"num_object_clones":0,"num_object_copies":18356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7958,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5363},{"shards":"0,2","objects":2740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11476","reported_seq":11804,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.485323+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:19.485323+0000","last_peered":"2024-09-10T15:57:19.485323+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:19.485323+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38051840,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8082,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5507},{"shards":"0,2","objects":2786}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11689","reported_seq":12015,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.408098+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:19.408098+0000","last_peered":"2024-09-10T15:57:19.408098+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:19.408098+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38539747,"num_objects":9410,"num_object_clones":0,"num_object_copies":18820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8508,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5501},{"shards":"0,2","objects":2869}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604672507,"num_objects":147627,"num_object_clones":0,"num_object_copies":295254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37934,"num_objects_misplaced":1564,"num_objects_unfound":0,"num_objects_dirty":147627,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145423,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7149,"num_bytes_recovered":25796608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1281134592,"data_stored":1281050575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626483,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601632,"kb_used_data":327056,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93770208,"statfs":{"total":96636764160,"available":96020692992,"internally_reserved":0,"allocated":334905344,"data_stored":334570185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":281145006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149766,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":840900,"kb_used_data":506608,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93530940,"statfs":{"total":96636764160,"available":95775682560,"internally_reserved":0,"allocated":518766592,"data_stored":518500539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342290093},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,2,4,3,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704548,"kb_used_data":420244,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93667292,"statfs":{"total":96636764160,"available":95915307008,"internally_reserved":0,"allocated":430329856,"data_stored":430059731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":517627904,"data_stored":517624291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334376960,"data_stored":334304241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":429129728,"data_stored":429122043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:24.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:57:24.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'11485", 'reported_seq': 11784, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:19.524572+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:57:19.524572+0000', 'last_peered': '2024-09-10T15:57:19.524572+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:57:19.524572+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38031360, 'num_objects': 9285, 'num_object_clones': 0, 'num_object_copies': 18570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3862, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9285, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11485, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 111, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5423}, {'shards': '0,2', 'objects': 2873}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:24.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:57:24.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'11434", 'reported_seq': 11647, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:19.899513+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:57:19.899513+0000', 'last_peered': '2024-09-10T15:57:19.899513+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:57:19.899513+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37662744, 'num_objects': 9196, 'num_object_clones': 0, 'num_object_copies': 18392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3759, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9196, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9702, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2683}, {'shards': '2', 'objects': 5438}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:24.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:57:24.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'11534", 'reported_seq': 16282, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:57:19.562394+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:57:19.562394+0000', 'last_peered': '2024-09-10T15:57:19.562394+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:57:19.562394+0000', 'last_undegraded': '2024-09-10T15:57:19.562394+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38227968, 'num_objects': 9333, 'num_object_clones': 0, 'num_object_copies': 18666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1564, 'num_objects_unfound': 0, 'num_objects_dirty': 9333, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8252, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2306, 'num_bytes_recovered': 9437184, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7769}, {'shards': '0,2', 'objects': 1564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:24.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:57:24.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'11263", 'reported_seq': 11572, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:19.426220+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:57:19.426220+0000', 'last_peered': '2024-09-10T15:57:19.426220+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:57:19.426220+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36958208, 'num_objects': 9023, 'num_object_clones': 0, 'num_object_copies': 18046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3752, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9023, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7913, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5273}, {'shards': '0,2', 'objects': 2717}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:24.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:57:24.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'11422", 'reported_seq': 11778, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:19.552831+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:57:19.552831+0000', 'last_peered': '2024-09-10T15:57:19.552831+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:57:19.552831+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37502976, 'num_objects': 9156, 'num_object_clones': 0, 'num_object_copies': 18312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3750, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8286, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 115, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5408}, {'shards': '0,2', 'objects': 2734}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:24.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:57:24.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'11512", 'reported_seq': 11719, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:20.000117+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:57:20.000117+0000', 'last_peered': '2024-09-10T15:57:20.000117+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:57:20.000117+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37982208, 'num_objects': 9273, 'num_object_clones': 0, 'num_object_copies': 18546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3793, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9273, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9486, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2715}, {'shards': '2', 'objects': 5482}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:24.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:57:24.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'11371", 'reported_seq': 11569, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:19.975424+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:57:19.975424+0000', 'last_peered': '2024-09-10T15:57:19.975424+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:57:19.975424+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'1300", 'ondisk_log_start': "37'1300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37212160, 'num_objects': 9085, 'num_object_clones': 0, 'num_object_copies': 18170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3795, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9085, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9240, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2688}, {'shards': '2', 'objects': 5292}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:24.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:57:24.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'11553", 'reported_seq': 11933, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:19.590666+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:57:19.590666+0000', 'last_peered': '2024-09-10T15:57:19.590666+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:57:19.590666+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38096896, 'num_objects': 9301, 'num_object_clones': 0, 'num_object_copies': 18602, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3708, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9301, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8745, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 126, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5594}, {'shards': '0,2', 'objects': 2708}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:24.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:57:24.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'11540", 'reported_seq': 11908, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:19.544934+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:57:19.544934+0000', 'last_peered': '2024-09-10T15:57:19.544934+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:57:19.544934+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37593088, 'num_objects': 9178, 'num_object_clones': 0, 'num_object_copies': 18356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3818, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7958, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 102, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5363}, {'shards': '0,2', 'objects': 2740}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:24.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:57:24.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'11476", 'reported_seq': 11804, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:19.485323+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:57:19.485323+0000', 'last_peered': '2024-09-10T15:57:19.485323+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:57:19.485323+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38051840, 'num_objects': 9290, 'num_object_clones': 0, 'num_object_copies': 18580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3785, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9290, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8082, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 95, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5507}, {'shards': '0,2', 'objects': 2786}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:24.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:57:24.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'11689", 'reported_seq': 12015, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:19.408098+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:57:19.408098+0000', 'last_peered': '2024-09-10T15:57:19.408098+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:57:19.408098+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38539747, 'num_objects': 9410, 'num_object_clones': 0, 'num_object_copies': 18820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3912, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9410, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8508, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 94, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5501}, {'shards': '0,2', 'objects': 2869}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:24.447 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:57:25.125 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:25.125 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37934/295258 objects degraded (12.848%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":371,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":147629,"data_bytes":604672526,"bytes_used":2198609920,"bytes_avail":287711682560,"bytes_total":289910292480,"degraded_objects":37934,"degraded_total":295258,"degraded_ratio":0.12847746716431052,"misplaced_objects":1564,"misplaced_total":295258,"misplaced_ratio":0.005297062230320602,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":82577,"recovering_keys_per_sec":0,"num_objects_recovered":464,"num_bytes_recovered":991232,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":365},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-10T15:56:24.496337+0000","services":{}},"progress_events":{}} 2024-09-10T15:57:25.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:57:25.161 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:25.726 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:25.726 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:25.747 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-10T15:57:24.516358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604672526,"num_objects":147629,"num_object_clones":0,"num_object_copies":295258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37934,"num_objects_misplaced":1564,"num_objects_unfound":0,"num_objects_dirty":147629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145425,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7151,"num_bytes_recovered":25796627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2147080,"kb_used_data":1253908,"kb_used_omap":12,"kb_used_meta":893107,"kb_avail":280968440,"statfs":{"total":289910292480,"available":287711682560,"internally_reserved":0,"allocated":1284001792,"data_stored":1283130455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":914541573},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,5,6,8,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17985536,"num_objects":-4391,"num_object_clones":0,"num_object_copies":-8782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-222,"num_objects_misplaced":-242,"num_objects_unfound":0,"num_objects_dirty":-4391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4391,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":464,"num_bytes_recovered":991232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003613"},"pg_stats":[{"pgid":"2.f","version":"46'11485","reported_seq":11784,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.524572+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:19.524572+0000","last_peered":"2024-09-10T15:57:19.524572+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:19.524572+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38031360,"num_objects":9285,"num_object_clones":0,"num_object_copies":18570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11485,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5423},{"shards":"0,2","objects":2873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11713","reported_seq":11821,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:19.989047+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:19.989047+0000","last_peered":"2024-09-10T15:57:19.989047+0000","last_clean":"2024-09-10T15:57:19.989047+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:19.989047+0000","last_undegraded":"2024-09-10T15:57:19.989047+0000","last_fullsized":"2024-09-10T15:57:19.989047+0000","mapping_epoch":36,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38342656,"num_objects":9361,"num_object_clones":0,"num_object_copies":18722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10049,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11434","reported_seq":11647,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.899513+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:19.899513+0000","last_peered":"2024-09-10T15:57:19.899513+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:19.899513+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37662744,"num_objects":9196,"num_object_clones":0,"num_object_copies":18392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3759,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9196,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9702,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2683},{"shards":"2","objects":5438}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11501","reported_seq":11581,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:20.011056+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:20.011056+0000","last_peered":"2024-09-10T15:57:20.011056+0000","last_clean":"2024-09-10T15:57:20.011056+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:20.011056+0000","last_undegraded":"2024-09-10T15:57:20.011056+0000","last_fullsized":"2024-09-10T15:57:20.011056+0000","mapping_epoch":36,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37605376,"num_objects":9181,"num_object_clones":0,"num_object_copies":18362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11501,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11282","reported_seq":11409,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:19.594610+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:19.594610+0000","last_peered":"2024-09-10T15:57:19.594610+0000","last_clean":"2024-09-10T15:57:19.594610+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:19.594610+0000","last_undegraded":"2024-09-10T15:57:19.594610+0000","last_fullsized":"2024-09-10T15:57:19.594610+0000","mapping_epoch":35,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8353,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11534","reported_seq":16282,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:19.562394+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:19.562394+0000","last_peered":"2024-09-10T15:57:19.562394+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:19.562394+0000","last_undegraded":"2024-09-10T15:57:19.562394+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38227968,"num_objects":9333,"num_object_clones":0,"num_object_copies":18666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1564,"num_objects_unfound":0,"num_objects_dirty":9333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8252,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2306,"num_bytes_recovered":9437184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7769},{"shards":"0,2","objects":1564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11263","reported_seq":11572,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.426220+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:19.426220+0000","last_peered":"2024-09-10T15:57:19.426220+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:19.426220+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36958208,"num_objects":9023,"num_object_clones":0,"num_object_copies":18046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3752,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7913,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5273},{"shards":"0,2","objects":2717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11575","reported_seq":11706,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:19.309004+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:19.309004+0000","last_peered":"2024-09-10T15:57:19.309004+0000","last_clean":"2024-09-10T15:57:19.309004+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:19.309004+0000","last_undegraded":"2024-09-10T15:57:19.309004+0000","last_fullsized":"2024-09-10T15:57:19.309004+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38252544,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8334,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11552","reported_seq":19679,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:19.578533+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:19.578533+0000","last_peered":"2024-09-10T15:57:19.578533+0000","last_clean":"2024-09-10T15:57:19.578533+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:19.578533+0000","last_undegraded":"2024-09-10T15:57:19.578533+0000","last_fullsized":"2024-09-10T15:57:19.578533+0000","mapping_epoch":45,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37548032,"num_objects":9167,"num_object_clones":0,"num_object_copies":18334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9529,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11422","reported_seq":11778,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.552831+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:19.552831+0000","last_peered":"2024-09-10T15:57:19.552831+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:19.552831+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37502976,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8286,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5408},{"shards":"0,2","objects":2734}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11512","reported_seq":11719,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:20.000117+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:20.000117+0000","last_peered":"2024-09-10T15:57:20.000117+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:20.000117+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37982208,"num_objects":9273,"num_object_clones":0,"num_object_copies":18546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9486,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2715},{"shards":"2","objects":5482}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11371","reported_seq":11569,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.975424+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:19.975424+0000","last_peered":"2024-09-10T15:57:19.975424+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:19.975424+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37212160,"num_objects":9085,"num_object_clones":0,"num_object_copies":18170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9240,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2688},{"shards":"2","objects":5292}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11553","reported_seq":11933,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.590666+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:19.590666+0000","last_peered":"2024-09-10T15:57:19.590666+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:19.590666+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38096896,"num_objects":9301,"num_object_clones":0,"num_object_copies":18602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5594},{"shards":"0,2","objects":2708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11540","reported_seq":11908,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.544934+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:19.544934+0000","last_peered":"2024-09-10T15:57:19.544934+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:19.544934+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37593088,"num_objects":9178,"num_object_clones":0,"num_object_copies":18356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7958,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5363},{"shards":"0,2","objects":2740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11476","reported_seq":11804,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.485323+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:19.485323+0000","last_peered":"2024-09-10T15:57:19.485323+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:19.485323+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38051840,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8082,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5507},{"shards":"0,2","objects":2786}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11689","reported_seq":12015,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:19.408098+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:19.408098+0000","last_peered":"2024-09-10T15:57:19.408098+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:19.408098+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38539747,"num_objects":9410,"num_object_clones":0,"num_object_copies":18820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8508,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5501},{"shards":"0,2","objects":2869}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604672507,"num_objects":147627,"num_object_clones":0,"num_object_copies":295254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37934,"num_objects_misplaced":1564,"num_objects_unfound":0,"num_objects_dirty":147627,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145423,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7149,"num_bytes_recovered":25796608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1281134592,"data_stored":1281050575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626483,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601632,"kb_used_data":327056,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93770208,"statfs":{"total":96636764160,"available":96020692992,"internally_reserved":0,"allocated":334905344,"data_stored":334570185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":281145006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149766,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":840900,"kb_used_data":506608,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93530940,"statfs":{"total":96636764160,"available":95775682560,"internally_reserved":0,"allocated":518766592,"data_stored":518500539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342290093},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,2,4,3,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704548,"kb_used_data":420244,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93667292,"statfs":{"total":96636764160,"available":95915307008,"internally_reserved":0,"allocated":430329856,"data_stored":430059731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":517627904,"data_stored":517624291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334376960,"data_stored":334304241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":429129728,"data_stored":429122043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:28.752 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:29.213 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:29.224 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-10T15:57:28.517422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595698190,"num_objects":145438,"num_object_clones":0,"num_object_copies":290876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37798,"num_objects_misplaced":1412,"num_objects_unfound":0,"num_objects_dirty":145438,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147616,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7439,"num_bytes_recovered":26419219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2207840,"kb_used_data":1242476,"kb_used_omap":12,"kb_used_meta":965299,"kb_avail":280907680,"statfs":{"total":289910292480,"available":287649464320,"internally_reserved":0,"allocated":1272295424,"data_stored":1271424087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":988466181},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,4,4,1,4,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18083840,"num_objects":-4415,"num_object_clones":0,"num_object_copies":-8830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":-277,"num_objects_unfound":0,"num_objects_dirty":-4415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4415,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003314"},"pg_stats":[{"pgid":"2.f","version":"46'11641","reported_seq":11980,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.572794+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:24.572794+0000","last_peered":"2024-09-10T15:57:24.572794+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:24.572794+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392384,"num_objects":9129,"num_object_clones":0,"num_object_copies":18258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3841,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11641,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":2717}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11838","reported_seq":11946,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:24.907337+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:24.907337+0000","last_peered":"2024-09-10T15:57:24.907337+0000","last_clean":"2024-09-10T15:57:24.907337+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:24.907337+0000","last_undegraded":"2024-09-10T15:57:24.907337+0000","last_fullsized":"2024-09-10T15:57:24.907337+0000","mapping_epoch":36,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37830656,"num_objects":9236,"num_object_clones":0,"num_object_copies":18472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10174,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11580","reported_seq":11801,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:25.050206+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:25.050206+0000","last_peered":"2024-09-10T15:57:25.050206+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:25.050206+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064728,"num_objects":9050,"num_object_clones":0,"num_object_copies":18100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9848,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2537},{"shards":"2","objects":5297}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11638","reported_seq":11718,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:24.988046+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:24.988046+0000","last_peered":"2024-09-10T15:57:24.988046+0000","last_clean":"2024-09-10T15:57:24.988046+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:24.988046+0000","last_undegraded":"2024-09-10T15:57:24.988046+0000","last_fullsized":"2024-09-10T15:57:24.988046+0000","mapping_epoch":36,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37044224,"num_objects":9044,"num_object_clones":0,"num_object_copies":18088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11638,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11399","reported_seq":11526,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:24.532521+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:24.532521+0000","last_peered":"2024-09-10T15:57:24.532521+0000","last_clean":"2024-09-10T15:57:24.532521+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:24.532521+0000","last_undegraded":"2024-09-10T15:57:24.532521+0000","last_fullsized":"2024-09-10T15:57:24.532521+0000","mapping_epoch":35,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36585472,"num_objects":8932,"num_object_clones":0,"num_object_copies":17864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8470,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11663","reported_seq":16715,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:24.624684+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:24.624684+0000","last_peered":"2024-09-10T15:57:24.624684+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:24.624684+0000","last_undegraded":"2024-09-10T15:57:24.624684+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37699584,"num_objects":9204,"num_object_clones":0,"num_object_copies":18408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1412,"num_objects_unfound":0,"num_objects_dirty":9204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8381,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2458,"num_bytes_recovered":10059776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7792},{"shards":"0,2","objects":1412}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11408","reported_seq":11739,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.611142+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:24.611142+0000","last_peered":"2024-09-10T15:57:24.611142+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:24.611142+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36364288,"num_objects":8878,"num_object_clones":0,"num_object_copies":17756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3741,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8058,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5139},{"shards":"0,2","objects":2572}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11720","reported_seq":11851,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:24.560250+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:24.560250+0000","last_peered":"2024-09-10T15:57:24.560250+0000","last_clean":"2024-09-10T15:57:24.560250+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:24.560250+0000","last_undegraded":"2024-09-10T15:57:24.560250+0000","last_fullsized":"2024-09-10T15:57:24.560250+0000","mapping_epoch":35,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37658624,"num_objects":9194,"num_object_clones":0,"num_object_copies":18388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8479,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11698","reported_seq":19825,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:24.586176+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:24.586176+0000","last_peered":"2024-09-10T15:57:24.586176+0000","last_clean":"2024-09-10T15:57:24.586176+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:24.586176+0000","last_undegraded":"2024-09-10T15:57:24.586176+0000","last_fullsized":"2024-09-10T15:57:24.586176+0000","mapping_epoch":45,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36950016,"num_objects":9021,"num_object_clones":0,"num_object_copies":18042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9675,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11561","reported_seq":11948,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.620277+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:24.620277+0000","last_peered":"2024-09-10T15:57:24.620277+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:24.620277+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36933632,"num_objects":9017,"num_object_clones":0,"num_object_copies":18034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8425,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5285},{"shards":"0,2","objects":2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11639","reported_seq":11862,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:25.002566+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:25.002566+0000","last_peered":"2024-09-10T15:57:25.002566+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:25.002566+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37462016,"num_objects":9146,"num_object_clones":0,"num_object_copies":18292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9613,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2588},{"shards":"2","objects":5363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11500","reported_seq":11713,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.982249+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:24.982249+0000","last_peered":"2024-09-10T15:57:24.982249+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:24.982249+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36683776,"num_objects":8956,"num_object_clones":0,"num_object_copies":17912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9369,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2559},{"shards":"2","objects":5170}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11692","reported_seq":12100,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.540202+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:24.540202+0000","last_peered":"2024-09-10T15:57:24.540202+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:24.540202+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37527552,"num_objects":9162,"num_object_clones":0,"num_object_copies":18324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8884,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5469},{"shards":"0,2","objects":2569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11673","reported_seq":12077,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.610006+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:24.610006+0000","last_peered":"2024-09-10T15:57:24.610006+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:24.610006+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8091,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5248},{"shards":"0,2","objects":2607}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11615","reported_seq":11991,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.572448+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:24.572448+0000","last_peered":"2024-09-10T15:57:24.572448+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:24.572448+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37482496,"num_objects":9151,"num_object_clones":0,"num_object_copies":18302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8221,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5392},{"shards":"0,2","objects":2647}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11828","reported_seq":12177,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.600856+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:24.600856+0000","last_peered":"2024-09-10T15:57:24.600856+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:24.600856+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37970403,"num_objects":9271,"num_object_clones":0,"num_object_copies":18542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3900,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8647,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5374},{"shards":"0,2","objects":2730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":595698171,"num_objects":145436,"num_object_clones":0,"num_object_copies":290872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37798,"num_objects_misplaced":1412,"num_objects_unfound":0,"num_objects_dirty":145436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147614,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7437,"num_bytes_recovered":26419200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1269596160,"data_stored":1269512143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626484,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606652,"kb_used_data":326508,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93765188,"statfs":{"total":96636764160,"available":96015552512,"internally_reserved":0,"allocated":334344192,"data_stored":334009033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286846638},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149767,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":878676,"kb_used_data":499968,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93493164,"statfs":{"total":96636764160,"available":95736999936,"internally_reserved":0,"allocated":511967232,"data_stored":511701179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":387772077},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,1,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722512,"kb_used_data":416000,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93649328,"statfs":{"total":96636764160,"available":95896911872,"internally_reserved":0,"allocated":425984000,"data_stored":425713875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,0,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":510881792,"data_stored":510878179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333873152,"data_stored":333800433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":424841216,"data_stored":424833531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:29.226 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:29.752 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:29.755 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-10T15:57:28.517422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595698190,"num_objects":145438,"num_object_clones":0,"num_object_copies":290876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37798,"num_objects_misplaced":1412,"num_objects_unfound":0,"num_objects_dirty":145438,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147616,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7439,"num_bytes_recovered":26419219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2207840,"kb_used_data":1242476,"kb_used_omap":12,"kb_used_meta":965299,"kb_avail":280907680,"statfs":{"total":289910292480,"available":287649464320,"internally_reserved":0,"allocated":1272295424,"data_stored":1271424087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":988466181},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,4,4,1,4,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18083840,"num_objects":-4415,"num_object_clones":0,"num_object_copies":-8830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":-277,"num_objects_unfound":0,"num_objects_dirty":-4415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4415,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003314"},"pg_stats":[{"pgid":"2.f","version":"46'11641","reported_seq":11980,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.572794+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:24.572794+0000","last_peered":"2024-09-10T15:57:24.572794+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:24.572794+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392384,"num_objects":9129,"num_object_clones":0,"num_object_copies":18258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3841,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11641,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":2717}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11838","reported_seq":11946,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:24.907337+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:24.907337+0000","last_peered":"2024-09-10T15:57:24.907337+0000","last_clean":"2024-09-10T15:57:24.907337+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:24.907337+0000","last_undegraded":"2024-09-10T15:57:24.907337+0000","last_fullsized":"2024-09-10T15:57:24.907337+0000","mapping_epoch":36,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37830656,"num_objects":9236,"num_object_clones":0,"num_object_copies":18472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10174,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11580","reported_seq":11801,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:25.050206+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:25.050206+0000","last_peered":"2024-09-10T15:57:25.050206+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:25.050206+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064728,"num_objects":9050,"num_object_clones":0,"num_object_copies":18100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9848,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2537},{"shards":"2","objects":5297}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11638","reported_seq":11718,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:24.988046+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:24.988046+0000","last_peered":"2024-09-10T15:57:24.988046+0000","last_clean":"2024-09-10T15:57:24.988046+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:24.988046+0000","last_undegraded":"2024-09-10T15:57:24.988046+0000","last_fullsized":"2024-09-10T15:57:24.988046+0000","mapping_epoch":36,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37044224,"num_objects":9044,"num_object_clones":0,"num_object_copies":18088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11638,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11399","reported_seq":11526,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:24.532521+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:24.532521+0000","last_peered":"2024-09-10T15:57:24.532521+0000","last_clean":"2024-09-10T15:57:24.532521+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:24.532521+0000","last_undegraded":"2024-09-10T15:57:24.532521+0000","last_fullsized":"2024-09-10T15:57:24.532521+0000","mapping_epoch":35,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36585472,"num_objects":8932,"num_object_clones":0,"num_object_copies":17864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8470,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11663","reported_seq":16715,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:24.624684+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:24.624684+0000","last_peered":"2024-09-10T15:57:24.624684+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:24.624684+0000","last_undegraded":"2024-09-10T15:57:24.624684+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37699584,"num_objects":9204,"num_object_clones":0,"num_object_copies":18408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1412,"num_objects_unfound":0,"num_objects_dirty":9204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8381,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2458,"num_bytes_recovered":10059776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7792},{"shards":"0,2","objects":1412}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11408","reported_seq":11739,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.611142+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:24.611142+0000","last_peered":"2024-09-10T15:57:24.611142+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:24.611142+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36364288,"num_objects":8878,"num_object_clones":0,"num_object_copies":17756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3741,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8058,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5139},{"shards":"0,2","objects":2572}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11720","reported_seq":11851,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:24.560250+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:24.560250+0000","last_peered":"2024-09-10T15:57:24.560250+0000","last_clean":"2024-09-10T15:57:24.560250+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:24.560250+0000","last_undegraded":"2024-09-10T15:57:24.560250+0000","last_fullsized":"2024-09-10T15:57:24.560250+0000","mapping_epoch":35,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37658624,"num_objects":9194,"num_object_clones":0,"num_object_copies":18388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8479,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11698","reported_seq":19825,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:24.586176+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:24.586176+0000","last_peered":"2024-09-10T15:57:24.586176+0000","last_clean":"2024-09-10T15:57:24.586176+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:24.586176+0000","last_undegraded":"2024-09-10T15:57:24.586176+0000","last_fullsized":"2024-09-10T15:57:24.586176+0000","mapping_epoch":45,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36950016,"num_objects":9021,"num_object_clones":0,"num_object_copies":18042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9675,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11561","reported_seq":11948,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.620277+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:24.620277+0000","last_peered":"2024-09-10T15:57:24.620277+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:24.620277+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36933632,"num_objects":9017,"num_object_clones":0,"num_object_copies":18034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8425,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5285},{"shards":"0,2","objects":2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11639","reported_seq":11862,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:25.002566+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:25.002566+0000","last_peered":"2024-09-10T15:57:25.002566+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:25.002566+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37462016,"num_objects":9146,"num_object_clones":0,"num_object_copies":18292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9613,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2588},{"shards":"2","objects":5363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677573+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:20.677573+0000","last_peered":"2024-09-10T15:55:20.677573+0000","last_clean":"2024-09-10T15:55:20.677573+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:20.677573+0000","last_undegraded":"2024-09-10T15:55:20.677573+0000","last_fullsized":"2024-09-10T15:55:20.677573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11500","reported_seq":11713,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.982249+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:24.982249+0000","last_peered":"2024-09-10T15:57:24.982249+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:24.982249+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36683776,"num_objects":8956,"num_object_clones":0,"num_object_copies":17912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9369,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2559},{"shards":"2","objects":5170}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11692","reported_seq":12100,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.540202+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:24.540202+0000","last_peered":"2024-09-10T15:57:24.540202+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:24.540202+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37527552,"num_objects":9162,"num_object_clones":0,"num_object_copies":18324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8884,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5469},{"shards":"0,2","objects":2569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677836+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:20.677836+0000","last_peered":"2024-09-10T15:55:20.677836+0000","last_clean":"2024-09-10T15:55:20.677836+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:20.677836+0000","last_undegraded":"2024-09-10T15:55:20.677836+0000","last_fullsized":"2024-09-10T15:55:20.677836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11673","reported_seq":12077,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.610006+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:24.610006+0000","last_peered":"2024-09-10T15:57:24.610006+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:24.610006+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8091,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5248},{"shards":"0,2","objects":2607}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11615","reported_seq":11991,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.572448+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:24.572448+0000","last_peered":"2024-09-10T15:57:24.572448+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:24.572448+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37482496,"num_objects":9151,"num_object_clones":0,"num_object_copies":18302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8221,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5392},{"shards":"0,2","objects":2647}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677609+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:20.677609+0000","last_peered":"2024-09-10T15:55:20.677609+0000","last_clean":"2024-09-10T15:55:20.677609+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:20.677609+0000","last_undegraded":"2024-09-10T15:55:20.677609+0000","last_fullsized":"2024-09-10T15:55:20.677609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11828","reported_seq":12177,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:24.600856+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:24.600856+0000","last_peered":"2024-09-10T15:57:24.600856+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:24.600856+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37970403,"num_objects":9271,"num_object_clones":0,"num_object_copies":18542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3900,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8647,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5374},{"shards":"0,2","objects":2730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":595698171,"num_objects":145436,"num_object_clones":0,"num_object_copies":290872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37798,"num_objects_misplaced":1412,"num_objects_unfound":0,"num_objects_dirty":145436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147614,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7437,"num_bytes_recovered":26419200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1269596160,"data_stored":1269512143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626484,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606652,"kb_used_data":326508,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93765188,"statfs":{"total":96636764160,"available":96015552512,"internally_reserved":0,"allocated":334344192,"data_stored":334009033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286846638},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149767,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":878676,"kb_used_data":499968,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93493164,"statfs":{"total":96636764160,"available":95736999936,"internally_reserved":0,"allocated":511967232,"data_stored":511701179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":387772077},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,1,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722512,"kb_used_data":416000,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93649328,"statfs":{"total":96636764160,"available":95896911872,"internally_reserved":0,"allocated":425984000,"data_stored":425713875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,0,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":510881792,"data_stored":510878179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333873152,"data_stored":333800433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":424841216,"data_stored":424833531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:29.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:57:29.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'11641", 'reported_seq': 11980, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:24.572794+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:57:24.572794+0000', 'last_peered': '2024-09-10T15:57:24.572794+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:57:24.572794+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37392384, 'num_objects': 9129, 'num_object_clones': 0, 'num_object_copies': 18258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3841, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9129, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11641, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 132, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5288}, {'shards': '0,2', 'objects': 2717}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:29.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:57:29.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'11580", 'reported_seq': 11801, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:25.050206+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:57:25.050206+0000', 'last_peered': '2024-09-10T15:57:25.050206+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:57:25.050206+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37064728, 'num_objects': 9050, 'num_object_clones': 0, 'num_object_copies': 18100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3754, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9050, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9848, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 49, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2537}, {'shards': '2', 'objects': 5297}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:29.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:57:29.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'11663", 'reported_seq': 16715, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:57:24.624684+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:57:24.624684+0000', 'last_peered': '2024-09-10T15:57:24.624684+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:57:24.624684+0000', 'last_undegraded': '2024-09-10T15:57:24.624684+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37699584, 'num_objects': 9204, 'num_object_clones': 0, 'num_object_copies': 18408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1412, 'num_objects_unfound': 0, 'num_objects_dirty': 9204, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8381, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2458, 'num_bytes_recovered': 10059776, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7792}, {'shards': '0,2', 'objects': 1412}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:29.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:57:29.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'11408", 'reported_seq': 11739, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:24.611142+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:57:24.611142+0000', 'last_peered': '2024-09-10T15:57:24.611142+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:57:24.611142+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36364288, 'num_objects': 8878, 'num_object_clones': 0, 'num_object_copies': 17756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3741, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8878, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8058, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 96, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5139}, {'shards': '0,2', 'objects': 2572}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:29.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:57:29.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'11561", 'reported_seq': 11948, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:24.620277+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:57:24.620277+0000', 'last_peered': '2024-09-10T15:57:24.620277+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:57:24.620277+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36933632, 'num_objects': 9017, 'num_object_clones': 0, 'num_object_copies': 18034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3734, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9017, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8425, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 131, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5285}, {'shards': '0,2', 'objects': 2595}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:29.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:57:29.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'11639", 'reported_seq': 11862, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:25.002566+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:57:25.002566+0000', 'last_peered': '2024-09-10T15:57:25.002566+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:57:25.002566+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37462016, 'num_objects': 9146, 'num_object_clones': 0, 'num_object_copies': 18292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3785, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9146, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9613, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 49, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2588}, {'shards': '2', 'objects': 5363}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:29.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:57:29.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'11500", 'reported_seq': 11713, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:24.982249+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:57:24.982249+0000', 'last_peered': '2024-09-10T15:57:24.982249+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:57:24.982249+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36683776, 'num_objects': 8956, 'num_object_clones': 0, 'num_object_copies': 17912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3788, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8956, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9369, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2559}, {'shards': '2', 'objects': 5170}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:29.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:57:29.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'11692", 'reported_seq': 12100, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:24.540202+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:57:24.540202+0000', 'last_peered': '2024-09-10T15:57:24.540202+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:57:24.540202+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37527552, 'num_objects': 9162, 'num_object_clones': 0, 'num_object_copies': 18324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3694, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9162, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8884, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 140, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5469}, {'shards': '0,2', 'objects': 2569}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:29.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:57:29.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'11673", 'reported_seq': 12077, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:24.610006+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:57:24.610006+0000', 'last_peered': '2024-09-10T15:57:24.610006+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:57:24.610006+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37048320, 'num_objects': 9045, 'num_object_clones': 0, 'num_object_copies': 18090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3800, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8091, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 120, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5248}, {'shards': '0,2', 'objects': 2607}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:29.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:57:29.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'11615", 'reported_seq': 11991, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:24.572448+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:57:24.572448+0000', 'last_peered': '2024-09-10T15:57:24.572448+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:57:24.572448+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37482496, 'num_objects': 9151, 'num_object_clones': 0, 'num_object_copies': 18302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3761, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9151, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8221, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 119, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5392}, {'shards': '0,2', 'objects': 2647}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:29.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:57:29.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'11828", 'reported_seq': 12177, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:24.600856+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:57:24.600856+0000', 'last_peered': '2024-09-10T15:57:24.600856+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:57:24.600856+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'1800", 'ondisk_log_start': "37'1800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37970403, 'num_objects': 9271, 'num_object_clones': 0, 'num_object_copies': 18542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3900, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8647, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 106, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5374}, {'shards': '0,2', 'objects': 2730}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:29.785 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:57:30.471 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:30.471 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37798/290876 objects degraded (12.995%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":376,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":145438,"data_bytes":595698190,"bytes_used":2260828160,"bytes_avail":287649464320,"bytes_total":289910292480,"degraded_objects":37798,"degraded_total":290876,"degraded_ratio":0.12994540628996548,"misplaced_objects":1412,"misplaced_total":290876,"misplaced_ratio":0.0048543021768726194,"recovering_objects_per_sec":42,"recovering_bytes_per_sec":94523,"recovering_keys_per_sec":0,"num_objects_recovered":514,"num_bytes_recovered":1134592,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":367},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:57:30.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:57:30.493 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:31.041 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:31.042 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:31.075 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-10T15:57:30.517746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":587878926,"num_objects":143529,"num_object_clones":0,"num_object_copies":287058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37698,"num_objects_misplaced":1308,"num_objects_unfound":0,"num_objects_dirty":143529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149525,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7643,"num_bytes_recovered":26845203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160605,"ondisk_log_size":160605,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2214140,"kb_used_data":1232140,"kb_used_omap":12,"kb_used_meta":981939,"kb_avail":280901380,"statfs":{"total":289910292480,"available":287643013120,"internally_reserved":0,"allocated":1261711360,"data_stored":1260840023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1005505541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,3,2,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25903104,"num_objects":-6324,"num_object_clones":0,"num_object_copies":-12648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-337,"num_objects_misplaced":-381,"num_objects_unfound":0,"num_objects_dirty":-6324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6324,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":718,"num_bytes_recovered":1560576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002960"},"pg_stats":[{"pgid":"2.f","version":"46'11771","reported_seq":12130,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.649950+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:29.649950+0000","last_peered":"2024-09-10T15:57:29.649950+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:29.649950+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36859904,"num_objects":8999,"num_object_clones":0,"num_object_copies":17998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11771,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5167},{"shards":"0,2","objects":2587}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11947","reported_seq":12055,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:30.030576+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:30.030576+0000","last_peered":"2024-09-10T15:57:30.030576+0000","last_clean":"2024-09-10T15:57:30.030576+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:30.030576+0000","last_undegraded":"2024-09-10T15:57:30.030576+0000","last_fullsized":"2024-09-10T15:57:30.030576+0000","mapping_epoch":36,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11714","reported_seq":11950,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:30.037035+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:30.037035+0000","last_peered":"2024-09-10T15:57:30.037035+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:30.037035+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36515864,"num_objects":8916,"num_object_clones":0,"num_object_copies":17832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9982,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2403},{"shards":"2","objects":5170}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11763","reported_seq":11843,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.999833+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:29.999833+0000","last_peered":"2024-09-10T15:57:29.999833+0000","last_clean":"2024-09-10T15:57:29.999833+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:29.999833+0000","last_undegraded":"2024-09-10T15:57:29.999833+0000","last_fullsized":"2024-09-10T15:57:29.999833+0000","mapping_epoch":36,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36532224,"num_objects":8919,"num_object_clones":0,"num_object_copies":17838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11763,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11509","reported_seq":11636,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.646514+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:29.646514+0000","last_peered":"2024-09-10T15:57:29.646514+0000","last_clean":"2024-09-10T15:57:29.646514+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:29.646514+0000","last_undegraded":"2024-09-10T15:57:29.646514+0000","last_fullsized":"2024-09-10T15:57:29.646514+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8580,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11777","reported_seq":17037,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:29.643035+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:29.643035+0000","last_peered":"2024-09-10T15:57:29.643035+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:29.643035+0000","last_undegraded":"2024-09-10T15:57:29.643035+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37232640,"num_objects":9090,"num_object_clones":0,"num_object_copies":18180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1308,"num_objects_unfound":0,"num_objects_dirty":9090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8495,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2562,"num_bytes_recovered":10485760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7782},{"shards":"0,2","objects":1308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11510","reported_seq":11863,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.540585+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:29.540585+0000","last_peered":"2024-09-10T15:57:29.540585+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:29.540585+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35946496,"num_objects":8776,"num_object_clones":0,"num_object_copies":17552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8160,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5048},{"shards":"0,2","objects":2470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11848","reported_seq":11980,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.663546+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:29.663546+0000","last_peered":"2024-09-10T15:57:29.663546+0000","last_clean":"2024-09-10T15:57:29.663546+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:29.663546+0000","last_undegraded":"2024-09-10T15:57:29.663546+0000","last_fullsized":"2024-09-10T15:57:29.663546+0000","mapping_epoch":35,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37134336,"num_objects":9066,"num_object_clones":0,"num_object_copies":18132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8607,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11808","reported_seq":19936,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.619860+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:29.619860+0000","last_peered":"2024-09-10T15:57:29.619860+0000","last_clean":"2024-09-10T15:57:29.619860+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:29.619860+0000","last_undegraded":"2024-09-10T15:57:29.619860+0000","last_fullsized":"2024-09-10T15:57:29.619860+0000","mapping_epoch":45,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36499456,"num_objects":8911,"num_object_clones":0,"num_object_copies":17822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9785,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11674","reported_seq":12082,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.668345+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:29.668345+0000","last_peered":"2024-09-10T15:57:29.668345+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:29.668345+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36470784,"num_objects":8904,"num_object_clones":0,"num_object_copies":17808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3724,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8538,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5182},{"shards":"0,2","objects":2482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11758","reported_seq":12005,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.959872+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:29.959872+0000","last_peered":"2024-09-10T15:57:29.959872+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:29.959872+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36974592,"num_objects":9027,"num_object_clones":0,"num_object_copies":18054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9732,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2469},{"shards":"2","objects":5256}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11608","reported_seq":11833,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.933475+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:29.933475+0000","last_peered":"2024-09-10T15:57:29.933475+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:29.933475+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36241408,"num_objects":8848,"num_object_clones":0,"num_object_copies":17696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9477,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2451},{"shards":"2","objects":5068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11815","reported_seq":12239,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.600082+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:29.600082+0000","last_peered":"2024-09-10T15:57:29.600082+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:29.600082+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37023744,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9007,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":2446}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11808","reported_seq":12222,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.672869+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:29.672869+0000","last_peered":"2024-09-10T15:57:29.672869+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:29.672869+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36495360,"num_objects":8910,"num_object_clones":0,"num_object_copies":17820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8226,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5118},{"shards":"0,2","objects":2472}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11741","reported_seq":12155,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.673428+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:29.673428+0000","last_peered":"2024-09-10T15:57:29.673428+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:29.673428+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36966400,"num_objects":9025,"num_object_clones":0,"num_object_copies":18050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8347,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5285},{"shards":"0,2","objects":2521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11951","reported_seq":12328,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.654116+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:29.654116+0000","last_peered":"2024-09-10T15:57:29.654116+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:29.654116+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37466595,"num_objects":9148,"num_object_clones":0,"num_object_copies":18296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8770,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5264},{"shards":"0,2","objects":2607}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":587878907,"num_objects":143527,"num_object_clones":0,"num_object_copies":287054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37698,"num_objects_misplaced":1308,"num_objects_unfound":0,"num_objects_dirty":143527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149523,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7641,"num_bytes_recovered":26845184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1259401216,"data_stored":1259317199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160602,"ondisk_log_size":160602,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614416,"kb_used_data":325952,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93757424,"statfs":{"total":96636764160,"available":96007602176,"internally_reserved":0,"allocated":333774848,"data_stored":333439689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295366318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149768,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":876908,"kb_used_data":494044,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93494932,"statfs":{"total":96636764160,"available":95738810368,"internally_reserved":0,"allocated":505901056,"data_stored":505635003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392031917},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722816,"kb_used_data":412144,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93649024,"statfs":{"total":96636764160,"available":95896600576,"internally_reserved":0,"allocated":422035456,"data_stored":421765331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":505020416,"data_stored":505016803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333377536,"data_stored":333304817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":421003264,"data_stored":420995579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:34.077 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:34.631 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:34.632 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:34.657 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-10T15:57:34.518730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":587878926,"num_objects":143529,"num_object_clones":0,"num_object_copies":287058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37698,"num_objects_misplaced":1308,"num_objects_unfound":0,"num_objects_dirty":143529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149525,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7643,"num_bytes_recovered":26845203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160605,"ondisk_log_size":160605,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2214140,"kb_used_data":1232140,"kb_used_omap":12,"kb_used_meta":981939,"kb_avail":280901380,"statfs":{"total":289910292480,"available":287643013120,"internally_reserved":0,"allocated":1261711360,"data_stored":1260840023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1005505541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,3,2,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16793600,"num_objects":-4100,"num_object_clones":0,"num_object_copies":-8200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-236,"num_objects_misplaced":-256,"num_objects_unfound":0,"num_objects_dirty":-4100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4100,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002985"},"pg_stats":[{"pgid":"2.f","version":"46'11771","reported_seq":12130,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.649950+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:29.649950+0000","last_peered":"2024-09-10T15:57:29.649950+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:29.649950+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36859904,"num_objects":8999,"num_object_clones":0,"num_object_copies":17998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11771,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5167},{"shards":"0,2","objects":2587}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11947","reported_seq":12055,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:30.030576+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:30.030576+0000","last_peered":"2024-09-10T15:57:30.030576+0000","last_clean":"2024-09-10T15:57:30.030576+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:30.030576+0000","last_undegraded":"2024-09-10T15:57:30.030576+0000","last_fullsized":"2024-09-10T15:57:30.030576+0000","mapping_epoch":36,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11714","reported_seq":11950,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:30.037035+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:30.037035+0000","last_peered":"2024-09-10T15:57:30.037035+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:30.037035+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36515864,"num_objects":8916,"num_object_clones":0,"num_object_copies":17832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9982,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2403},{"shards":"2","objects":5170}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11763","reported_seq":11843,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.999833+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:29.999833+0000","last_peered":"2024-09-10T15:57:29.999833+0000","last_clean":"2024-09-10T15:57:29.999833+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:29.999833+0000","last_undegraded":"2024-09-10T15:57:29.999833+0000","last_fullsized":"2024-09-10T15:57:29.999833+0000","mapping_epoch":36,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36532224,"num_objects":8919,"num_object_clones":0,"num_object_copies":17838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11763,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11509","reported_seq":11636,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.646514+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:29.646514+0000","last_peered":"2024-09-10T15:57:29.646514+0000","last_clean":"2024-09-10T15:57:29.646514+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:29.646514+0000","last_undegraded":"2024-09-10T15:57:29.646514+0000","last_fullsized":"2024-09-10T15:57:29.646514+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8580,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11777","reported_seq":17037,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:29.643035+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:29.643035+0000","last_peered":"2024-09-10T15:57:29.643035+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:29.643035+0000","last_undegraded":"2024-09-10T15:57:29.643035+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37232640,"num_objects":9090,"num_object_clones":0,"num_object_copies":18180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1308,"num_objects_unfound":0,"num_objects_dirty":9090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8495,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2562,"num_bytes_recovered":10485760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7782},{"shards":"0,2","objects":1308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11510","reported_seq":11863,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.540585+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:29.540585+0000","last_peered":"2024-09-10T15:57:29.540585+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:29.540585+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35946496,"num_objects":8776,"num_object_clones":0,"num_object_copies":17552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8160,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5048},{"shards":"0,2","objects":2470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11848","reported_seq":11980,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.663546+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:29.663546+0000","last_peered":"2024-09-10T15:57:29.663546+0000","last_clean":"2024-09-10T15:57:29.663546+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:29.663546+0000","last_undegraded":"2024-09-10T15:57:29.663546+0000","last_fullsized":"2024-09-10T15:57:29.663546+0000","mapping_epoch":35,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37134336,"num_objects":9066,"num_object_clones":0,"num_object_copies":18132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8607,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11808","reported_seq":19936,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.619860+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:29.619860+0000","last_peered":"2024-09-10T15:57:29.619860+0000","last_clean":"2024-09-10T15:57:29.619860+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:29.619860+0000","last_undegraded":"2024-09-10T15:57:29.619860+0000","last_fullsized":"2024-09-10T15:57:29.619860+0000","mapping_epoch":45,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36499456,"num_objects":8911,"num_object_clones":0,"num_object_copies":17822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9785,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11674","reported_seq":12082,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.668345+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:29.668345+0000","last_peered":"2024-09-10T15:57:29.668345+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:29.668345+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36470784,"num_objects":8904,"num_object_clones":0,"num_object_copies":17808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3724,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8538,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5182},{"shards":"0,2","objects":2482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11758","reported_seq":12005,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.959872+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:29.959872+0000","last_peered":"2024-09-10T15:57:29.959872+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:29.959872+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36974592,"num_objects":9027,"num_object_clones":0,"num_object_copies":18054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9732,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2469},{"shards":"2","objects":5256}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11608","reported_seq":11833,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.933475+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:29.933475+0000","last_peered":"2024-09-10T15:57:29.933475+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:29.933475+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36241408,"num_objects":8848,"num_object_clones":0,"num_object_copies":17696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9477,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2451},{"shards":"2","objects":5068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11815","reported_seq":12239,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.600082+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:29.600082+0000","last_peered":"2024-09-10T15:57:29.600082+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:29.600082+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37023744,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9007,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":2446}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11808","reported_seq":12222,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.672869+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:29.672869+0000","last_peered":"2024-09-10T15:57:29.672869+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:29.672869+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36495360,"num_objects":8910,"num_object_clones":0,"num_object_copies":17820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8226,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5118},{"shards":"0,2","objects":2472}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11741","reported_seq":12155,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.673428+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:29.673428+0000","last_peered":"2024-09-10T15:57:29.673428+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:29.673428+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36966400,"num_objects":9025,"num_object_clones":0,"num_object_copies":18050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8347,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5285},{"shards":"0,2","objects":2521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11951","reported_seq":12328,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.654116+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:29.654116+0000","last_peered":"2024-09-10T15:57:29.654116+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:29.654116+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37466595,"num_objects":9148,"num_object_clones":0,"num_object_copies":18296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8770,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5264},{"shards":"0,2","objects":2607}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":587878907,"num_objects":143527,"num_object_clones":0,"num_object_copies":287054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37698,"num_objects_misplaced":1308,"num_objects_unfound":0,"num_objects_dirty":143527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149523,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7641,"num_bytes_recovered":26845184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1259401216,"data_stored":1259317199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160602,"ondisk_log_size":160602,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614416,"kb_used_data":325952,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93757424,"statfs":{"total":96636764160,"available":96007602176,"internally_reserved":0,"allocated":333774848,"data_stored":333439689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295366318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149768,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":876908,"kb_used_data":494044,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93494932,"statfs":{"total":96636764160,"available":95738810368,"internally_reserved":0,"allocated":505901056,"data_stored":505635003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392031917},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722816,"kb_used_data":412144,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93649024,"statfs":{"total":96636764160,"available":95896600576,"internally_reserved":0,"allocated":422035456,"data_stored":421765331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":505020416,"data_stored":505016803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333377536,"data_stored":333304817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":421003264,"data_stored":420995579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:34.659 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:35.026 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:35.026 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:35.050 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-10T15:57:34.518730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":587878926,"num_objects":143529,"num_object_clones":0,"num_object_copies":287058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37698,"num_objects_misplaced":1308,"num_objects_unfound":0,"num_objects_dirty":143529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149525,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7643,"num_bytes_recovered":26845203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160605,"ondisk_log_size":160605,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2214140,"kb_used_data":1232140,"kb_used_omap":12,"kb_used_meta":981939,"kb_avail":280901380,"statfs":{"total":289910292480,"available":287643013120,"internally_reserved":0,"allocated":1261711360,"data_stored":1260840023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1005505541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,3,2,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16793600,"num_objects":-4100,"num_object_clones":0,"num_object_copies":-8200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-236,"num_objects_misplaced":-256,"num_objects_unfound":0,"num_objects_dirty":-4100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4100,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002985"},"pg_stats":[{"pgid":"2.f","version":"46'11771","reported_seq":12130,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.649950+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:29.649950+0000","last_peered":"2024-09-10T15:57:29.649950+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:29.649950+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36859904,"num_objects":8999,"num_object_clones":0,"num_object_copies":17998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11771,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5167},{"shards":"0,2","objects":2587}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11947","reported_seq":12055,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:30.030576+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:30.030576+0000","last_peered":"2024-09-10T15:57:30.030576+0000","last_clean":"2024-09-10T15:57:30.030576+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:30.030576+0000","last_undegraded":"2024-09-10T15:57:30.030576+0000","last_fullsized":"2024-09-10T15:57:30.030576+0000","mapping_epoch":36,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11714","reported_seq":11950,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:30.037035+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:30.037035+0000","last_peered":"2024-09-10T15:57:30.037035+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:30.037035+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36515864,"num_objects":8916,"num_object_clones":0,"num_object_copies":17832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9982,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2403},{"shards":"2","objects":5170}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11763","reported_seq":11843,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.999833+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:29.999833+0000","last_peered":"2024-09-10T15:57:29.999833+0000","last_clean":"2024-09-10T15:57:29.999833+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:29.999833+0000","last_undegraded":"2024-09-10T15:57:29.999833+0000","last_fullsized":"2024-09-10T15:57:29.999833+0000","mapping_epoch":36,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36532224,"num_objects":8919,"num_object_clones":0,"num_object_copies":17838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11763,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11509","reported_seq":11636,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.646514+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:29.646514+0000","last_peered":"2024-09-10T15:57:29.646514+0000","last_clean":"2024-09-10T15:57:29.646514+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:29.646514+0000","last_undegraded":"2024-09-10T15:57:29.646514+0000","last_fullsized":"2024-09-10T15:57:29.646514+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8580,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11777","reported_seq":17037,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:29.643035+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:29.643035+0000","last_peered":"2024-09-10T15:57:29.643035+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:29.643035+0000","last_undegraded":"2024-09-10T15:57:29.643035+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37232640,"num_objects":9090,"num_object_clones":0,"num_object_copies":18180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1308,"num_objects_unfound":0,"num_objects_dirty":9090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8495,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2562,"num_bytes_recovered":10485760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7782},{"shards":"0,2","objects":1308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11510","reported_seq":11863,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.540585+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:29.540585+0000","last_peered":"2024-09-10T15:57:29.540585+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:29.540585+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35946496,"num_objects":8776,"num_object_clones":0,"num_object_copies":17552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8160,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5048},{"shards":"0,2","objects":2470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11848","reported_seq":11980,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.663546+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:29.663546+0000","last_peered":"2024-09-10T15:57:29.663546+0000","last_clean":"2024-09-10T15:57:29.663546+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:29.663546+0000","last_undegraded":"2024-09-10T15:57:29.663546+0000","last_fullsized":"2024-09-10T15:57:29.663546+0000","mapping_epoch":35,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37134336,"num_objects":9066,"num_object_clones":0,"num_object_copies":18132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8607,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11808","reported_seq":19936,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.619860+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:29.619860+0000","last_peered":"2024-09-10T15:57:29.619860+0000","last_clean":"2024-09-10T15:57:29.619860+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:29.619860+0000","last_undegraded":"2024-09-10T15:57:29.619860+0000","last_fullsized":"2024-09-10T15:57:29.619860+0000","mapping_epoch":45,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36499456,"num_objects":8911,"num_object_clones":0,"num_object_copies":17822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9785,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11674","reported_seq":12082,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.668345+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:29.668345+0000","last_peered":"2024-09-10T15:57:29.668345+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:29.668345+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36470784,"num_objects":8904,"num_object_clones":0,"num_object_copies":17808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3724,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8538,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5182},{"shards":"0,2","objects":2482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11758","reported_seq":12005,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.959872+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:29.959872+0000","last_peered":"2024-09-10T15:57:29.959872+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:29.959872+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36974592,"num_objects":9027,"num_object_clones":0,"num_object_copies":18054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9732,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2469},{"shards":"2","objects":5256}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11608","reported_seq":11833,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.933475+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:29.933475+0000","last_peered":"2024-09-10T15:57:29.933475+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:29.933475+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36241408,"num_objects":8848,"num_object_clones":0,"num_object_copies":17696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9477,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2451},{"shards":"2","objects":5068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11815","reported_seq":12239,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.600082+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:29.600082+0000","last_peered":"2024-09-10T15:57:29.600082+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:29.600082+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37023744,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9007,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":2446}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11808","reported_seq":12222,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.672869+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:29.672869+0000","last_peered":"2024-09-10T15:57:29.672869+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:29.672869+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36495360,"num_objects":8910,"num_object_clones":0,"num_object_copies":17820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8226,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5118},{"shards":"0,2","objects":2472}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11741","reported_seq":12155,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.673428+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:29.673428+0000","last_peered":"2024-09-10T15:57:29.673428+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:29.673428+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36966400,"num_objects":9025,"num_object_clones":0,"num_object_copies":18050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8347,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5285},{"shards":"0,2","objects":2521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11951","reported_seq":12328,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.654116+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:29.654116+0000","last_peered":"2024-09-10T15:57:29.654116+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:29.654116+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37466595,"num_objects":9148,"num_object_clones":0,"num_object_copies":18296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8770,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5264},{"shards":"0,2","objects":2607}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":587878907,"num_objects":143527,"num_object_clones":0,"num_object_copies":287054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37698,"num_objects_misplaced":1308,"num_objects_unfound":0,"num_objects_dirty":143527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149523,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7641,"num_bytes_recovered":26845184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1259401216,"data_stored":1259317199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160602,"ondisk_log_size":160602,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614416,"kb_used_data":325952,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93757424,"statfs":{"total":96636764160,"available":96007602176,"internally_reserved":0,"allocated":333774848,"data_stored":333439689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295366318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149768,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":876908,"kb_used_data":494044,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93494932,"statfs":{"total":96636764160,"available":95738810368,"internally_reserved":0,"allocated":505901056,"data_stored":505635003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392031917},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722816,"kb_used_data":412144,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93649024,"statfs":{"total":96636764160,"available":95896600576,"internally_reserved":0,"allocated":422035456,"data_stored":421765331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":505020416,"data_stored":505016803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333377536,"data_stored":333304817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":421003264,"data_stored":420995579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:35.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:57:35.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'11771", 'reported_seq': 12130, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:29.649950+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:57:29.649950+0000', 'last_peered': '2024-09-10T15:57:29.649950+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:57:29.649950+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36859904, 'num_objects': 8999, 'num_object_clones': 0, 'num_object_copies': 17998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3832, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8999, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11771, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 141, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5167}, {'shards': '0,2', 'objects': 2587}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:35.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:57:35.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'11714", 'reported_seq': 11950, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:30.037035+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:57:30.037035+0000', 'last_peered': '2024-09-10T15:57:30.037035+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:57:30.037035+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36515864, 'num_objects': 8916, 'num_object_clones': 0, 'num_object_copies': 17832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3747, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8916, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9982, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2403}, {'shards': '2', 'objects': 5170}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:35.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:57:35.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'11777", 'reported_seq': 17037, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:57:29.643035+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:57:29.643035+0000', 'last_peered': '2024-09-10T15:57:29.643035+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:57:29.643035+0000', 'last_undegraded': '2024-09-10T15:57:29.643035+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37232640, 'num_objects': 9090, 'num_object_clones': 0, 'num_object_copies': 18180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1308, 'num_objects_unfound': 0, 'num_objects_dirty': 9090, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8495, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2562, 'num_bytes_recovered': 10485760, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7782}, {'shards': '0,2', 'objects': 1308}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:35.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:57:35.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'11510", 'reported_seq': 11863, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:29.540585+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:57:29.540585+0000', 'last_peered': '2024-09-10T15:57:29.540585+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:57:29.540585+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35946496, 'num_objects': 8776, 'num_object_clones': 0, 'num_object_copies': 17552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3730, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8160, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 107, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5048}, {'shards': '0,2', 'objects': 2470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:35.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:57:35.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'11674", 'reported_seq': 12082, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:29.668345+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:57:29.668345+0000', 'last_peered': '2024-09-10T15:57:29.668345+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:57:29.668345+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36470784, 'num_objects': 8904, 'num_object_clones': 0, 'num_object_copies': 17808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3724, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8904, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8538, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 141, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5182}, {'shards': '0,2', 'objects': 2482}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:35.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:57:35.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'11758", 'reported_seq': 12005, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:29.959872+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:57:29.959872+0000', 'last_peered': '2024-09-10T15:57:29.959872+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:57:29.959872+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36974592, 'num_objects': 9027, 'num_object_clones': 0, 'num_object_copies': 18054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3773, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9027, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9732, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2469}, {'shards': '2', 'objects': 5256}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:35.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:57:35.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'11608", 'reported_seq': 11833, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:29.933475+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:57:29.933475+0000', 'last_peered': '2024-09-10T15:57:29.933475+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:57:29.933475+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36241408, 'num_objects': 8848, 'num_object_clones': 0, 'num_object_copies': 17696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3782, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8848, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9477, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 49, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2451}, {'shards': '2', 'objects': 5068}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:35.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:57:35.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'11815", 'reported_seq': 12239, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:29.600082+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:57:29.600082+0000', 'last_peered': '2024-09-10T15:57:29.600082+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:57:29.600082+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'1800", 'ondisk_log_start': "37'1800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37023744, 'num_objects': 9039, 'num_object_clones': 0, 'num_object_copies': 18078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3686, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9039, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9007, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 148, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5354}, {'shards': '0,2', 'objects': 2446}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:35.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:57:35.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'11808", 'reported_seq': 12222, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:29.672869+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:57:29.672869+0000', 'last_peered': '2024-09-10T15:57:29.672869+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:57:29.672869+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'1800", 'ondisk_log_start': "37'1800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36495360, 'num_objects': 8910, 'num_object_clones': 0, 'num_object_copies': 17820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3795, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8910, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8226, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 125, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5118}, {'shards': '0,2', 'objects': 2472}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:35.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:57:35.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'11741", 'reported_seq': 12155, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:29.673428+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:57:29.673428+0000', 'last_peered': '2024-09-10T15:57:29.673428+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:57:29.673428+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36966400, 'num_objects': 9025, 'num_object_clones': 0, 'num_object_copies': 18050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3742, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9025, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8347, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 138, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5285}, {'shards': '0,2', 'objects': 2521}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:35.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:57:35.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'11951", 'reported_seq': 12328, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:29.654116+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:57:29.654116+0000', 'last_peered': '2024-09-10T15:57:29.654116+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:57:29.654116+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'1900", 'ondisk_log_start': "37'1900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37466595, 'num_objects': 9148, 'num_object_clones': 0, 'num_object_copies': 18296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3887, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9148, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8770, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 119, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5264}, {'shards': '0,2', 'objects': 2607}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:35.053 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:57:35.758 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:35.758 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37698/287058 objects degraded (13.133%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":381,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":143529,"data_bytes":587878926,"bytes_used":2267279360,"bytes_avail":287643013120,"bytes_total":289910292480,"degraded_objects":37698,"degraded_total":287058,"degraded_ratio":0.13132537675313002,"misplaced_objects":1308,"misplaced_total":287058,"misplaced_ratio":0.0045565704491775183,"recovering_objects_per_sec":40,"recovering_bytes_per_sec":87359,"recovering_keys_per_sec":0,"num_objects_recovered":492,"num_bytes_recovered":1048576,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":341},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:57:35.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:57:35.782 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:36.337 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:36.338 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:36.357 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-10T15:57:34.518730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":587878926,"num_objects":143529,"num_object_clones":0,"num_object_copies":287058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37698,"num_objects_misplaced":1308,"num_objects_unfound":0,"num_objects_dirty":143529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149525,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7643,"num_bytes_recovered":26845203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160605,"ondisk_log_size":160605,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2214140,"kb_used_data":1232140,"kb_used_omap":12,"kb_used_meta":981939,"kb_avail":280901380,"statfs":{"total":289910292480,"available":287643013120,"internally_reserved":0,"allocated":1261711360,"data_stored":1260840023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1005505541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,3,2,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16793600,"num_objects":-4100,"num_object_clones":0,"num_object_copies":-8200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-236,"num_objects_misplaced":-256,"num_objects_unfound":0,"num_objects_dirty":-4100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4100,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002985"},"pg_stats":[{"pgid":"2.f","version":"46'11771","reported_seq":12130,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.649950+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:29.649950+0000","last_peered":"2024-09-10T15:57:29.649950+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:29.649950+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36859904,"num_objects":8999,"num_object_clones":0,"num_object_copies":17998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11771,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5167},{"shards":"0,2","objects":2587}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'11947","reported_seq":12055,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:30.030576+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:30.030576+0000","last_peered":"2024-09-10T15:57:30.030576+0000","last_clean":"2024-09-10T15:57:30.030576+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:30.030576+0000","last_undegraded":"2024-09-10T15:57:30.030576+0000","last_fullsized":"2024-09-10T15:57:30.030576+0000","mapping_epoch":36,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11714","reported_seq":11950,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:30.037035+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:30.037035+0000","last_peered":"2024-09-10T15:57:30.037035+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:30.037035+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36515864,"num_objects":8916,"num_object_clones":0,"num_object_copies":17832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9982,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2403},{"shards":"2","objects":5170}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11763","reported_seq":11843,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.999833+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:29.999833+0000","last_peered":"2024-09-10T15:57:29.999833+0000","last_clean":"2024-09-10T15:57:29.999833+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:29.999833+0000","last_undegraded":"2024-09-10T15:57:29.999833+0000","last_fullsized":"2024-09-10T15:57:29.999833+0000","mapping_epoch":36,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36532224,"num_objects":8919,"num_object_clones":0,"num_object_copies":17838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11763,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11509","reported_seq":11636,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.646514+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:29.646514+0000","last_peered":"2024-09-10T15:57:29.646514+0000","last_clean":"2024-09-10T15:57:29.646514+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:29.646514+0000","last_undegraded":"2024-09-10T15:57:29.646514+0000","last_fullsized":"2024-09-10T15:57:29.646514+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8580,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11777","reported_seq":17037,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:29.643035+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:29.643035+0000","last_peered":"2024-09-10T15:57:29.643035+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:29.643035+0000","last_undegraded":"2024-09-10T15:57:29.643035+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37232640,"num_objects":9090,"num_object_clones":0,"num_object_copies":18180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1308,"num_objects_unfound":0,"num_objects_dirty":9090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8495,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2562,"num_bytes_recovered":10485760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7782},{"shards":"0,2","objects":1308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11510","reported_seq":11863,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.540585+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:29.540585+0000","last_peered":"2024-09-10T15:57:29.540585+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:29.540585+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35946496,"num_objects":8776,"num_object_clones":0,"num_object_copies":17552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8160,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5048},{"shards":"0,2","objects":2470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11848","reported_seq":11980,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.663546+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:29.663546+0000","last_peered":"2024-09-10T15:57:29.663546+0000","last_clean":"2024-09-10T15:57:29.663546+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:29.663546+0000","last_undegraded":"2024-09-10T15:57:29.663546+0000","last_fullsized":"2024-09-10T15:57:29.663546+0000","mapping_epoch":35,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37134336,"num_objects":9066,"num_object_clones":0,"num_object_copies":18132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8607,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11808","reported_seq":19936,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:29.619860+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:29.619860+0000","last_peered":"2024-09-10T15:57:29.619860+0000","last_clean":"2024-09-10T15:57:29.619860+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:29.619860+0000","last_undegraded":"2024-09-10T15:57:29.619860+0000","last_fullsized":"2024-09-10T15:57:29.619860+0000","mapping_epoch":45,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36499456,"num_objects":8911,"num_object_clones":0,"num_object_copies":17822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9785,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11674","reported_seq":12082,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.668345+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:29.668345+0000","last_peered":"2024-09-10T15:57:29.668345+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:29.668345+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36470784,"num_objects":8904,"num_object_clones":0,"num_object_copies":17808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3724,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8538,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5182},{"shards":"0,2","objects":2482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11758","reported_seq":12005,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.959872+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:29.959872+0000","last_peered":"2024-09-10T15:57:29.959872+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:29.959872+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36974592,"num_objects":9027,"num_object_clones":0,"num_object_copies":18054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9732,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2469},{"shards":"2","objects":5256}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11608","reported_seq":11833,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.933475+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:29.933475+0000","last_peered":"2024-09-10T15:57:29.933475+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:29.933475+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36241408,"num_objects":8848,"num_object_clones":0,"num_object_copies":17696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9477,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2451},{"shards":"2","objects":5068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11815","reported_seq":12239,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.600082+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:29.600082+0000","last_peered":"2024-09-10T15:57:29.600082+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:29.600082+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37023744,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9007,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":2446}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11808","reported_seq":12222,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.672869+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:29.672869+0000","last_peered":"2024-09-10T15:57:29.672869+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:29.672869+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36495360,"num_objects":8910,"num_object_clones":0,"num_object_copies":17820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3795,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8226,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5118},{"shards":"0,2","objects":2472}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11741","reported_seq":12155,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.673428+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:29.673428+0000","last_peered":"2024-09-10T15:57:29.673428+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:29.673428+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36966400,"num_objects":9025,"num_object_clones":0,"num_object_copies":18050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8347,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5285},{"shards":"0,2","objects":2521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.677705+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:20.677705+0000","last_peered":"2024-09-10T15:55:20.677705+0000","last_clean":"2024-09-10T15:55:20.677705+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:20.677705+0000","last_undegraded":"2024-09-10T15:55:20.677705+0000","last_fullsized":"2024-09-10T15:55:20.677705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'11951","reported_seq":12328,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:29.654116+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:29.654116+0000","last_peered":"2024-09-10T15:57:29.654116+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:29.654116+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37466595,"num_objects":9148,"num_object_clones":0,"num_object_copies":18296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8770,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5264},{"shards":"0,2","objects":2607}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":587878907,"num_objects":143527,"num_object_clones":0,"num_object_copies":287054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37698,"num_objects_misplaced":1308,"num_objects_unfound":0,"num_objects_dirty":143527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149523,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7641,"num_bytes_recovered":26845184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1259401216,"data_stored":1259317199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160602,"ondisk_log_size":160602,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614416,"kb_used_data":325952,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93757424,"statfs":{"total":96636764160,"available":96007602176,"internally_reserved":0,"allocated":333774848,"data_stored":333439689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295366318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149768,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":876908,"kb_used_data":494044,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93494932,"statfs":{"total":96636764160,"available":95738810368,"internally_reserved":0,"allocated":505901056,"data_stored":505635003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392031917},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722816,"kb_used_data":412144,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93649024,"statfs":{"total":96636764160,"available":95896600576,"internally_reserved":0,"allocated":422035456,"data_stored":421765331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":505020416,"data_stored":505016803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333377536,"data_stored":333304817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":421003264,"data_stored":420995579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:39.359 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:39.931 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:39.933 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:39.954 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-10T15:57:38.519970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":578892302,"num_objects":141335,"num_object_clones":0,"num_object_copies":282670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37630,"num_objects_misplaced":1141,"num_objects_unfound":0,"num_objects_dirty":141335,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":151719,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7878,"num_bytes_recovered":27529235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2182192,"kb_used_data":1220984,"kb_used_omap":12,"kb_used_meta":961139,"kb_avail":280933328,"statfs":{"total":289910292480,"available":287675727872,"internally_reserved":0,"allocated":1250287616,"data_stored":1249416279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":984206341},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,12,6,7,4,1,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16805888,"num_objects":-4103,"num_object_clones":0,"num_object_copies":-8206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-168,"num_objects_misplaced":-271,"num_objects_unfound":0,"num_objects_dirty":-4103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4103,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":439,"num_bytes_recovered":1110016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002938"},"pg_stats":[{"pgid":"2.f","version":"46'11915","reported_seq":12292,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.643197+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:34.643197+0000","last_peered":"2024-09-10T15:57:34.643197+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:34.643197+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36270080,"num_objects":8855,"num_object_clones":0,"num_object_copies":17710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11915,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5032},{"shards":"0,2","objects":2443}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12080","reported_seq":12188,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:35.007102+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:35.007102+0000","last_peered":"2024-09-10T15:57:35.007102+0000","last_clean":"2024-09-10T15:57:35.007102+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:35.007102+0000","last_undegraded":"2024-09-10T15:57:35.007102+0000","last_fullsized":"2024-09-10T15:57:35.007102+0000","mapping_epoch":36,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36839424,"num_objects":8994,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10416,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11856","reported_seq":12105,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:35.016266+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:35.016266+0000","last_peered":"2024-09-10T15:57:35.016266+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:35.016266+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35934232,"num_objects":8774,"num_object_clones":0,"num_object_copies":17548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3741,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8774,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10124,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2261},{"shards":"2","objects":5034}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11896","reported_seq":11976,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.938157+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:34.938157+0000","last_peered":"2024-09-10T15:57:34.938157+0000","last_clean":"2024-09-10T15:57:34.938157+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:34.938157+0000","last_undegraded":"2024-09-10T15:57:34.938157+0000","last_fullsized":"2024-09-10T15:57:34.938157+0000","mapping_epoch":36,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35987456,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11896,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11648","reported_seq":11775,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.639470+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:34.639470+0000","last_peered":"2024-09-10T15:57:34.639470+0000","last_clean":"2024-09-10T15:57:34.639470+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:34.639470+0000","last_undegraded":"2024-09-10T15:57:34.639470+0000","last_fullsized":"2024-09-10T15:57:34.639470+0000","mapping_epoch":35,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8719,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11913","reported_seq":17506,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:34.705330+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:34.705330+0000","last_peered":"2024-09-10T15:57:34.705330+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:34.705330+0000","last_undegraded":"2024-09-10T15:57:34.705330+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36675584,"num_objects":8954,"num_object_clones":0,"num_object_copies":17908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1141,"num_objects_unfound":0,"num_objects_dirty":8954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8631,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2729,"num_bytes_recovered":11169792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7813},{"shards":"0,2","objects":1141}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11653","reported_seq":12016,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.672951+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:34.672951+0000","last_peered":"2024-09-10T15:57:34.672951+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:34.672951+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3725,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8303,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4910},{"shards":"0,2","objects":2327}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11996","reported_seq":12128,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.546519+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:34.546519+0000","last_peered":"2024-09-10T15:57:34.546519+0000","last_clean":"2024-09-10T15:57:34.546519+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:34.546519+0000","last_undegraded":"2024-09-10T15:57:34.546519+0000","last_fullsized":"2024-09-10T15:57:34.546519+0000","mapping_epoch":35,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36528128,"num_objects":8918,"num_object_clones":0,"num_object_copies":17836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8755,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11955","reported_seq":20083,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.687617+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:34.687617+0000","last_peered":"2024-09-10T15:57:34.687617+0000","last_clean":"2024-09-10T15:57:34.687617+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:34.687617+0000","last_undegraded":"2024-09-10T15:57:34.687617+0000","last_fullsized":"2024-09-10T15:57:34.687617+0000","mapping_epoch":45,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35897344,"num_objects":8764,"num_object_clones":0,"num_object_copies":17528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9932,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11786","reported_seq":12214,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.616637+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:34.616637+0000","last_peered":"2024-09-10T15:57:34.616637+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:34.616637+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36012032,"num_objects":8792,"num_object_clones":0,"num_object_copies":17584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3714,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8650,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5080},{"shards":"0,2","objects":2370}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11917","reported_seq":12176,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:35.032155+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:35.032155+0000","last_peered":"2024-09-10T15:57:35.032155+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:35.032155+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36323328,"num_objects":8868,"num_object_clones":0,"num_object_copies":17736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9891,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2310},{"shards":"2","objects":5103}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11721","reported_seq":11952,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.619118+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:34.619118+0000","last_peered":"2024-09-10T15:57:34.619118+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:34.619118+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35778560,"num_objects":8735,"num_object_clones":0,"num_object_copies":17470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3779,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9590,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2338},{"shards":"2","objects":4958}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11949","reported_seq":12391,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.593757+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:34.593757+0000","last_peered":"2024-09-10T15:57:34.593757+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:34.593757+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36474880,"num_objects":8905,"num_object_clones":0,"num_object_copies":17810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9141,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5229},{"shards":"0,2","objects":2312}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11945","reported_seq":12365,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.652863+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:34.652863+0000","last_peered":"2024-09-10T15:57:34.652863+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:34.652863+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35934208,"num_objects":8773,"num_object_clones":0,"num_object_copies":17546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3792,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8363,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4984},{"shards":"0,2","objects":2335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11874","reported_seq":12310,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.659148+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:34.659148+0000","last_peered":"2024-09-10T15:57:34.659148+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:34.659148+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36421632,"num_objects":8892,"num_object_clones":0,"num_object_copies":17784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8480,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5163},{"shards":"0,2","objects":2388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12092","reported_seq":12481,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.563139+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:34.563139+0000","last_peered":"2024-09-10T15:57:34.563139+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:34.563139+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36889059,"num_objects":9007,"num_object_clones":0,"num_object_copies":18014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3881,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8911,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5129},{"shards":"0,2","objects":2466}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":578892283,"num_objects":141333,"num_object_clones":0,"num_object_copies":282666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37630,"num_objects_misplaced":1141,"num_objects_unfound":0,"num_objects_dirty":141333,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":151717,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7876,"num_bytes_recovered":27529216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1247608832,"data_stored":1247524815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626486,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618256,"kb_used_data":325632,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93753584,"statfs":{"total":96636764160,"available":96003670016,"internally_reserved":0,"allocated":333447168,"data_stored":333112009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299626158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149769,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":858004,"kb_used_data":487612,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93513836,"statfs":{"total":96636764160,"available":95758168064,"internally_reserved":0,"allocated":499314688,"data_stored":499048635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":379252397},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,2,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705932,"kb_used_data":407740,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93665908,"statfs":{"total":96636764160,"available":95913889792,"internally_reserved":0,"allocated":417525760,"data_stored":417255635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,1,1,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":498233344,"data_stored":498229731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":332906496,"data_stored":332833777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":416468992,"data_stored":416461307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:39.955 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:40.512 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:40.513 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:40.543 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-10T15:57:38.519970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":578892302,"num_objects":141335,"num_object_clones":0,"num_object_copies":282670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37630,"num_objects_misplaced":1141,"num_objects_unfound":0,"num_objects_dirty":141335,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":151719,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7878,"num_bytes_recovered":27529235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2182192,"kb_used_data":1220984,"kb_used_omap":12,"kb_used_meta":961139,"kb_avail":280933328,"statfs":{"total":289910292480,"available":287675727872,"internally_reserved":0,"allocated":1250287616,"data_stored":1249416279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":984206341},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,12,6,7,4,1,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16805888,"num_objects":-4103,"num_object_clones":0,"num_object_copies":-8206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-168,"num_objects_misplaced":-271,"num_objects_unfound":0,"num_objects_dirty":-4103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4103,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":439,"num_bytes_recovered":1110016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002938"},"pg_stats":[{"pgid":"2.f","version":"46'11915","reported_seq":12292,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.643197+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:34.643197+0000","last_peered":"2024-09-10T15:57:34.643197+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:34.643197+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36270080,"num_objects":8855,"num_object_clones":0,"num_object_copies":17710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11915,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5032},{"shards":"0,2","objects":2443}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12080","reported_seq":12188,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:35.007102+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:35.007102+0000","last_peered":"2024-09-10T15:57:35.007102+0000","last_clean":"2024-09-10T15:57:35.007102+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:35.007102+0000","last_undegraded":"2024-09-10T15:57:35.007102+0000","last_fullsized":"2024-09-10T15:57:35.007102+0000","mapping_epoch":36,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36839424,"num_objects":8994,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10416,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11856","reported_seq":12105,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:35.016266+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:35.016266+0000","last_peered":"2024-09-10T15:57:35.016266+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:35.016266+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35934232,"num_objects":8774,"num_object_clones":0,"num_object_copies":17548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3741,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8774,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10124,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2261},{"shards":"2","objects":5034}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'11896","reported_seq":11976,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.938157+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:34.938157+0000","last_peered":"2024-09-10T15:57:34.938157+0000","last_clean":"2024-09-10T15:57:34.938157+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:34.938157+0000","last_undegraded":"2024-09-10T15:57:34.938157+0000","last_fullsized":"2024-09-10T15:57:34.938157+0000","mapping_epoch":36,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35987456,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11896,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11648","reported_seq":11775,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.639470+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:34.639470+0000","last_peered":"2024-09-10T15:57:34.639470+0000","last_clean":"2024-09-10T15:57:34.639470+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:34.639470+0000","last_undegraded":"2024-09-10T15:57:34.639470+0000","last_fullsized":"2024-09-10T15:57:34.639470+0000","mapping_epoch":35,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8719,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'11913","reported_seq":17506,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:34.705330+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:34.705330+0000","last_peered":"2024-09-10T15:57:34.705330+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:34.705330+0000","last_undegraded":"2024-09-10T15:57:34.705330+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36675584,"num_objects":8954,"num_object_clones":0,"num_object_copies":17908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1141,"num_objects_unfound":0,"num_objects_dirty":8954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8631,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2729,"num_bytes_recovered":11169792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7813},{"shards":"0,2","objects":1141}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11653","reported_seq":12016,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.672951+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:34.672951+0000","last_peered":"2024-09-10T15:57:34.672951+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:34.672951+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3725,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8303,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4910},{"shards":"0,2","objects":2327}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'11996","reported_seq":12128,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.546519+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:34.546519+0000","last_peered":"2024-09-10T15:57:34.546519+0000","last_clean":"2024-09-10T15:57:34.546519+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:34.546519+0000","last_undegraded":"2024-09-10T15:57:34.546519+0000","last_fullsized":"2024-09-10T15:57:34.546519+0000","mapping_epoch":35,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36528128,"num_objects":8918,"num_object_clones":0,"num_object_copies":17836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8755,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'11955","reported_seq":20083,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:34.687617+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:34.687617+0000","last_peered":"2024-09-10T15:57:34.687617+0000","last_clean":"2024-09-10T15:57:34.687617+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:34.687617+0000","last_undegraded":"2024-09-10T15:57:34.687617+0000","last_fullsized":"2024-09-10T15:57:34.687617+0000","mapping_epoch":45,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35897344,"num_objects":8764,"num_object_clones":0,"num_object_copies":17528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9932,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11786","reported_seq":12214,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.616637+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:34.616637+0000","last_peered":"2024-09-10T15:57:34.616637+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:34.616637+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36012032,"num_objects":8792,"num_object_clones":0,"num_object_copies":17584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3714,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8650,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5080},{"shards":"0,2","objects":2370}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'11917","reported_seq":12176,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:35.032155+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:35.032155+0000","last_peered":"2024-09-10T15:57:35.032155+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:35.032155+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36323328,"num_objects":8868,"num_object_clones":0,"num_object_copies":17736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9891,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2310},{"shards":"2","objects":5103}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11721","reported_seq":11952,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.619118+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:34.619118+0000","last_peered":"2024-09-10T15:57:34.619118+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:34.619118+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35778560,"num_objects":8735,"num_object_clones":0,"num_object_copies":17470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3779,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9590,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2338},{"shards":"2","objects":4958}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'11949","reported_seq":12391,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.593757+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:34.593757+0000","last_peered":"2024-09-10T15:57:34.593757+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:34.593757+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36474880,"num_objects":8905,"num_object_clones":0,"num_object_copies":17810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9141,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5229},{"shards":"0,2","objects":2312}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'11945","reported_seq":12365,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.652863+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:34.652863+0000","last_peered":"2024-09-10T15:57:34.652863+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:34.652863+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35934208,"num_objects":8773,"num_object_clones":0,"num_object_copies":17546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3792,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8363,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4984},{"shards":"0,2","objects":2335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'11874","reported_seq":12310,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.659148+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:34.659148+0000","last_peered":"2024-09-10T15:57:34.659148+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:34.659148+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36421632,"num_objects":8892,"num_object_clones":0,"num_object_copies":17784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8480,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5163},{"shards":"0,2","objects":2388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12092","reported_seq":12481,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:34.563139+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:34.563139+0000","last_peered":"2024-09-10T15:57:34.563139+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:34.563139+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36889059,"num_objects":9007,"num_object_clones":0,"num_object_copies":18014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3881,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8911,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5129},{"shards":"0,2","objects":2466}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":578892283,"num_objects":141333,"num_object_clones":0,"num_object_copies":282666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37630,"num_objects_misplaced":1141,"num_objects_unfound":0,"num_objects_dirty":141333,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":151717,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7876,"num_bytes_recovered":27529216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1247608832,"data_stored":1247524815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626486,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618256,"kb_used_data":325632,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93753584,"statfs":{"total":96636764160,"available":96003670016,"internally_reserved":0,"allocated":333447168,"data_stored":333112009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299626158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149769,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":858004,"kb_used_data":487612,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93513836,"statfs":{"total":96636764160,"available":95758168064,"internally_reserved":0,"allocated":499314688,"data_stored":499048635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":379252397},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,2,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705932,"kb_used_data":407740,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93665908,"statfs":{"total":96636764160,"available":95913889792,"internally_reserved":0,"allocated":417525760,"data_stored":417255635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,1,1,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":498233344,"data_stored":498229731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":332906496,"data_stored":332833777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":416468992,"data_stored":416461307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:40.544 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:57:40.544 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'11915", 'reported_seq': 12292, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:34.643197+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:57:34.643197+0000', 'last_peered': '2024-09-10T15:57:34.643197+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:57:34.643197+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'1900", 'ondisk_log_start': "37'1900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36270080, 'num_objects': 8855, 'num_object_clones': 0, 'num_object_copies': 17710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3823, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8855, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11915, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 150, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5032}, {'shards': '0,2', 'objects': 2443}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:40.544 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:57:40.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'11856", 'reported_seq': 12105, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:35.016266+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:57:35.016266+0000', 'last_peered': '2024-09-10T15:57:35.016266+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:57:35.016266+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'1800", 'ondisk_log_start': "37'1800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35934232, 'num_objects': 8774, 'num_object_clones': 0, 'num_object_copies': 17548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3741, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8774, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10124, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2261}, {'shards': '2', 'objects': 5034}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:40.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:57:40.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'11913", 'reported_seq': 17506, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:57:34.705330+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:57:34.705330+0000', 'last_peered': '2024-09-10T15:57:34.705330+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:57:34.705330+0000', 'last_undegraded': '2024-09-10T15:57:34.705330+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'1900", 'ondisk_log_start': "37'1900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36675584, 'num_objects': 8954, 'num_object_clones': 0, 'num_object_copies': 17908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1141, 'num_objects_unfound': 0, 'num_objects_dirty': 8954, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8631, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2729, 'num_bytes_recovered': 11169792, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7813}, {'shards': '0,2', 'objects': 1141}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:40.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:57:40.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'11653", 'reported_seq': 12016, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:34.672951+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:57:34.672951+0000', 'last_peered': '2024-09-10T15:57:34.672951+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:57:34.672951+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35360768, 'num_objects': 8633, 'num_object_clones': 0, 'num_object_copies': 17266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3725, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8303, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 112, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4910}, {'shards': '0,2', 'objects': 2327}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:40.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:57:40.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'11786", 'reported_seq': 12214, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:34.616637+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:57:34.616637+0000', 'last_peered': '2024-09-10T15:57:34.616637+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:57:34.616637+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36012032, 'num_objects': 8792, 'num_object_clones': 0, 'num_object_copies': 17584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3714, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8792, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8650, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 151, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5080}, {'shards': '0,2', 'objects': 2370}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:40.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:57:40.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'11917", 'reported_seq': 12176, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:35.032155+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:57:35.032155+0000', 'last_peered': '2024-09-10T15:57:35.032155+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:57:35.032155+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'1900", 'ondisk_log_start': "37'1900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36323328, 'num_objects': 8868, 'num_object_clones': 0, 'num_object_copies': 17736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3767, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9891, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2310}, {'shards': '2', 'objects': 5103}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:40.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:57:40.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'11721", 'reported_seq': 11952, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:34.619118+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:57:34.619118+0000', 'last_peered': '2024-09-10T15:57:34.619118+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:57:34.619118+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35778560, 'num_objects': 8735, 'num_object_clones': 0, 'num_object_copies': 17470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3779, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8735, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9590, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2338}, {'shards': '2', 'objects': 4958}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:40.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:57:40.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'11949", 'reported_seq': 12391, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:34.593757+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:57:34.593757+0000', 'last_peered': '2024-09-10T15:57:34.593757+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:57:34.593757+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'1900", 'ondisk_log_start': "37'1900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36474880, 'num_objects': 8905, 'num_object_clones': 0, 'num_object_copies': 17810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3677, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9141, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 157, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5229}, {'shards': '0,2', 'objects': 2312}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:40.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:57:40.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'11945", 'reported_seq': 12365, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:34.652863+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:57:34.652863+0000', 'last_peered': '2024-09-10T15:57:34.652863+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:57:34.652863+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'1900", 'ondisk_log_start': "37'1900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35934208, 'num_objects': 8773, 'num_object_clones': 0, 'num_object_copies': 17546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3792, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8773, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8363, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 128, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4984}, {'shards': '0,2', 'objects': 2335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:40.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:57:40.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'11874", 'reported_seq': 12310, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:34.659148+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:57:34.659148+0000', 'last_peered': '2024-09-10T15:57:34.659148+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:57:34.659148+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'1800", 'ondisk_log_start': "37'1800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36421632, 'num_objects': 8892, 'num_object_clones': 0, 'num_object_copies': 17784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3731, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8892, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8480, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 149, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5163}, {'shards': '0,2', 'objects': 2388}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:40.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:57:40.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'12092", 'reported_seq': 12481, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:34.563139+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:57:34.563139+0000', 'last_peered': '2024-09-10T15:57:34.563139+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:57:34.563139+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36889059, 'num_objects': 9007, 'num_object_clones': 0, 'num_object_copies': 18014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3881, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9007, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8911, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 125, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5129}, {'shards': '0,2', 'objects': 2466}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:40.547 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:57:41.222 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:41.222 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37511/278286 objects degraded (13.479%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":387,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":139143,"data_bytes":569913870,"bytes_used":2226356224,"bytes_avail":287683936256,"bytes_total":289910292480,"degraded_objects":37511,"degraded_total":278286,"degraded_ratio":0.1347929827587446,"misplaced_objects":1013,"misplaced_total":278286,"misplaced_ratio":0.0036401399998562626,"recovering_objects_per_sec":57,"recovering_bytes_per_sec":136155,"recovering_keys_per_sec":0,"num_objects_recovered":686,"num_bytes_recovered":1634304,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":524},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:57:41.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:57:41.245 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:41.782 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:41.783 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:41.808 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-10T15:57:40.520610+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":569913870,"num_objects":139143,"num_object_clones":0,"num_object_copies":278286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37511,"num_objects_misplaced":1013,"num_objects_unfound":0,"num_objects_dirty":139143,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153911,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8125,"num_bytes_recovered":28053523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174176,"kb_used_data":1208816,"kb_used_omap":12,"kb_used_meta":965299,"kb_avail":280941344,"statfs":{"total":289910292480,"available":287683936256,"internally_reserved":0,"allocated":1237827584,"data_stored":1236956247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":988466181},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,7,5,9,4,1,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25784320,"num_objects":-6295,"num_object_clones":0,"num_object_copies":-12590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-287,"num_objects_misplaced":-399,"num_objects_unfound":0,"num_objects_dirty":-6295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6295,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":1634304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003188"},"pg_stats":[{"pgid":"2.f","version":"46'12063","reported_seq":12458,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.661335+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:39.661335+0000","last_peered":"2024-09-10T15:57:39.661335+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:39.661335+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35663872,"num_objects":8707,"num_object_clones":0,"num_object_copies":17414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3814,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12063,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4893},{"shards":"0,2","objects":2295}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12216","reported_seq":12321,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.903220+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:39.903220+0000","last_peered":"2024-09-10T15:57:39.903220+0000","last_clean":"2024-09-10T15:57:39.903220+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:39.903220+0000","last_undegraded":"2024-09-10T15:57:39.903220+0000","last_fullsized":"2024-09-10T15:57:39.903220+0000","mapping_epoch":36,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36282368,"num_objects":8858,"num_object_clones":0,"num_object_copies":17716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10552,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11979","reported_seq":12230,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:40.019001+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:40.019001+0000","last_peered":"2024-09-10T15:57:40.019001+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:40.019001+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35430424,"num_objects":8651,"num_object_clones":0,"num_object_copies":17302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8651,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10247,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2138},{"shards":"2","objects":4912}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12022","reported_seq":12102,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.968089+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:39.968089+0000","last_peered":"2024-09-10T15:57:39.968089+0000","last_clean":"2024-09-10T15:57:39.968089+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:39.968089+0000","last_undegraded":"2024-09-10T15:57:39.968089+0000","last_fullsized":"2024-09-10T15:57:39.968089+0000","mapping_epoch":36,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35471360,"num_objects":8660,"num_object_clones":0,"num_object_copies":17320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12022,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11790","reported_seq":11917,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.644476+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:39.644476+0000","last_peered":"2024-09-10T15:57:39.644476+0000","last_clean":"2024-09-10T15:57:39.644476+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:39.644476+0000","last_undegraded":"2024-09-10T15:57:39.644476+0000","last_fullsized":"2024-09-10T15:57:39.644476+0000","mapping_epoch":35,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8861,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12068","reported_seq":17916,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:39.701924+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:39.701924+0000","last_peered":"2024-09-10T15:57:39.701924+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:39.701924+0000","last_undegraded":"2024-09-10T15:57:39.701924+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36040704,"num_objects":8799,"num_object_clones":0,"num_object_copies":17598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1013,"num_objects_unfound":0,"num_objects_dirty":8799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8786,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2857,"num_bytes_recovered":11694080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7786},{"shards":"0,2","objects":1013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11785","reported_seq":12200,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.618437+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:39.618437+0000","last_peered":"2024-09-10T15:57:39.618437+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:39.618437+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34820096,"num_objects":8501,"num_object_clones":0,"num_object_copies":17002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3699,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8435,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4804},{"shards":"0,2","objects":2195}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12143","reported_seq":12275,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.586956+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:39.586956+0000","last_peered":"2024-09-10T15:57:39.586956+0000","last_clean":"2024-09-10T15:57:39.586956+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:39.586956+0000","last_undegraded":"2024-09-10T15:57:39.586956+0000","last_fullsized":"2024-09-10T15:57:39.586956+0000","mapping_epoch":35,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35926016,"num_objects":8771,"num_object_clones":0,"num_object_copies":17542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8902,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12088","reported_seq":20216,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.720918+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:39.720918+0000","last_peered":"2024-09-10T15:57:39.720918+0000","last_clean":"2024-09-10T15:57:39.720918+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:39.720918+0000","last_undegraded":"2024-09-10T15:57:39.720918+0000","last_fullsized":"2024-09-10T15:57:39.720918+0000","mapping_epoch":45,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35352576,"num_objects":8631,"num_object_clones":0,"num_object_copies":17262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10065,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11902","reported_seq":12365,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.671435+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:39.671435+0000","last_peered":"2024-09-10T15:57:39.671435+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:39.671435+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35536896,"num_objects":8676,"num_object_clones":0,"num_object_copies":17352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8766,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4981},{"shards":"0,2","objects":2254}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12058","reported_seq":12328,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:40.027103+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:40.027103+0000","last_peered":"2024-09-10T15:57:40.027103+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:40.027103+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35745792,"num_objects":8727,"num_object_clones":0,"num_object_copies":17454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10032,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2169},{"shards":"2","objects":4968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11838","reported_seq":12073,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.890114+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:39.890114+0000","last_peered":"2024-09-10T15:57:39.890114+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:39.890114+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35299328,"num_objects":8618,"num_object_clones":0,"num_object_copies":17236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3777,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9707,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2221},{"shards":"2","objects":4843}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12085","reported_seq":12551,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.555656+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:39.555656+0000","last_peered":"2024-09-10T15:57:39.555656+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:39.555656+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35917824,"num_objects":8769,"num_object_clones":0,"num_object_copies":17538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9277,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5105},{"shards":"0,2","objects":2176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12075","reported_seq":12512,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.685342+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:39.685342+0000","last_peered":"2024-09-10T15:57:39.685342+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:39.685342+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35401728,"num_objects":8643,"num_object_clones":0,"num_object_copies":17286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8493,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4863},{"shards":"0,2","objects":2205}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12019","reported_seq":12497,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.671088+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:39.671088+0000","last_peered":"2024-09-10T15:57:39.671088+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:39.671088+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35827712,"num_objects":8747,"num_object_clones":0,"num_object_copies":17494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8625,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5039},{"shards":"0,2","objects":2243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12257","reported_seq":12676,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.703893+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:39.703893+0000","last_peered":"2024-09-10T15:57:39.703893+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:39.703893+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36213219,"num_objects":8842,"num_object_clones":0,"num_object_copies":17684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9076,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4980},{"shards":"0,2","objects":2301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":569913851,"num_objects":139141,"num_object_clones":0,"num_object_copies":278282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37511,"num_objects_misplaced":1013,"num_objects_unfound":0,"num_objects_dirty":139141,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153909,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8123,"num_bytes_recovered":28053504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1235689472,"data_stored":1235605455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160888,"ondisk_log_size":160888,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626487,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609296,"kb_used_data":324992,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93762544,"statfs":{"total":96636764160,"available":96012845056,"internally_reserved":0,"allocated":332791808,"data_stored":332456649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291106478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149770,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859740,"kb_used_data":481036,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93512100,"statfs":{"total":96636764160,"available":95756390400,"internally_reserved":0,"allocated":492580864,"data_stored":492314811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":387772077},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,2,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705140,"kb_used_data":402788,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93666700,"statfs":{"total":96636764160,"available":95914700800,"internally_reserved":0,"allocated":412454912,"data_stored":412184787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2,1,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":491409408,"data_stored":491405795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":332251136,"data_stored":332178417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":412028928,"data_stored":412021243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:44.810 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:45.348 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:45.348 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:45.378 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-10T15:57:44.522941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":569913870,"num_objects":139143,"num_object_clones":0,"num_object_copies":278286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37511,"num_objects_misplaced":1013,"num_objects_unfound":0,"num_objects_dirty":139143,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153911,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8125,"num_bytes_recovered":28053523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174176,"kb_used_data":1208816,"kb_used_omap":12,"kb_used_meta":965299,"kb_avail":280941344,"statfs":{"total":289910292480,"available":287683936256,"internally_reserved":0,"allocated":1237827584,"data_stored":1236956247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":988466181},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,7,5,9,4,1,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17965056,"num_objects":-4386,"num_object_clones":0,"num_object_copies":-8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-187,"num_objects_misplaced":-295,"num_objects_unfound":0,"num_objects_dirty":-4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4386,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":1208320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004652"},"pg_stats":[{"pgid":"2.f","version":"46'12063","reported_seq":12458,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.661335+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:39.661335+0000","last_peered":"2024-09-10T15:57:39.661335+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:39.661335+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35663872,"num_objects":8707,"num_object_clones":0,"num_object_copies":17414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3814,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12063,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4893},{"shards":"0,2","objects":2295}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12216","reported_seq":12321,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.903220+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:39.903220+0000","last_peered":"2024-09-10T15:57:39.903220+0000","last_clean":"2024-09-10T15:57:39.903220+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:39.903220+0000","last_undegraded":"2024-09-10T15:57:39.903220+0000","last_fullsized":"2024-09-10T15:57:39.903220+0000","mapping_epoch":36,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36282368,"num_objects":8858,"num_object_clones":0,"num_object_copies":17716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10552,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11979","reported_seq":12230,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:40.019001+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:40.019001+0000","last_peered":"2024-09-10T15:57:40.019001+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:40.019001+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35430424,"num_objects":8651,"num_object_clones":0,"num_object_copies":17302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8651,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10247,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2138},{"shards":"2","objects":4912}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12022","reported_seq":12102,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.968089+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:39.968089+0000","last_peered":"2024-09-10T15:57:39.968089+0000","last_clean":"2024-09-10T15:57:39.968089+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:39.968089+0000","last_undegraded":"2024-09-10T15:57:39.968089+0000","last_fullsized":"2024-09-10T15:57:39.968089+0000","mapping_epoch":36,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35471360,"num_objects":8660,"num_object_clones":0,"num_object_copies":17320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12022,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11790","reported_seq":11917,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.644476+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:39.644476+0000","last_peered":"2024-09-10T15:57:39.644476+0000","last_clean":"2024-09-10T15:57:39.644476+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:39.644476+0000","last_undegraded":"2024-09-10T15:57:39.644476+0000","last_fullsized":"2024-09-10T15:57:39.644476+0000","mapping_epoch":35,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8861,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12068","reported_seq":17916,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:39.701924+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:39.701924+0000","last_peered":"2024-09-10T15:57:39.701924+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:39.701924+0000","last_undegraded":"2024-09-10T15:57:39.701924+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36040704,"num_objects":8799,"num_object_clones":0,"num_object_copies":17598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1013,"num_objects_unfound":0,"num_objects_dirty":8799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8786,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2857,"num_bytes_recovered":11694080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7786},{"shards":"0,2","objects":1013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11785","reported_seq":12200,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.618437+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:39.618437+0000","last_peered":"2024-09-10T15:57:39.618437+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:39.618437+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34820096,"num_objects":8501,"num_object_clones":0,"num_object_copies":17002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3699,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8435,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4804},{"shards":"0,2","objects":2195}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12143","reported_seq":12275,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.586956+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:39.586956+0000","last_peered":"2024-09-10T15:57:39.586956+0000","last_clean":"2024-09-10T15:57:39.586956+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:39.586956+0000","last_undegraded":"2024-09-10T15:57:39.586956+0000","last_fullsized":"2024-09-10T15:57:39.586956+0000","mapping_epoch":35,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35926016,"num_objects":8771,"num_object_clones":0,"num_object_copies":17542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8902,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12088","reported_seq":20216,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.720918+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:39.720918+0000","last_peered":"2024-09-10T15:57:39.720918+0000","last_clean":"2024-09-10T15:57:39.720918+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:39.720918+0000","last_undegraded":"2024-09-10T15:57:39.720918+0000","last_fullsized":"2024-09-10T15:57:39.720918+0000","mapping_epoch":45,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35352576,"num_objects":8631,"num_object_clones":0,"num_object_copies":17262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10065,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11902","reported_seq":12365,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.671435+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:39.671435+0000","last_peered":"2024-09-10T15:57:39.671435+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:39.671435+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35536896,"num_objects":8676,"num_object_clones":0,"num_object_copies":17352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8766,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4981},{"shards":"0,2","objects":2254}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12058","reported_seq":12328,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:40.027103+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:40.027103+0000","last_peered":"2024-09-10T15:57:40.027103+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:40.027103+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35745792,"num_objects":8727,"num_object_clones":0,"num_object_copies":17454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10032,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2169},{"shards":"2","objects":4968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11838","reported_seq":12073,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.890114+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:39.890114+0000","last_peered":"2024-09-10T15:57:39.890114+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:39.890114+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35299328,"num_objects":8618,"num_object_clones":0,"num_object_copies":17236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3777,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9707,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2221},{"shards":"2","objects":4843}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12085","reported_seq":12551,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.555656+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:39.555656+0000","last_peered":"2024-09-10T15:57:39.555656+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:39.555656+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35917824,"num_objects":8769,"num_object_clones":0,"num_object_copies":17538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9277,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5105},{"shards":"0,2","objects":2176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12075","reported_seq":12512,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.685342+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:39.685342+0000","last_peered":"2024-09-10T15:57:39.685342+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:39.685342+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35401728,"num_objects":8643,"num_object_clones":0,"num_object_copies":17286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8493,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4863},{"shards":"0,2","objects":2205}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12019","reported_seq":12497,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.671088+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:39.671088+0000","last_peered":"2024-09-10T15:57:39.671088+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:39.671088+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35827712,"num_objects":8747,"num_object_clones":0,"num_object_copies":17494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8625,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5039},{"shards":"0,2","objects":2243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12257","reported_seq":12676,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.703893+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:39.703893+0000","last_peered":"2024-09-10T15:57:39.703893+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:39.703893+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36213219,"num_objects":8842,"num_object_clones":0,"num_object_copies":17684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9076,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4980},{"shards":"0,2","objects":2301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":569913851,"num_objects":139141,"num_object_clones":0,"num_object_copies":278282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37511,"num_objects_misplaced":1013,"num_objects_unfound":0,"num_objects_dirty":139141,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153909,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8123,"num_bytes_recovered":28053504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1235689472,"data_stored":1235605455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160888,"ondisk_log_size":160888,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626487,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609296,"kb_used_data":324992,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93762544,"statfs":{"total":96636764160,"available":96012845056,"internally_reserved":0,"allocated":332791808,"data_stored":332456649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291106478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149770,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859740,"kb_used_data":481036,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93512100,"statfs":{"total":96636764160,"available":95756390400,"internally_reserved":0,"allocated":492580864,"data_stored":492314811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":387772077},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,2,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705140,"kb_used_data":402788,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93666700,"statfs":{"total":96636764160,"available":95914700800,"internally_reserved":0,"allocated":412454912,"data_stored":412184787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2,1,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":491409408,"data_stored":491405795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":332251136,"data_stored":332178417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":412028928,"data_stored":412021243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:45.380 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:45.915 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:45.916 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:45.927 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-10T15:57:44.522941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":569913870,"num_objects":139143,"num_object_clones":0,"num_object_copies":278286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37511,"num_objects_misplaced":1013,"num_objects_unfound":0,"num_objects_dirty":139143,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153911,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8125,"num_bytes_recovered":28053523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2174176,"kb_used_data":1208816,"kb_used_omap":12,"kb_used_meta":965299,"kb_avail":280941344,"statfs":{"total":289910292480,"available":287683936256,"internally_reserved":0,"allocated":1237827584,"data_stored":1236956247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":988466181},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,7,5,9,4,1,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17965056,"num_objects":-4386,"num_object_clones":0,"num_object_copies":-8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-187,"num_objects_misplaced":-295,"num_objects_unfound":0,"num_objects_dirty":-4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4386,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":1208320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004652"},"pg_stats":[{"pgid":"2.f","version":"46'12063","reported_seq":12458,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.661335+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:39.661335+0000","last_peered":"2024-09-10T15:57:39.661335+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:39.661335+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35663872,"num_objects":8707,"num_object_clones":0,"num_object_copies":17414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3814,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12063,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4893},{"shards":"0,2","objects":2295}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12216","reported_seq":12321,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.903220+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:39.903220+0000","last_peered":"2024-09-10T15:57:39.903220+0000","last_clean":"2024-09-10T15:57:39.903220+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:39.903220+0000","last_undegraded":"2024-09-10T15:57:39.903220+0000","last_fullsized":"2024-09-10T15:57:39.903220+0000","mapping_epoch":36,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36282368,"num_objects":8858,"num_object_clones":0,"num_object_copies":17716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10552,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'11979","reported_seq":12230,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:40.019001+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:40.019001+0000","last_peered":"2024-09-10T15:57:40.019001+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:40.019001+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35430424,"num_objects":8651,"num_object_clones":0,"num_object_copies":17302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8651,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10247,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2138},{"shards":"2","objects":4912}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12022","reported_seq":12102,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.968089+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:39.968089+0000","last_peered":"2024-09-10T15:57:39.968089+0000","last_clean":"2024-09-10T15:57:39.968089+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:39.968089+0000","last_undegraded":"2024-09-10T15:57:39.968089+0000","last_fullsized":"2024-09-10T15:57:39.968089+0000","mapping_epoch":36,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35471360,"num_objects":8660,"num_object_clones":0,"num_object_copies":17320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12022,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11790","reported_seq":11917,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.644476+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:39.644476+0000","last_peered":"2024-09-10T15:57:39.644476+0000","last_clean":"2024-09-10T15:57:39.644476+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:39.644476+0000","last_undegraded":"2024-09-10T15:57:39.644476+0000","last_fullsized":"2024-09-10T15:57:39.644476+0000","mapping_epoch":35,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8861,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12068","reported_seq":17916,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:39.701924+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:39.701924+0000","last_peered":"2024-09-10T15:57:39.701924+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:39.701924+0000","last_undegraded":"2024-09-10T15:57:39.701924+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36040704,"num_objects":8799,"num_object_clones":0,"num_object_copies":17598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1013,"num_objects_unfound":0,"num_objects_dirty":8799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8786,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2857,"num_bytes_recovered":11694080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7786},{"shards":"0,2","objects":1013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11785","reported_seq":12200,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.618437+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:39.618437+0000","last_peered":"2024-09-10T15:57:39.618437+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:39.618437+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34820096,"num_objects":8501,"num_object_clones":0,"num_object_copies":17002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3699,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8435,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4804},{"shards":"0,2","objects":2195}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12143","reported_seq":12275,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.586956+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:39.586956+0000","last_peered":"2024-09-10T15:57:39.586956+0000","last_clean":"2024-09-10T15:57:39.586956+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:39.586956+0000","last_undegraded":"2024-09-10T15:57:39.586956+0000","last_fullsized":"2024-09-10T15:57:39.586956+0000","mapping_epoch":35,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35926016,"num_objects":8771,"num_object_clones":0,"num_object_copies":17542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8902,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12088","reported_seq":20216,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:39.720918+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:39.720918+0000","last_peered":"2024-09-10T15:57:39.720918+0000","last_clean":"2024-09-10T15:57:39.720918+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:39.720918+0000","last_undegraded":"2024-09-10T15:57:39.720918+0000","last_fullsized":"2024-09-10T15:57:39.720918+0000","mapping_epoch":45,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35352576,"num_objects":8631,"num_object_clones":0,"num_object_copies":17262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10065,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'11902","reported_seq":12365,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.671435+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:39.671435+0000","last_peered":"2024-09-10T15:57:39.671435+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:39.671435+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35536896,"num_objects":8676,"num_object_clones":0,"num_object_copies":17352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8766,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4981},{"shards":"0,2","objects":2254}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12058","reported_seq":12328,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:40.027103+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:40.027103+0000","last_peered":"2024-09-10T15:57:40.027103+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:40.027103+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35745792,"num_objects":8727,"num_object_clones":0,"num_object_copies":17454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10032,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2169},{"shards":"2","objects":4968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11838","reported_seq":12073,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.890114+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:39.890114+0000","last_peered":"2024-09-10T15:57:39.890114+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:39.890114+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35299328,"num_objects":8618,"num_object_clones":0,"num_object_copies":17236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3777,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9707,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2221},{"shards":"2","objects":4843}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12085","reported_seq":12551,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.555656+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:39.555656+0000","last_peered":"2024-09-10T15:57:39.555656+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:39.555656+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35917824,"num_objects":8769,"num_object_clones":0,"num_object_copies":17538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9277,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5105},{"shards":"0,2","objects":2176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12075","reported_seq":12512,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.685342+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:39.685342+0000","last_peered":"2024-09-10T15:57:39.685342+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:39.685342+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35401728,"num_objects":8643,"num_object_clones":0,"num_object_copies":17286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8493,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4863},{"shards":"0,2","objects":2205}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12019","reported_seq":12497,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.671088+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:39.671088+0000","last_peered":"2024-09-10T15:57:39.671088+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:39.671088+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35827712,"num_objects":8747,"num_object_clones":0,"num_object_copies":17494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8625,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5039},{"shards":"0,2","objects":2243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12257","reported_seq":12676,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:39.703893+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:39.703893+0000","last_peered":"2024-09-10T15:57:39.703893+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:39.703893+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36213219,"num_objects":8842,"num_object_clones":0,"num_object_copies":17684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9076,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4980},{"shards":"0,2","objects":2301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":569913851,"num_objects":139141,"num_object_clones":0,"num_object_copies":278282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37511,"num_objects_misplaced":1013,"num_objects_unfound":0,"num_objects_dirty":139141,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153909,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8123,"num_bytes_recovered":28053504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1235689472,"data_stored":1235605455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160888,"ondisk_log_size":160888,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626487,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609296,"kb_used_data":324992,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93762544,"statfs":{"total":96636764160,"available":96012845056,"internally_reserved":0,"allocated":332791808,"data_stored":332456649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291106478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149770,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859740,"kb_used_data":481036,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93512100,"statfs":{"total":96636764160,"available":95756390400,"internally_reserved":0,"allocated":492580864,"data_stored":492314811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":387772077},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,2,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705140,"kb_used_data":402788,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93666700,"statfs":{"total":96636764160,"available":95914700800,"internally_reserved":0,"allocated":412454912,"data_stored":412184787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2,1,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":491409408,"data_stored":491405795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":332251136,"data_stored":332178417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":412028928,"data_stored":412021243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:45.928 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:57:45.928 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'12063", 'reported_seq': 12458, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:39.661335+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:57:39.661335+0000', 'last_peered': '2024-09-10T15:57:39.661335+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:57:39.661335+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35663872, 'num_objects': 8707, 'num_object_clones': 0, 'num_object_copies': 17414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3814, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8707, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12063, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 159, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4893}, {'shards': '0,2', 'objects': 2295}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:45.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:57:45.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'11979", 'reported_seq': 12230, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:40.019001+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:57:40.019001+0000', 'last_peered': '2024-09-10T15:57:40.019001+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:57:40.019001+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'1900", 'ondisk_log_start': "37'1900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35430424, 'num_objects': 8651, 'num_object_clones': 0, 'num_object_copies': 17302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3740, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8651, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10247, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 63, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2138}, {'shards': '2', 'objects': 4912}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:45.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:57:45.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'12068", 'reported_seq': 17916, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:57:39.701924+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:57:39.701924+0000', 'last_peered': '2024-09-10T15:57:39.701924+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:57:39.701924+0000', 'last_undegraded': '2024-09-10T15:57:39.701924+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36040704, 'num_objects': 8799, 'num_object_clones': 0, 'num_object_copies': 17598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1013, 'num_objects_unfound': 0, 'num_objects_dirty': 8799, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8786, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2857, 'num_bytes_recovered': 11694080, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7786}, {'shards': '0,2', 'objects': 1013}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:45.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:57:45.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'11785", 'reported_seq': 12200, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:39.618437+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:57:39.618437+0000', 'last_peered': '2024-09-10T15:57:39.618437+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:57:39.618437+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34820096, 'num_objects': 8501, 'num_object_clones': 0, 'num_object_copies': 17002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3699, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8501, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8435, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 138, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4804}, {'shards': '0,2', 'objects': 2195}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:45.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:57:45.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'11902", 'reported_seq': 12365, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:39.671435+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:57:39.671435+0000', 'last_peered': '2024-09-10T15:57:39.671435+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:57:39.671435+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'1900", 'ondisk_log_start': "37'1900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35536896, 'num_objects': 8676, 'num_object_clones': 0, 'num_object_copies': 17352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8676, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8766, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 168, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4981}, {'shards': '0,2', 'objects': 2254}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:45.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:57:45.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'12058", 'reported_seq': 12328, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:40.027103+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:57:40.027103+0000', 'last_peered': '2024-09-10T15:57:40.027103+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:57:40.027103+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35745792, 'num_objects': 8727, 'num_object_clones': 0, 'num_object_copies': 17454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3761, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8727, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10032, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 73, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2169}, {'shards': '2', 'objects': 4968}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:45.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:57:45.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'11838", 'reported_seq': 12073, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:39.890114+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:57:39.890114+0000', 'last_peered': '2024-09-10T15:57:39.890114+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:57:39.890114+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'1800", 'ondisk_log_start': "37'1800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35299328, 'num_objects': 8618, 'num_object_clones': 0, 'num_object_copies': 17236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3777, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8618, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9707, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 54, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2221}, {'shards': '2', 'objects': 4843}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:45.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:57:45.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'12085", 'reported_seq': 12551, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:39.555656+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:57:39.555656+0000', 'last_peered': '2024-09-10T15:57:39.555656+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:57:39.555656+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35917824, 'num_objects': 8769, 'num_object_clones': 0, 'num_object_copies': 17538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3665, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8769, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9277, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 169, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5105}, {'shards': '0,2', 'objects': 2176}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:45.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:57:45.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'12075", 'reported_seq': 12512, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:39.685342+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:57:39.685342+0000', 'last_peered': '2024-09-10T15:57:39.685342+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:57:39.685342+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35401728, 'num_objects': 8643, 'num_object_clones': 0, 'num_object_copies': 17286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3783, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8643, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8493, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 137, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4863}, {'shards': '0,2', 'objects': 2205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:45.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:57:45.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'12019", 'reported_seq': 12497, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:39.671088+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:57:39.671088+0000', 'last_peered': '2024-09-10T15:57:39.671088+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:57:39.671088+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35827712, 'num_objects': 8747, 'num_object_clones': 0, 'num_object_copies': 17494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3710, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8747, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8625, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 170, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5039}, {'shards': '0,2', 'objects': 2243}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:45.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:57:45.931 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'12257", 'reported_seq': 12676, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:39.703893+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:57:39.703893+0000', 'last_peered': '2024-09-10T15:57:39.703893+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:57:39.703893+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36213219, 'num_objects': 8842, 'num_object_clones': 0, 'num_object_copies': 17684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3865, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8842, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9076, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 141, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4980}, {'shards': '0,2', 'objects': 2301}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:45.931 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:57:46.494 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:46.495 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37511/278286 objects degraded (13.479%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":392,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":139143,"data_bytes":569913870,"bytes_used":2226356224,"bytes_avail":287683936256,"bytes_total":289910292480,"degraded_objects":37511,"degraded_total":278286,"degraded_ratio":0.1347929827587446,"misplaced_objects":1013,"misplaced_total":278286,"misplaced_ratio":0.0036401399998562626,"recovering_objects_per_sec":40,"recovering_bytes_per_sec":100654,"recovering_keys_per_sec":0,"num_objects_recovered":482,"num_bytes_recovered":1208320,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":365},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:57:46.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:57:46.524 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:47.051 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:47.053 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:47.077 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-10T15:57:46.523402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":560738830,"num_objects":136903,"num_object_clones":0,"num_object_copies":273806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37402,"num_objects_misplaced":851,"num_objects_unfound":0,"num_objects_dirty":136903,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156151,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8396,"num_bytes_recovered":28717075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2175636,"kb_used_data":1197792,"kb_used_omap":12,"kb_used_meta":977779,"kb_avail":280939884,"statfs":{"total":289910292480,"available":287682441216,"internally_reserved":0,"allocated":1226539008,"data_stored":1225667671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1001245701},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,0,2,15,5,0,3,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27140096,"num_objects":-6626,"num_object_clones":0,"num_object_copies":-13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-296,"num_objects_misplaced":-457,"num_objects_unfound":0,"num_objects_dirty":-6626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6626,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":753,"num_bytes_recovered":1871872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004672"},"pg_stats":[{"pgid":"2.f","version":"46'12216","reported_seq":12643,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:44.667897+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:44.667897+0000","last_peered":"2024-09-10T15:57:44.667897+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:44.667897+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35037184,"num_objects":8554,"num_object_clones":0,"num_object_copies":17108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3798,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12216,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4756},{"shards":"0,2","objects":2142}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12360","reported_seq":12468,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:45.017171+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:45.017171+0000","last_peered":"2024-09-10T15:57:45.017171+0000","last_clean":"2024-09-10T15:57:45.017171+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:45.017171+0000","last_undegraded":"2024-09-10T15:57:45.017171+0000","last_fullsized":"2024-09-10T15:57:45.017171+0000","mapping_epoch":36,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35692544,"num_objects":8714,"num_object_clones":0,"num_object_copies":17428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10696,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12118","reported_seq":12381,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:45.084832+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:45.084832+0000","last_peered":"2024-09-10T15:57:45.084832+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:45.084832+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34861080,"num_objects":8512,"num_object_clones":0,"num_object_copies":17024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8512,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10386,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1999},{"shards":"2","objects":4779}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12163","reported_seq":12243,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.989495+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:44.989495+0000","last_peered":"2024-09-10T15:57:44.989495+0000","last_clean":"2024-09-10T15:57:44.989495+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:44.989495+0000","last_undegraded":"2024-09-10T15:57:44.989495+0000","last_fullsized":"2024-09-10T15:57:44.989495+0000","mapping_epoch":36,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34893824,"num_objects":8519,"num_object_clones":0,"num_object_copies":17038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12163,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'11933","reported_seq":12060,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.649128+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:44.649128+0000","last_peered":"2024-09-10T15:57:44.649128+0000","last_clean":"2024-09-10T15:57:44.649128+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:44.649128+0000","last_undegraded":"2024-09-10T15:57:44.649128+0000","last_fullsized":"2024-09-10T15:57:44.649128+0000","mapping_epoch":35,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34398208,"num_objects":8398,"num_object_clones":0,"num_object_copies":16796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9004,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12186","reported_seq":18360,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:44.736554+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:44.736554+0000","last_peered":"2024-09-10T15:57:44.736554+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:44.736554+0000","last_undegraded":"2024-09-10T15:57:44.736554+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35557376,"num_objects":8681,"num_object_clones":0,"num_object_copies":17362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":851,"num_objects_unfound":0,"num_objects_dirty":8681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8904,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3019,"num_bytes_recovered":12357632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7830},{"shards":"0,2","objects":851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'11935","reported_seq":12392,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:44.746145+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:44.746145+0000","last_peered":"2024-09-10T15:57:44.746145+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:44.746145+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34205696,"num_objects":8351,"num_object_clones":0,"num_object_copies":16702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3678,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8585,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4675},{"shards":"0,2","objects":2045}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12291","reported_seq":12423,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.570456+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:44.570456+0000","last_peered":"2024-09-10T15:57:44.570456+0000","last_clean":"2024-09-10T15:57:44.570456+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:44.570456+0000","last_undegraded":"2024-09-10T15:57:44.570456+0000","last_fullsized":"2024-09-10T15:57:44.570456+0000","mapping_epoch":35,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35319808,"num_objects":8623,"num_object_clones":0,"num_object_copies":17246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9050,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12242","reported_seq":20370,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:44.671946+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:44.671946+0000","last_peered":"2024-09-10T15:57:44.671946+0000","last_clean":"2024-09-10T15:57:44.671946+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:44.671946+0000","last_undegraded":"2024-09-10T15:57:44.671946+0000","last_fullsized":"2024-09-10T15:57:44.671946+0000","mapping_epoch":45,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34721792,"num_objects":8477,"num_object_clones":0,"num_object_copies":16954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10219,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12042","reported_seq":12531,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:44.688072+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:44.688072+0000","last_peered":"2024-09-10T15:57:44.688072+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:44.688072+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34963456,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3684,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8906,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4854},{"shards":"0,2","objects":2114}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12190","reported_seq":12467,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:44.910060+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:44.910060+0000","last_peered":"2024-09-10T15:57:44.910060+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:44.910060+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35205120,"num_objects":8595,"num_object_clones":0,"num_object_copies":17190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10164,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2037},{"shards":"2","objects":4839}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'11969","reported_seq":12214,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:45.034885+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:45.034885+0000","last_peered":"2024-09-10T15:57:45.034885+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:45.034885+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34762752,"num_objects":8487,"num_object_clones":0,"num_object_copies":16974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3772,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9838,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2090},{"shards":"2","objects":4717}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12229","reported_seq":12725,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:44.670103+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:44.670103+0000","last_peered":"2024-09-10T15:57:44.670103+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:44.670103+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35328000,"num_objects":8625,"num_object_clones":0,"num_object_copies":17250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9421,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4976},{"shards":"0,2","objects":2032}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12204","reported_seq":12652,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:44.744131+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:44.744131+0000","last_peered":"2024-09-10T15:57:44.744131+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:44.744131+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3778,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8622,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4739},{"shards":"0,2","objects":2076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12151","reported_seq":12653,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:44.654840+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:44.654840+0000","last_peered":"2024-09-10T15:57:44.654840+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:44.654840+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35287040,"num_objects":8615,"num_object_clones":0,"num_object_copies":17230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3698,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8757,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4919},{"shards":"0,2","objects":2111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12399","reported_seq":12845,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:44.752698+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:44.752698+0000","last_peered":"2024-09-10T15:57:44.752698+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:44.752698+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35631587,"num_objects":8700,"num_object_clones":0,"num_object_copies":17400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3852,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9218,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4851},{"shards":"0,2","objects":2159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":560738811,"num_objects":136901,"num_object_clones":0,"num_object_copies":273802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37402,"num_objects_misplaced":851,"num_objects_unfound":0,"num_objects_dirty":136901,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156149,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8394,"num_bytes_recovered":28717056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1223700480,"data_stored":1223616463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626488,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612872,"kb_used_data":324408,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93758968,"statfs":{"total":96636764160,"available":96009183232,"internally_reserved":0,"allocated":332193792,"data_stored":331858633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295366318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149771,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":857176,"kb_used_data":474308,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93514664,"statfs":{"total":96636764160,"available":95759015936,"internally_reserved":0,"allocated":485691392,"data_stored":485425339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392031917},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,7,2,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705588,"kb_used_data":399076,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93666252,"statfs":{"total":96636764160,"available":95914242048,"internally_reserved":0,"allocated":408653824,"data_stored":408383699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,3,1,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484519936,"data_stored":484516323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331804672,"data_stored":331731953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":407375872,"data_stored":407368187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:50.079 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:50.640 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:50.640 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:50.663 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-10T15:57:50.524686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":552018446,"num_objects":134774,"num_object_clones":0,"num_object_copies":269548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37307,"num_objects_misplaced":694,"num_objects_unfound":0,"num_objects_dirty":134774,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158280,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8648,"num_bytes_recovered":29360147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160660,"ondisk_log_size":160660,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2163640,"kb_used_data":1185796,"kb_used_omap":12,"kb_used_meta":977779,"kb_avail":280951880,"statfs":{"total":289910292480,"available":287694725120,"internally_reserved":0,"allocated":1214255104,"data_stored":1213383767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1001245701},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,6,3,4,6,0,0,1,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26873856,"num_objects":-6561,"num_object_clones":0,"num_object_copies":-13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-323,"num_objects_misplaced":-447,"num_objects_unfound":0,"num_objects_dirty":-6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6561,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":770,"num_bytes_recovered":1830912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004716"},"pg_stats":[{"pgid":"2.f","version":"46'12359","reported_seq":12818,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.775356+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:49.775356+0000","last_peered":"2024-09-10T15:57:49.775356+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:49.775356+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34451456,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":1999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12504","reported_seq":12612,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:50.025169+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:50.025169+0000","last_peered":"2024-09-10T15:57:50.025169+0000","last_clean":"2024-09-10T15:57:50.025169+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:50.025169+0000","last_undegraded":"2024-09-10T15:57:50.025169+0000","last_fullsized":"2024-09-10T15:57:50.025169+0000","mapping_epoch":36,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35102720,"num_objects":8570,"num_object_clones":0,"num_object_copies":17140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10840,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12248","reported_seq":12516,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:50.119385+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:50.119385+0000","last_peered":"2024-09-10T15:57:50.119385+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:50.119385+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34328600,"num_objects":8382,"num_object_clones":0,"num_object_copies":16764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8382,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10516,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1869},{"shards":"2","objects":4652}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12302","reported_seq":12382,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:50.106913+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:50.106913+0000","last_peered":"2024-09-10T15:57:50.106913+0000","last_clean":"2024-09-10T15:57:50.106913+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:50.106913+0000","last_undegraded":"2024-09-10T15:57:50.106913+0000","last_fullsized":"2024-09-10T15:57:50.106913+0000","mapping_epoch":36,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34324480,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12302,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12069","reported_seq":12196,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:49.763251+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:49.763251+0000","last_peered":"2024-09-10T15:57:49.763251+0000","last_clean":"2024-09-10T15:57:49.763251+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:49.763251+0000","last_undegraded":"2024-09-10T15:57:49.763251+0000","last_fullsized":"2024-09-10T15:57:49.763251+0000","mapping_epoch":35,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33841152,"num_objects":8262,"num_object_clones":0,"num_object_copies":16524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9140,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12309","reported_seq":18797,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:49.745169+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:49.745169+0000","last_peered":"2024-09-10T15:57:49.745169+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:49.745169+0000","last_undegraded":"2024-09-10T15:57:49.745169+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35053568,"num_objects":8558,"num_object_clones":0,"num_object_copies":17116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":694,"num_objects_unfound":0,"num_objects_dirty":8558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9027,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3176,"num_bytes_recovered":13000704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7864},{"shards":"0,2","objects":694}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12074","reported_seq":12553,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.698940+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:49.698940+0000","last_peered":"2024-09-10T15:57:49.698940+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:49.698940+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8724,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4547},{"shards":"0,2","objects":1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12422","reported_seq":12554,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:49.725322+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:49.725322+0000","last_peered":"2024-09-10T15:57:49.725322+0000","last_clean":"2024-09-10T15:57:49.725322+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:49.725322+0000","last_undegraded":"2024-09-10T15:57:49.725322+0000","last_fullsized":"2024-09-10T15:57:49.725322+0000","mapping_epoch":35,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34783232,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9181,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12373","reported_seq":20501,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:49.811237+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:49.811237+0000","last_peered":"2024-09-10T15:57:49.811237+0000","last_clean":"2024-09-10T15:57:49.811237+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:49.811237+0000","last_undegraded":"2024-09-10T15:57:49.811237+0000","last_fullsized":"2024-09-10T15:57:49.811237+0000","mapping_epoch":45,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34185216,"num_objects":8346,"num_object_clones":0,"num_object_copies":16692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10350,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12179","reported_seq":12682,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.754146+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:49.754146+0000","last_peered":"2024-09-10T15:57:49.754146+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:49.754146+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9043,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4724},{"shards":"0,2","objects":1977}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12311","reported_seq":12614,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:50.060776+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:50.060776+0000","last_peered":"2024-09-10T15:57:50.060776+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:50.060776+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34709504,"num_objects":8474,"num_object_clones":0,"num_object_copies":16948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3745,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10285,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1916},{"shards":"2","objects":4731}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12116","reported_seq":12377,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:50.079118+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:50.079118+0000","last_peered":"2024-09-10T15:57:50.079118+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:50.079118+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34160640,"num_objects":8340,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9985,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1943},{"shards":"2","objects":4578}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12347","reported_seq":12869,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.711451+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:49.711451+0000","last_peered":"2024-09-10T15:57:49.711451+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:49.711451+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34844672,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9539,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4871},{"shards":"0,2","objects":1914}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12316","reported_seq":12774,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.713245+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:49.713245+0000","last_peered":"2024-09-10T15:57:49.713245+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:49.713245+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34414592,"num_objects":8402,"num_object_clones":0,"num_object_copies":16804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8734,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4632},{"shards":"0,2","objects":1964}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12284","reported_seq":12800,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.711252+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:49.711252+0000","last_peered":"2024-09-10T15:57:49.711252+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:49.711252+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34742272,"num_objects":8482,"num_object_clones":0,"num_object_copies":16964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3691,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8890,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4793},{"shards":"0,2","objects":1978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12544","reported_seq":13015,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.741025+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:49.741025+0000","last_peered":"2024-09-10T15:57:49.741025+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:49.741025+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35037667,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3840,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9363,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4718},{"shards":"0,2","objects":2014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":552018427,"num_objects":134772,"num_object_clones":0,"num_object_copies":269544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37307,"num_objects_misplaced":694,"num_objects_unfound":0,"num_objects_dirty":134772,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158278,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8646,"num_bytes_recovered":29360128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1212334080,"data_stored":1212250063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160657,"ondisk_log_size":160657,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620836,"kb_used_data":324052,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93751004,"statfs":{"total":96636764160,"available":96001028096,"internally_reserved":0,"allocated":331829248,"data_stored":331494089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303885998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149772,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838200,"kb_used_data":467812,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93533640,"statfs":{"total":96636764160,"available":95778447360,"internally_reserved":0,"allocated":479039488,"data_stored":478773435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":379252397},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,1,3,4,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704604,"kb_used_data":393932,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93667236,"statfs":{"total":96636764160,"available":95915249664,"internally_reserved":0,"allocated":403386368,"data_stored":403116243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477958144,"data_stored":477954531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331403264,"data_stored":331330545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402972672,"data_stored":402964987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:50.664 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:51.222 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:51.223 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:51.245 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-10T15:57:50.524686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":552018446,"num_objects":134774,"num_object_clones":0,"num_object_copies":269548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37307,"num_objects_misplaced":694,"num_objects_unfound":0,"num_objects_dirty":134774,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158280,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8648,"num_bytes_recovered":29360147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160660,"ondisk_log_size":160660,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2163640,"kb_used_data":1185796,"kb_used_omap":12,"kb_used_meta":977779,"kb_avail":280951880,"statfs":{"total":289910292480,"available":287694725120,"internally_reserved":0,"allocated":1214255104,"data_stored":1213383767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1001245701},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,6,3,4,6,0,0,1,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26873856,"num_objects":-6561,"num_object_clones":0,"num_object_copies":-13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-323,"num_objects_misplaced":-447,"num_objects_unfound":0,"num_objects_dirty":-6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6561,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":770,"num_bytes_recovered":1830912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004716"},"pg_stats":[{"pgid":"2.f","version":"46'12359","reported_seq":12818,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.775356+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:49.775356+0000","last_peered":"2024-09-10T15:57:49.775356+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:49.775356+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34451456,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":1999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12504","reported_seq":12612,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:50.025169+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:50.025169+0000","last_peered":"2024-09-10T15:57:50.025169+0000","last_clean":"2024-09-10T15:57:50.025169+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:50.025169+0000","last_undegraded":"2024-09-10T15:57:50.025169+0000","last_fullsized":"2024-09-10T15:57:50.025169+0000","mapping_epoch":36,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35102720,"num_objects":8570,"num_object_clones":0,"num_object_copies":17140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10840,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12248","reported_seq":12516,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:50.119385+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:50.119385+0000","last_peered":"2024-09-10T15:57:50.119385+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:50.119385+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34328600,"num_objects":8382,"num_object_clones":0,"num_object_copies":16764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8382,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10516,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1869},{"shards":"2","objects":4652}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12302","reported_seq":12382,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:50.106913+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:50.106913+0000","last_peered":"2024-09-10T15:57:50.106913+0000","last_clean":"2024-09-10T15:57:50.106913+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:50.106913+0000","last_undegraded":"2024-09-10T15:57:50.106913+0000","last_fullsized":"2024-09-10T15:57:50.106913+0000","mapping_epoch":36,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34324480,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12302,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12069","reported_seq":12196,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:49.763251+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:49.763251+0000","last_peered":"2024-09-10T15:57:49.763251+0000","last_clean":"2024-09-10T15:57:49.763251+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:49.763251+0000","last_undegraded":"2024-09-10T15:57:49.763251+0000","last_fullsized":"2024-09-10T15:57:49.763251+0000","mapping_epoch":35,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33841152,"num_objects":8262,"num_object_clones":0,"num_object_copies":16524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9140,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12309","reported_seq":18797,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:49.745169+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:49.745169+0000","last_peered":"2024-09-10T15:57:49.745169+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:49.745169+0000","last_undegraded":"2024-09-10T15:57:49.745169+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35053568,"num_objects":8558,"num_object_clones":0,"num_object_copies":17116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":694,"num_objects_unfound":0,"num_objects_dirty":8558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9027,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3176,"num_bytes_recovered":13000704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7864},{"shards":"0,2","objects":694}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12074","reported_seq":12553,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.698940+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:49.698940+0000","last_peered":"2024-09-10T15:57:49.698940+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:49.698940+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8724,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4547},{"shards":"0,2","objects":1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12422","reported_seq":12554,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:49.725322+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:49.725322+0000","last_peered":"2024-09-10T15:57:49.725322+0000","last_clean":"2024-09-10T15:57:49.725322+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:49.725322+0000","last_undegraded":"2024-09-10T15:57:49.725322+0000","last_fullsized":"2024-09-10T15:57:49.725322+0000","mapping_epoch":35,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34783232,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9181,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12373","reported_seq":20501,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:49.811237+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:49.811237+0000","last_peered":"2024-09-10T15:57:49.811237+0000","last_clean":"2024-09-10T15:57:49.811237+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:49.811237+0000","last_undegraded":"2024-09-10T15:57:49.811237+0000","last_fullsized":"2024-09-10T15:57:49.811237+0000","mapping_epoch":45,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34185216,"num_objects":8346,"num_object_clones":0,"num_object_copies":16692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10350,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12179","reported_seq":12682,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.754146+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:49.754146+0000","last_peered":"2024-09-10T15:57:49.754146+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:49.754146+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9043,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4724},{"shards":"0,2","objects":1977}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12311","reported_seq":12614,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:50.060776+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:50.060776+0000","last_peered":"2024-09-10T15:57:50.060776+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:50.060776+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34709504,"num_objects":8474,"num_object_clones":0,"num_object_copies":16948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3745,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10285,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1916},{"shards":"2","objects":4731}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12116","reported_seq":12377,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:50.079118+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:50.079118+0000","last_peered":"2024-09-10T15:57:50.079118+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:50.079118+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34160640,"num_objects":8340,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9985,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1943},{"shards":"2","objects":4578}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12347","reported_seq":12869,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.711451+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:49.711451+0000","last_peered":"2024-09-10T15:57:49.711451+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:49.711451+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34844672,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9539,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4871},{"shards":"0,2","objects":1914}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12316","reported_seq":12774,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.713245+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:49.713245+0000","last_peered":"2024-09-10T15:57:49.713245+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:49.713245+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34414592,"num_objects":8402,"num_object_clones":0,"num_object_copies":16804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8734,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4632},{"shards":"0,2","objects":1964}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12284","reported_seq":12800,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.711252+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:49.711252+0000","last_peered":"2024-09-10T15:57:49.711252+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:49.711252+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34742272,"num_objects":8482,"num_object_clones":0,"num_object_copies":16964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3691,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8890,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4793},{"shards":"0,2","objects":1978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12544","reported_seq":13015,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.741025+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:49.741025+0000","last_peered":"2024-09-10T15:57:49.741025+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:49.741025+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35037667,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3840,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9363,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4718},{"shards":"0,2","objects":2014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":552018427,"num_objects":134772,"num_object_clones":0,"num_object_copies":269544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37307,"num_objects_misplaced":694,"num_objects_unfound":0,"num_objects_dirty":134772,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158278,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8646,"num_bytes_recovered":29360128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1212334080,"data_stored":1212250063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160657,"ondisk_log_size":160657,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620836,"kb_used_data":324052,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93751004,"statfs":{"total":96636764160,"available":96001028096,"internally_reserved":0,"allocated":331829248,"data_stored":331494089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303885998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149772,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838200,"kb_used_data":467812,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93533640,"statfs":{"total":96636764160,"available":95778447360,"internally_reserved":0,"allocated":479039488,"data_stored":478773435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":379252397},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,1,3,4,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704604,"kb_used_data":393932,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93667236,"statfs":{"total":96636764160,"available":95915249664,"internally_reserved":0,"allocated":403386368,"data_stored":403116243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477958144,"data_stored":477954531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331403264,"data_stored":331330545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402972672,"data_stored":402964987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:51.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:57:51.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'12359", 'reported_seq': 12818, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:49.775356+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:57:49.775356+0000', 'last_peered': '2024-09-10T15:57:49.775356+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:57:49.775356+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'2300", 'ondisk_log_start': "37'2300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34451456, 'num_objects': 8411, 'num_object_clones': 0, 'num_object_copies': 16822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3782, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8411, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12359, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 191, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4629}, {'shards': '0,2', 'objects': 1999}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:51.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:57:51.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'12248", 'reported_seq': 12516, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:50.119385+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:57:50.119385+0000', 'last_peered': '2024-09-10T15:57:50.119385+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:57:50.119385+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34328600, 'num_objects': 8382, 'num_object_clones': 0, 'num_object_copies': 16764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3731, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8382, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10516, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 72, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1869}, {'shards': '2', 'objects': 4652}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:51.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:57:51.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'12309", 'reported_seq': 18797, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:57:49.745169+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:57:49.745169+0000', 'last_peered': '2024-09-10T15:57:49.745169+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:57:49.745169+0000', 'last_undegraded': '2024-09-10T15:57:49.745169+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'2300", 'ondisk_log_start': "37'2300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35053568, 'num_objects': 8558, 'num_object_clones': 0, 'num_object_copies': 17116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 694, 'num_objects_unfound': 0, 'num_objects_dirty': 8558, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9027, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3176, 'num_bytes_recovered': 13000704, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7864}, {'shards': '0,2', 'objects': 694}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:51.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:57:51.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'12074", 'reported_seq': 12553, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:49.698940+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:57:49.698940+0000', 'last_peered': '2024-09-10T15:57:49.698940+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:57:49.698940+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33636352, 'num_objects': 8212, 'num_object_clones': 0, 'num_object_copies': 16424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3667, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8212, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8724, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 170, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4547}, {'shards': '0,2', 'objects': 1906}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:51.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:57:51.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'12179", 'reported_seq': 12682, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:49.754146+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:57:49.754146+0000', 'last_peered': '2024-09-10T15:57:49.754146+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:57:49.754146+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'2100", 'ondisk_log_start': "37'2100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34402304, 'num_objects': 8399, 'num_object_clones': 0, 'num_object_copies': 16798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3677, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8399, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9043, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 188, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4724}, {'shards': '0,2', 'objects': 1977}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:51.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:57:51.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'12311", 'reported_seq': 12614, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:50.060776+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:57:50.060776+0000', 'last_peered': '2024-09-10T15:57:50.060776+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:57:50.060776+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'2300", 'ondisk_log_start': "37'2300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34709504, 'num_objects': 8474, 'num_object_clones': 0, 'num_object_copies': 16948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3745, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10285, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 89, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1916}, {'shards': '2', 'objects': 4731}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:51.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:57:51.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'12116", 'reported_seq': 12377, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:50.079118+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:57:50.079118+0000', 'last_peered': '2024-09-10T15:57:50.079118+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:57:50.079118+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'2100", 'ondisk_log_start': "37'2100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34160640, 'num_objects': 8340, 'num_object_clones': 0, 'num_object_copies': 16680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3764, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8340, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9985, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1943}, {'shards': '2', 'objects': 4578}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:51.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:57:51.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'12347", 'reported_seq': 12869, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:49.711451+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:57:49.711451+0000', 'last_peered': '2024-09-10T15:57:49.711451+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:57:49.711451+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'2300", 'ondisk_log_start': "37'2300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34844672, 'num_objects': 8507, 'num_object_clones': 0, 'num_object_copies': 17014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3637, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8507, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9539, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 197, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4871}, {'shards': '0,2', 'objects': 1914}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:51.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:57:51.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'12316", 'reported_seq': 12774, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:49.713245+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:57:49.713245+0000', 'last_peered': '2024-09-10T15:57:49.713245+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:57:49.713245+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'2300", 'ondisk_log_start': "37'2300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34414592, 'num_objects': 8402, 'num_object_clones': 0, 'num_object_copies': 16804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3773, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8402, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8734, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 147, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4632}, {'shards': '0,2', 'objects': 1964}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:51.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:57:51.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'12284", 'reported_seq': 12800, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:49.711252+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:57:49.711252+0000', 'last_peered': '2024-09-10T15:57:49.711252+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:57:49.711252+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34742272, 'num_objects': 8482, 'num_object_clones': 0, 'num_object_copies': 16964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3691, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8482, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8890, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 189, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4793}, {'shards': '0,2', 'objects': 1978}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:51.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:57:51.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'12544", 'reported_seq': 13015, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:49.741025+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:57:49.741025+0000', 'last_peered': '2024-09-10T15:57:49.741025+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:57:49.741025+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'2500", 'ondisk_log_start': "37'2500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35037667, 'num_objects': 8555, 'num_object_clones': 0, 'num_object_copies': 17110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3840, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8555, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9363, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 166, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4718}, {'shards': '0,2', 'objects': 2014}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:51.249 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:57:51.942 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:51.943 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37307/269548 objects degraded (13.841%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":397,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":134774,"data_bytes":552018446,"bytes_used":2215567360,"bytes_avail":287694725120,"bytes_total":289910292480,"degraded_objects":37307,"degraded_total":269548,"degraded_ratio":0.13840577559469927,"misplaced_objects":694,"misplaced_total":269548,"misplaced_ratio":0.0025746805763722973,"recovering_objects_per_sec":64,"recovering_bytes_per_sec":152516,"recovering_keys_per_sec":0,"num_objects_recovered":770,"num_bytes_recovered":1830912,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":546},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:57:51.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:57:51.962 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:52.510 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:52.511 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:52.548 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-10T15:57:50.524686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":552018446,"num_objects":134774,"num_object_clones":0,"num_object_copies":269548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37307,"num_objects_misplaced":694,"num_objects_unfound":0,"num_objects_dirty":134774,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158280,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8648,"num_bytes_recovered":29360147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160660,"ondisk_log_size":160660,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2163640,"kb_used_data":1185796,"kb_used_omap":12,"kb_used_meta":977779,"kb_avail":280951880,"statfs":{"total":289910292480,"available":287694725120,"internally_reserved":0,"allocated":1214255104,"data_stored":1213383767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1001245701},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,6,3,4,6,0,0,1,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26873856,"num_objects":-6561,"num_object_clones":0,"num_object_copies":-13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-323,"num_objects_misplaced":-447,"num_objects_unfound":0,"num_objects_dirty":-6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6561,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":770,"num_bytes_recovered":1830912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004716"},"pg_stats":[{"pgid":"2.f","version":"46'12359","reported_seq":12818,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.775356+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:49.775356+0000","last_peered":"2024-09-10T15:57:49.775356+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:49.775356+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34451456,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":1999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12504","reported_seq":12612,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:50.025169+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:50.025169+0000","last_peered":"2024-09-10T15:57:50.025169+0000","last_clean":"2024-09-10T15:57:50.025169+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:50.025169+0000","last_undegraded":"2024-09-10T15:57:50.025169+0000","last_fullsized":"2024-09-10T15:57:50.025169+0000","mapping_epoch":36,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35102720,"num_objects":8570,"num_object_clones":0,"num_object_copies":17140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10840,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12248","reported_seq":12516,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:50.119385+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:50.119385+0000","last_peered":"2024-09-10T15:57:50.119385+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:50.119385+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34328600,"num_objects":8382,"num_object_clones":0,"num_object_copies":16764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8382,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10516,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1869},{"shards":"2","objects":4652}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12302","reported_seq":12382,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:50.106913+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:50.106913+0000","last_peered":"2024-09-10T15:57:50.106913+0000","last_clean":"2024-09-10T15:57:50.106913+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:50.106913+0000","last_undegraded":"2024-09-10T15:57:50.106913+0000","last_fullsized":"2024-09-10T15:57:50.106913+0000","mapping_epoch":36,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34324480,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12302,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12069","reported_seq":12196,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:49.763251+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:49.763251+0000","last_peered":"2024-09-10T15:57:49.763251+0000","last_clean":"2024-09-10T15:57:49.763251+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:49.763251+0000","last_undegraded":"2024-09-10T15:57:49.763251+0000","last_fullsized":"2024-09-10T15:57:49.763251+0000","mapping_epoch":35,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33841152,"num_objects":8262,"num_object_clones":0,"num_object_copies":16524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9140,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12309","reported_seq":18797,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:49.745169+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:49.745169+0000","last_peered":"2024-09-10T15:57:49.745169+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:49.745169+0000","last_undegraded":"2024-09-10T15:57:49.745169+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35053568,"num_objects":8558,"num_object_clones":0,"num_object_copies":17116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":694,"num_objects_unfound":0,"num_objects_dirty":8558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9027,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3176,"num_bytes_recovered":13000704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7864},{"shards":"0,2","objects":694}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12074","reported_seq":12553,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.698940+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:49.698940+0000","last_peered":"2024-09-10T15:57:49.698940+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:49.698940+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8724,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4547},{"shards":"0,2","objects":1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12422","reported_seq":12554,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:49.725322+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:49.725322+0000","last_peered":"2024-09-10T15:57:49.725322+0000","last_clean":"2024-09-10T15:57:49.725322+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:49.725322+0000","last_undegraded":"2024-09-10T15:57:49.725322+0000","last_fullsized":"2024-09-10T15:57:49.725322+0000","mapping_epoch":35,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34783232,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9181,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12373","reported_seq":20501,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:49.811237+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:49.811237+0000","last_peered":"2024-09-10T15:57:49.811237+0000","last_clean":"2024-09-10T15:57:49.811237+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:49.811237+0000","last_undegraded":"2024-09-10T15:57:49.811237+0000","last_fullsized":"2024-09-10T15:57:49.811237+0000","mapping_epoch":45,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34185216,"num_objects":8346,"num_object_clones":0,"num_object_copies":16692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10350,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12179","reported_seq":12682,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.754146+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:49.754146+0000","last_peered":"2024-09-10T15:57:49.754146+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:49.754146+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9043,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4724},{"shards":"0,2","objects":1977}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12311","reported_seq":12614,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:50.060776+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:50.060776+0000","last_peered":"2024-09-10T15:57:50.060776+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:50.060776+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34709504,"num_objects":8474,"num_object_clones":0,"num_object_copies":16948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3745,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10285,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1916},{"shards":"2","objects":4731}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12116","reported_seq":12377,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:50.079118+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:50.079118+0000","last_peered":"2024-09-10T15:57:50.079118+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:50.079118+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34160640,"num_objects":8340,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9985,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1943},{"shards":"2","objects":4578}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12347","reported_seq":12869,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.711451+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:49.711451+0000","last_peered":"2024-09-10T15:57:49.711451+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:49.711451+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34844672,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9539,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4871},{"shards":"0,2","objects":1914}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12316","reported_seq":12774,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.713245+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:49.713245+0000","last_peered":"2024-09-10T15:57:49.713245+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:49.713245+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34414592,"num_objects":8402,"num_object_clones":0,"num_object_copies":16804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8734,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4632},{"shards":"0,2","objects":1964}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12284","reported_seq":12800,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.711252+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:49.711252+0000","last_peered":"2024-09-10T15:57:49.711252+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:49.711252+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34742272,"num_objects":8482,"num_object_clones":0,"num_object_copies":16964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3691,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8890,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4793},{"shards":"0,2","objects":1978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12544","reported_seq":13015,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.741025+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:49.741025+0000","last_peered":"2024-09-10T15:57:49.741025+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:49.741025+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35037667,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3840,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9363,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4718},{"shards":"0,2","objects":2014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":552018427,"num_objects":134772,"num_object_clones":0,"num_object_copies":269544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37307,"num_objects_misplaced":694,"num_objects_unfound":0,"num_objects_dirty":134772,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158278,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8646,"num_bytes_recovered":29360128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1212334080,"data_stored":1212250063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160657,"ondisk_log_size":160657,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620836,"kb_used_data":324052,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93751004,"statfs":{"total":96636764160,"available":96001028096,"internally_reserved":0,"allocated":331829248,"data_stored":331494089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303885998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149772,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838200,"kb_used_data":467812,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93533640,"statfs":{"total":96636764160,"available":95778447360,"internally_reserved":0,"allocated":479039488,"data_stored":478773435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":379252397},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,1,3,4,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704604,"kb_used_data":393932,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93667236,"statfs":{"total":96636764160,"available":95915249664,"internally_reserved":0,"allocated":403386368,"data_stored":403116243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477958144,"data_stored":477954531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331403264,"data_stored":331330545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402972672,"data_stored":402964987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:55.551 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:56.088 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:56.089 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:56.111 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-10T15:57:54.525969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":552018446,"num_objects":134774,"num_object_clones":0,"num_object_copies":269548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37307,"num_objects_misplaced":694,"num_objects_unfound":0,"num_objects_dirty":134774,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158280,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8648,"num_bytes_recovered":29360147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160660,"ondisk_log_size":160660,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2163640,"kb_used_data":1185796,"kb_used_omap":12,"kb_used_meta":977779,"kb_avail":280951880,"statfs":{"total":289910292480,"available":287694725120,"internally_reserved":0,"allocated":1214255104,"data_stored":1213383767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1001245701},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,6,3,4,6,0,0,1,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17895424,"num_objects":-4369,"num_object_clones":0,"num_object_copies":-8738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-204,"num_objects_misplaced":-319,"num_objects_unfound":0,"num_objects_dirty":-4369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4369,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":1306624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004661"},"pg_stats":[{"pgid":"2.f","version":"46'12359","reported_seq":12818,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.775356+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:49.775356+0000","last_peered":"2024-09-10T15:57:49.775356+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:49.775356+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34451456,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":1999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12504","reported_seq":12612,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:50.025169+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:50.025169+0000","last_peered":"2024-09-10T15:57:50.025169+0000","last_clean":"2024-09-10T15:57:50.025169+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:50.025169+0000","last_undegraded":"2024-09-10T15:57:50.025169+0000","last_fullsized":"2024-09-10T15:57:50.025169+0000","mapping_epoch":36,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35102720,"num_objects":8570,"num_object_clones":0,"num_object_copies":17140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10840,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12248","reported_seq":12516,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:50.119385+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:50.119385+0000","last_peered":"2024-09-10T15:57:50.119385+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:50.119385+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34328600,"num_objects":8382,"num_object_clones":0,"num_object_copies":16764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8382,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10516,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1869},{"shards":"2","objects":4652}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12302","reported_seq":12382,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:50.106913+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:50.106913+0000","last_peered":"2024-09-10T15:57:50.106913+0000","last_clean":"2024-09-10T15:57:50.106913+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:50.106913+0000","last_undegraded":"2024-09-10T15:57:50.106913+0000","last_fullsized":"2024-09-10T15:57:50.106913+0000","mapping_epoch":36,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34324480,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12302,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12069","reported_seq":12196,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:49.763251+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:49.763251+0000","last_peered":"2024-09-10T15:57:49.763251+0000","last_clean":"2024-09-10T15:57:49.763251+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:49.763251+0000","last_undegraded":"2024-09-10T15:57:49.763251+0000","last_fullsized":"2024-09-10T15:57:49.763251+0000","mapping_epoch":35,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33841152,"num_objects":8262,"num_object_clones":0,"num_object_copies":16524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9140,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12309","reported_seq":18797,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:49.745169+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:49.745169+0000","last_peered":"2024-09-10T15:57:49.745169+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:49.745169+0000","last_undegraded":"2024-09-10T15:57:49.745169+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35053568,"num_objects":8558,"num_object_clones":0,"num_object_copies":17116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":694,"num_objects_unfound":0,"num_objects_dirty":8558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9027,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3176,"num_bytes_recovered":13000704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7864},{"shards":"0,2","objects":694}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12074","reported_seq":12553,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.698940+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:49.698940+0000","last_peered":"2024-09-10T15:57:49.698940+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:49.698940+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8724,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4547},{"shards":"0,2","objects":1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12422","reported_seq":12554,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:49.725322+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:49.725322+0000","last_peered":"2024-09-10T15:57:49.725322+0000","last_clean":"2024-09-10T15:57:49.725322+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:49.725322+0000","last_undegraded":"2024-09-10T15:57:49.725322+0000","last_fullsized":"2024-09-10T15:57:49.725322+0000","mapping_epoch":35,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34783232,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9181,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12373","reported_seq":20501,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:49.811237+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:49.811237+0000","last_peered":"2024-09-10T15:57:49.811237+0000","last_clean":"2024-09-10T15:57:49.811237+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:49.811237+0000","last_undegraded":"2024-09-10T15:57:49.811237+0000","last_fullsized":"2024-09-10T15:57:49.811237+0000","mapping_epoch":45,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34185216,"num_objects":8346,"num_object_clones":0,"num_object_copies":16692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10350,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12179","reported_seq":12682,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.754146+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:49.754146+0000","last_peered":"2024-09-10T15:57:49.754146+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:49.754146+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9043,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4724},{"shards":"0,2","objects":1977}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12311","reported_seq":12614,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:50.060776+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:50.060776+0000","last_peered":"2024-09-10T15:57:50.060776+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:50.060776+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34709504,"num_objects":8474,"num_object_clones":0,"num_object_copies":16948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3745,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10285,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1916},{"shards":"2","objects":4731}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12116","reported_seq":12377,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:50.079118+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:50.079118+0000","last_peered":"2024-09-10T15:57:50.079118+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:50.079118+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34160640,"num_objects":8340,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3764,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9985,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1943},{"shards":"2","objects":4578}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12347","reported_seq":12869,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.711451+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:49.711451+0000","last_peered":"2024-09-10T15:57:49.711451+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:49.711451+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34844672,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9539,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4871},{"shards":"0,2","objects":1914}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12316","reported_seq":12774,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.713245+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:49.713245+0000","last_peered":"2024-09-10T15:57:49.713245+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:49.713245+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34414592,"num_objects":8402,"num_object_clones":0,"num_object_copies":16804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8734,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4632},{"shards":"0,2","objects":1964}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12284","reported_seq":12800,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.711252+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:49.711252+0000","last_peered":"2024-09-10T15:57:49.711252+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:49.711252+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34742272,"num_objects":8482,"num_object_clones":0,"num_object_copies":16964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3691,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8890,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4793},{"shards":"0,2","objects":1978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12544","reported_seq":13015,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:49.741025+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:49.741025+0000","last_peered":"2024-09-10T15:57:49.741025+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:49.741025+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35037667,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3840,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9363,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4718},{"shards":"0,2","objects":2014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":552018427,"num_objects":134772,"num_object_clones":0,"num_object_copies":269544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37307,"num_objects_misplaced":694,"num_objects_unfound":0,"num_objects_dirty":134772,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158278,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8646,"num_bytes_recovered":29360128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1212334080,"data_stored":1212250063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160657,"ondisk_log_size":160657,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620836,"kb_used_data":324052,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93751004,"statfs":{"total":96636764160,"available":96001028096,"internally_reserved":0,"allocated":331829248,"data_stored":331494089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303885998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149772,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838200,"kb_used_data":467812,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93533640,"statfs":{"total":96636764160,"available":95778447360,"internally_reserved":0,"allocated":479039488,"data_stored":478773435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":379252397},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,1,3,4,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704604,"kb_used_data":393932,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93667236,"statfs":{"total":96636764160,"available":95915249664,"internally_reserved":0,"allocated":403386368,"data_stored":403116243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477958144,"data_stored":477954531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331403264,"data_stored":331330545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402972672,"data_stored":402964987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:56.113 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:56.650 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:56.650 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:56.680 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-10T15:57:56.526633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":543203854,"num_objects":132622,"num_object_clones":0,"num_object_copies":265244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37213,"num_objects_misplaced":539,"num_objects_unfound":0,"num_objects_dirty":132622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160432,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8897,"num_bytes_recovered":29995027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2131496,"kb_used_data":1174452,"kb_used_omap":12,"kb_used_meta":956979,"kb_avail":280984024,"statfs":{"total":289910292480,"available":287727640576,"internally_reserved":0,"allocated":1202638848,"data_stored":1201767511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":979946501},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,7,3,13,0,0,0,6,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26710016,"num_objects":-6521,"num_object_clones":0,"num_object_copies":-13042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-298,"num_objects_misplaced":-474,"num_objects_unfound":0,"num_objects_dirty":-6521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6521,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":772,"num_bytes_recovered":1941504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003692"},"pg_stats":[{"pgid":"2.f","version":"46'12495","reported_seq":12980,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.748983+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:54.748983+0000","last_peered":"2024-09-10T15:57:54.748983+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:54.748983+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33894400,"num_objects":8275,"num_object_clones":0,"num_object_copies":16550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12495,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4507},{"shards":"0,2","objects":1863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12643","reported_seq":12750,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:55.104317+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:55.104317+0000","last_peered":"2024-09-10T15:57:55.104317+0000","last_clean":"2024-09-10T15:57:55.104317+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:55.104317+0000","last_undegraded":"2024-09-10T15:57:55.104317+0000","last_fullsized":"2024-09-10T15:57:55.104317+0000","mapping_epoch":36,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34533376,"num_objects":8431,"num_object_clones":0,"num_object_copies":16862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10979,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12378","reported_seq":12657,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:55.029128+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:55.029128+0000","last_peered":"2024-09-10T15:57:55.029128+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:55.029128+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33796120,"num_objects":8252,"num_object_clones":0,"num_object_copies":16504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8252,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10646,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1739},{"shards":"2","objects":4527}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12429","reported_seq":12509,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:55.087376+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:55.087376+0000","last_peered":"2024-09-10T15:57:55.087376+0000","last_clean":"2024-09-10T15:57:55.087376+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:55.087376+0000","last_undegraded":"2024-09-10T15:57:55.087376+0000","last_fullsized":"2024-09-10T15:57:55.087376+0000","mapping_epoch":36,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33804288,"num_objects":8253,"num_object_clones":0,"num_object_copies":16506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12429,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12204","reported_seq":12331,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:54.774223+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:54.774223+0000","last_peered":"2024-09-10T15:57:54.774223+0000","last_clean":"2024-09-10T15:57:54.774223+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:54.774223+0000","last_undegraded":"2024-09-10T15:57:54.774223+0000","last_fullsized":"2024-09-10T15:57:54.774223+0000","mapping_epoch":35,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9275,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12453","reported_seq":19251,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:54.773972+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:54.773972+0000","last_peered":"2024-09-10T15:57:54.773972+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:54.773972+0000","last_undegraded":"2024-09-10T15:57:54.773972+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34463744,"num_objects":8414,"num_object_clones":0,"num_object_copies":16828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":539,"num_objects_unfound":0,"num_objects_dirty":8414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9171,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3331,"num_bytes_recovered":13635584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7875},{"shards":"0,2","objects":539}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12212","reported_seq":12707,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.788480+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:54.788480+0000","last_peered":"2024-09-10T15:57:54.788480+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:54.788480+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33071104,"num_objects":8074,"num_object_clones":0,"num_object_copies":16148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8862,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4417},{"shards":"0,2","objects":1768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12568","reported_seq":12700,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:54.705058+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:54.705058+0000","last_peered":"2024-09-10T15:57:54.705058+0000","last_clean":"2024-09-10T15:57:54.705058+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:54.705058+0000","last_undegraded":"2024-09-10T15:57:54.705058+0000","last_fullsized":"2024-09-10T15:57:54.705058+0000","mapping_epoch":35,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34185216,"num_objects":8346,"num_object_clones":0,"num_object_copies":16692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9327,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12514","reported_seq":20642,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:54.830293+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:54.830293+0000","last_peered":"2024-09-10T15:57:54.830293+0000","last_clean":"2024-09-10T15:57:54.830293+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:54.830293+0000","last_undegraded":"2024-09-10T15:57:54.830293+0000","last_fullsized":"2024-09-10T15:57:54.830293+0000","mapping_epoch":45,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33607680,"num_objects":8205,"num_object_clones":0,"num_object_copies":16410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10491,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12294","reported_seq":12815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.762903+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:54.762903+0000","last_peered":"2024-09-10T15:57:54.762903+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:54.762903+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33931264,"num_objects":8284,"num_object_clones":0,"num_object_copies":16568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3668,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9158,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4618},{"shards":"0,2","objects":1862}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12440","reported_seq":12759,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:55.035292+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:55.035292+0000","last_peered":"2024-09-10T15:57:55.035292+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:55.035292+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34181120,"num_objects":8345,"num_object_clones":0,"num_object_copies":16690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3737,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10414,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1787},{"shards":"2","objects":4610}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12246","reported_seq":12519,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:55.062992+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:55.062992+0000","last_peered":"2024-09-10T15:57:55.062992+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:55.062992+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33628160,"num_objects":8210,"num_object_clones":0,"num_object_copies":16420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10115,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1813},{"shards":"2","objects":4454}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12470","reported_seq":13007,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.792615+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:54.792615+0000","last_peered":"2024-09-10T15:57:54.792615+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:54.792615+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34340864,"num_objects":8384,"num_object_clones":0,"num_object_copies":16768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9662,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4756},{"shards":"0,2","objects":1791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12453","reported_seq":12943,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.727022+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:54.727022+0000","last_peered":"2024-09-10T15:57:54.727022+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:54.727022+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33853440,"num_objects":8265,"num_object_clones":0,"num_object_copies":16530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3757,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8871,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4511},{"shards":"0,2","objects":1827}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12432","reported_seq":12974,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.807831+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:54.807831+0000","last_peered":"2024-09-10T15:57:54.807831+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:54.807831+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136064,"num_objects":8334,"num_object_clones":0,"num_object_copies":16668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3678,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9038,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4658},{"shards":"0,2","objects":1830}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12678","reported_seq":13163,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.765644+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:54.765644+0000","last_peered":"2024-09-10T15:57:54.765644+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:54.765644+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34488803,"num_objects":8421,"num_object_clones":0,"num_object_copies":16842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9497,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4591},{"shards":"0,2","objects":1880}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":543203835,"num_objects":132620,"num_object_clones":0,"num_object_copies":265240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37213,"num_objects_misplaced":539,"num_objects_unfound":0,"num_objects_dirty":132620,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160430,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8895,"num_bytes_recovered":29995008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1200734208,"data_stored":1200650191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626490,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607836,"kb_used_data":323528,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93764004,"statfs":{"total":96636764160,"available":96014340096,"internally_reserved":0,"allocated":331292672,"data_stored":330957513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291106478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149773,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":835824,"kb_used_data":461280,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93536016,"statfs":{"total":96636764160,"available":95780880384,"internally_reserved":0,"allocated":472350720,"data_stored":472084667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":383512237},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,1,13,0,0,0,0,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687836,"kb_used_data":389644,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93684004,"statfs":{"total":96636764160,"available":95932420096,"internally_reserved":0,"allocated":398995456,"data_stored":398725331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":471334912,"data_stored":471331299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330858496,"data_stored":330785777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":398540800,"data_stored":398533115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:57:56.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:57:56.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'12495", 'reported_seq': 12980, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:54.748983+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:57:54.748983+0000', 'last_peered': '2024-09-10T15:57:54.748983+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:57:54.748983+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33894400, 'num_objects': 8275, 'num_object_clones': 0, 'num_object_copies': 16550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3768, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8275, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12495, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 205, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4507}, {'shards': '0,2', 'objects': 1863}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:56.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:57:56.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'12378", 'reported_seq': 12657, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:55.029128+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:57:55.029128+0000', 'last_peered': '2024-09-10T15:57:55.029128+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:57:55.029128+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'2300", 'ondisk_log_start': "37'2300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33796120, 'num_objects': 8252, 'num_object_clones': 0, 'num_object_copies': 16504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3726, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8252, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10646, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1739}, {'shards': '2', 'objects': 4527}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:56.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:57:56.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'12453", 'reported_seq': 19251, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:57:54.773972+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:57:54.773972+0000', 'last_peered': '2024-09-10T15:57:54.773972+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:57:54.773972+0000', 'last_undegraded': '2024-09-10T15:57:54.773972+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34463744, 'num_objects': 8414, 'num_object_clones': 0, 'num_object_copies': 16828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 539, 'num_objects_unfound': 0, 'num_objects_dirty': 8414, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9171, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3331, 'num_bytes_recovered': 13635584, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7875}, {'shards': '0,2', 'objects': 539}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:56.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:57:56.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'12212", 'reported_seq': 12707, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:54.788480+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:57:54.788480+0000', 'last_peered': '2024-09-10T15:57:54.788480+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:57:54.788480+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33071104, 'num_objects': 8074, 'num_object_clones': 0, 'num_object_copies': 16148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3659, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8074, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8862, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 178, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4417}, {'shards': '0,2', 'objects': 1768}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:56.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:57:56.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'12294", 'reported_seq': 12815, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:54.762903+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:57:54.762903+0000', 'last_peered': '2024-09-10T15:57:54.762903+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:57:54.762903+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33931264, 'num_objects': 8284, 'num_object_clones': 0, 'num_object_copies': 16568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3668, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8284, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9158, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 197, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4618}, {'shards': '0,2', 'objects': 1862}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:56.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:57:56.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'12440", 'reported_seq': 12759, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:55.035292+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:57:55.035292+0000', 'last_peered': '2024-09-10T15:57:55.035292+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:57:55.035292+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34181120, 'num_objects': 8345, 'num_object_clones': 0, 'num_object_copies': 16690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3737, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8345, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10414, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 97, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1787}, {'shards': '2', 'objects': 4610}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:56.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:57:56.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'12246", 'reported_seq': 12519, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:55.062992+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:57:55.062992+0000', 'last_peered': '2024-09-10T15:57:55.062992+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:57:55.062992+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33628160, 'num_objects': 8210, 'num_object_clones': 0, 'num_object_copies': 16420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3758, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8210, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10115, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 73, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1813}, {'shards': '2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:57:56.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:57:56.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'12470", 'reported_seq': 13007, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:54.792615+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:57:54.792615+0000', 'last_peered': '2024-09-10T15:57:54.792615+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:57:54.792615+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34340864, 'num_objects': 8384, 'num_object_clones': 0, 'num_object_copies': 16768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8384, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9662, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 205, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4756}, {'shards': '0,2', 'objects': 1791}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:56.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:57:56.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'12453", 'reported_seq': 12943, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:54.727022+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:57:54.727022+0000', 'last_peered': '2024-09-10T15:57:54.727022+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:57:54.727022+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33853440, 'num_objects': 8265, 'num_object_clones': 0, 'num_object_copies': 16530, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3757, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8265, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8871, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 163, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4511}, {'shards': '0,2', 'objects': 1827}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:56.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:57:56.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'12432", 'reported_seq': 12974, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:54.807831+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:57:54.807831+0000', 'last_peered': '2024-09-10T15:57:54.807831+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:57:54.807831+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34136064, 'num_objects': 8334, 'num_object_clones': 0, 'num_object_copies': 16668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3678, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8334, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9038, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 202, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4658}, {'shards': '0,2', 'objects': 1830}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:56.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:57:56.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'12678", 'reported_seq': 13163, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:54.765644+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:57:54.765644+0000', 'last_peered': '2024-09-10T15:57:54.765644+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:57:54.765644+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34488803, 'num_objects': 8421, 'num_object_clones': 0, 'num_object_copies': 16842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3833, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8421, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9497, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4591}, {'shards': '0,2', 'objects': 1880}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:57:56.684 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:57:57.387 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:57.387 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37213/265244 objects degraded (14.030%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":403,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":132622,"data_bytes":543203854,"bytes_used":2182651904,"bytes_avail":287727640576,"bytes_total":289910292480,"degraded_objects":37213,"degraded_total":265244,"degraded_ratio":0.14029723575274086,"misplaced_objects":539,"misplaced_total":265244,"misplaced_ratio":0.0020320912065871426,"recovering_objects_per_sec":64,"recovering_bytes_per_sec":161742,"recovering_keys_per_sec":0,"num_objects_recovered":772,"num_bytes_recovered":1941504,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":543},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:57:57.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:57:57.412 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:57:57.945 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:57:57.946 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:57:57.972 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-10T15:57:56.526633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":543203854,"num_objects":132622,"num_object_clones":0,"num_object_copies":265244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37213,"num_objects_misplaced":539,"num_objects_unfound":0,"num_objects_dirty":132622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160432,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8897,"num_bytes_recovered":29995027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2131496,"kb_used_data":1174452,"kb_used_omap":12,"kb_used_meta":956979,"kb_avail":280984024,"statfs":{"total":289910292480,"available":287727640576,"internally_reserved":0,"allocated":1202638848,"data_stored":1201767511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":979946501},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,7,3,13,0,0,0,6,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26710016,"num_objects":-6521,"num_object_clones":0,"num_object_copies":-13042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-298,"num_objects_misplaced":-474,"num_objects_unfound":0,"num_objects_dirty":-6521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6521,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":772,"num_bytes_recovered":1941504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003692"},"pg_stats":[{"pgid":"2.f","version":"46'12495","reported_seq":12980,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.748983+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:54.748983+0000","last_peered":"2024-09-10T15:57:54.748983+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:54.748983+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33894400,"num_objects":8275,"num_object_clones":0,"num_object_copies":16550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12495,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4507},{"shards":"0,2","objects":1863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12643","reported_seq":12750,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:55.104317+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:57:55.104317+0000","last_peered":"2024-09-10T15:57:55.104317+0000","last_clean":"2024-09-10T15:57:55.104317+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:57:55.104317+0000","last_undegraded":"2024-09-10T15:57:55.104317+0000","last_fullsized":"2024-09-10T15:57:55.104317+0000","mapping_epoch":36,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34533376,"num_objects":8431,"num_object_clones":0,"num_object_copies":16862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10979,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12378","reported_seq":12657,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:55.029128+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:57:55.029128+0000","last_peered":"2024-09-10T15:57:55.029128+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:57:55.029128+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33796120,"num_objects":8252,"num_object_clones":0,"num_object_copies":16504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8252,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10646,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1739},{"shards":"2","objects":4527}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12429","reported_seq":12509,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:55.087376+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:57:55.087376+0000","last_peered":"2024-09-10T15:57:55.087376+0000","last_clean":"2024-09-10T15:57:55.087376+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:57:55.087376+0000","last_undegraded":"2024-09-10T15:57:55.087376+0000","last_fullsized":"2024-09-10T15:57:55.087376+0000","mapping_epoch":36,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33804288,"num_objects":8253,"num_object_clones":0,"num_object_copies":16506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12429,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12204","reported_seq":12331,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:54.774223+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:54.774223+0000","last_peered":"2024-09-10T15:57:54.774223+0000","last_clean":"2024-09-10T15:57:54.774223+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:54.774223+0000","last_undegraded":"2024-09-10T15:57:54.774223+0000","last_fullsized":"2024-09-10T15:57:54.774223+0000","mapping_epoch":35,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9275,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12453","reported_seq":19251,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:54.773972+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:54.773972+0000","last_peered":"2024-09-10T15:57:54.773972+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:54.773972+0000","last_undegraded":"2024-09-10T15:57:54.773972+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34463744,"num_objects":8414,"num_object_clones":0,"num_object_copies":16828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":539,"num_objects_unfound":0,"num_objects_dirty":8414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9171,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3331,"num_bytes_recovered":13635584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7875},{"shards":"0,2","objects":539}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12212","reported_seq":12707,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.788480+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:54.788480+0000","last_peered":"2024-09-10T15:57:54.788480+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:54.788480+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33071104,"num_objects":8074,"num_object_clones":0,"num_object_copies":16148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8862,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4417},{"shards":"0,2","objects":1768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12568","reported_seq":12700,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:54.705058+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:54.705058+0000","last_peered":"2024-09-10T15:57:54.705058+0000","last_clean":"2024-09-10T15:57:54.705058+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:54.705058+0000","last_undegraded":"2024-09-10T15:57:54.705058+0000","last_fullsized":"2024-09-10T15:57:54.705058+0000","mapping_epoch":35,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34185216,"num_objects":8346,"num_object_clones":0,"num_object_copies":16692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9327,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12514","reported_seq":20642,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:54.830293+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:54.830293+0000","last_peered":"2024-09-10T15:57:54.830293+0000","last_clean":"2024-09-10T15:57:54.830293+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:54.830293+0000","last_undegraded":"2024-09-10T15:57:54.830293+0000","last_fullsized":"2024-09-10T15:57:54.830293+0000","mapping_epoch":45,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33607680,"num_objects":8205,"num_object_clones":0,"num_object_copies":16410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10491,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12294","reported_seq":12815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.762903+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:54.762903+0000","last_peered":"2024-09-10T15:57:54.762903+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:54.762903+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33931264,"num_objects":8284,"num_object_clones":0,"num_object_copies":16568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3668,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9158,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4618},{"shards":"0,2","objects":1862}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12440","reported_seq":12759,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:55.035292+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:57:55.035292+0000","last_peered":"2024-09-10T15:57:55.035292+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:57:55.035292+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34181120,"num_objects":8345,"num_object_clones":0,"num_object_copies":16690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3737,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10414,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1787},{"shards":"2","objects":4610}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12246","reported_seq":12519,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:55.062992+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:57:55.062992+0000","last_peered":"2024-09-10T15:57:55.062992+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:57:55.062992+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33628160,"num_objects":8210,"num_object_clones":0,"num_object_copies":16420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10115,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1813},{"shards":"2","objects":4454}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12470","reported_seq":13007,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.792615+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:54.792615+0000","last_peered":"2024-09-10T15:57:54.792615+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:54.792615+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34340864,"num_objects":8384,"num_object_clones":0,"num_object_copies":16768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9662,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4756},{"shards":"0,2","objects":1791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12453","reported_seq":12943,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.727022+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:54.727022+0000","last_peered":"2024-09-10T15:57:54.727022+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:54.727022+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33853440,"num_objects":8265,"num_object_clones":0,"num_object_copies":16530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3757,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8871,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4511},{"shards":"0,2","objects":1827}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12432","reported_seq":12974,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.807831+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:54.807831+0000","last_peered":"2024-09-10T15:57:54.807831+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:54.807831+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136064,"num_objects":8334,"num_object_clones":0,"num_object_copies":16668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3678,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9038,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4658},{"shards":"0,2","objects":1830}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12678","reported_seq":13163,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:54.765644+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:54.765644+0000","last_peered":"2024-09-10T15:57:54.765644+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:54.765644+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34488803,"num_objects":8421,"num_object_clones":0,"num_object_copies":16842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9497,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4591},{"shards":"0,2","objects":1880}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":543203835,"num_objects":132620,"num_object_clones":0,"num_object_copies":265240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37213,"num_objects_misplaced":539,"num_objects_unfound":0,"num_objects_dirty":132620,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160430,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8895,"num_bytes_recovered":29995008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1200734208,"data_stored":1200650191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626490,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607836,"kb_used_data":323528,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93764004,"statfs":{"total":96636764160,"available":96014340096,"internally_reserved":0,"allocated":331292672,"data_stored":330957513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291106478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149773,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":835824,"kb_used_data":461280,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93536016,"statfs":{"total":96636764160,"available":95780880384,"internally_reserved":0,"allocated":472350720,"data_stored":472084667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":383512237},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,1,13,0,0,0,0,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687836,"kb_used_data":389644,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93684004,"statfs":{"total":96636764160,"available":95932420096,"internally_reserved":0,"allocated":398995456,"data_stored":398725331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":471334912,"data_stored":471331299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330858496,"data_stored":330785777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":398540800,"data_stored":398533115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:00.975 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:01.543 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:01.546 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-10T15:58:00.528961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":534147598,"num_objects":130411,"num_object_clones":0,"num_object_copies":260822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37075,"num_objects_misplaced":361,"num_objects_unfound":0,"num_objects_dirty":130411,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162643,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9213,"num_bytes_recovered":30724115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160823,"ondisk_log_size":160823,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2159168,"kb_used_data":1163268,"kb_used_omap":12,"kb_used_meta":995827,"kb_avail":280956352,"statfs":{"total":289910292480,"available":287699304448,"internally_reserved":0,"allocated":1191186432,"data_stored":1190315095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1019726853},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,5,7,5,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26591232,"num_objects":-6492,"num_object_clones":0,"num_object_copies":-12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-327,"num_objects_misplaced":-490,"num_objects_unfound":0,"num_objects_dirty":-6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6492,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":817,"num_bytes_recovered":2007040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004844"},"pg_stats":[{"pgid":"2.f","version":"46'12650","reported_seq":13162,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.813729+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:59.813729+0000","last_peered":"2024-09-10T15:57:59.813729+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:59.813729+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33259520,"num_objects":8120,"num_object_clones":0,"num_object_copies":16240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12650,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4365},{"shards":"0,2","objects":1708}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12786","reported_seq":12894,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:00.051826+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:00.051826+0000","last_peered":"2024-09-10T15:58:00.051826+0000","last_clean":"2024-09-10T15:58:00.051826+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:00.051826+0000","last_undegraded":"2024-09-10T15:58:00.051826+0000","last_fullsized":"2024-09-10T15:58:00.051826+0000","mapping_epoch":36,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33947648,"num_objects":8288,"num_object_clones":0,"num_object_copies":16576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11122,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12502","reported_seq":12805,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:00.140331+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:00.140331+0000","last_peered":"2024-09-10T15:58:00.140331+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:00.140331+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288216,"num_objects":8128,"num_object_clones":0,"num_object_copies":16256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3714,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8128,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10770,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1615},{"shards":"2","objects":4415}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12560","reported_seq":12640,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:00.085130+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:00.085130+0000","last_peered":"2024-09-10T15:58:00.085130+0000","last_clean":"2024-09-10T15:58:00.085130+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:00.085130+0000","last_undegraded":"2024-09-10T15:58:00.085130+0000","last_fullsized":"2024-09-10T15:58:00.085130+0000","mapping_epoch":36,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33267712,"num_objects":8122,"num_object_clones":0,"num_object_copies":16244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12560,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12344","reported_seq":12471,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:59.808645+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:59.808645+0000","last_peered":"2024-09-10T15:57:59.808645+0000","last_clean":"2024-09-10T15:57:59.808645+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:59.808645+0000","last_undegraded":"2024-09-10T15:57:59.808645+0000","last_fullsized":"2024-09-10T15:57:59.808645+0000","mapping_epoch":35,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32714752,"num_objects":7987,"num_object_clones":0,"num_object_copies":15974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9415,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12600","reported_seq":19754,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:59.818294+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:59.818294+0000","last_peered":"2024-09-10T15:57:59.818294+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:59.818294+0000","last_undegraded":"2024-09-10T15:57:59.818294+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33861632,"num_objects":8267,"num_object_clones":0,"num_object_copies":16534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":361,"num_objects_unfound":0,"num_objects_dirty":8267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9318,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3509,"num_bytes_recovered":14364672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7906},{"shards":"0,2","objects":361}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12357","reported_seq":12890,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.734232+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:59.734232+0000","last_peered":"2024-09-10T15:57:59.734232+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:59.734232+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32477184,"num_objects":7929,"num_object_clones":0,"num_object_copies":15858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9007,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4293},{"shards":"0,2","objects":1623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12702","reported_seq":12834,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:59.666983+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:59.666983+0000","last_peered":"2024-09-10T15:57:59.666983+0000","last_clean":"2024-09-10T15:57:59.666983+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:59.666983+0000","last_undegraded":"2024-09-10T15:57:59.666983+0000","last_fullsized":"2024-09-10T15:57:59.666983+0000","mapping_epoch":35,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9461,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12644","reported_seq":20772,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:59.868009+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:59.868009+0000","last_peered":"2024-09-10T15:57:59.868009+0000","last_clean":"2024-09-10T15:57:59.868009+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:59.868009+0000","last_undegraded":"2024-09-10T15:57:59.868009+0000","last_fullsized":"2024-09-10T15:57:59.868009+0000","mapping_epoch":45,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33075200,"num_objects":8075,"num_object_clones":0,"num_object_copies":16150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10621,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12448","reported_seq":13005,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.790474+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:59.790474+0000","last_peered":"2024-09-10T15:57:59.790474+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:59.790474+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33300480,"num_objects":8130,"num_object_clones":0,"num_object_copies":16260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9312,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4482},{"shards":"0,2","objects":1708}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12565","reported_seq":12914,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:00.140473+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:58:00.140473+0000","last_peered":"2024-09-10T15:58:00.140473+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:00.140473+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33669120,"num_objects":8220,"num_object_clones":0,"num_object_copies":16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3722,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10539,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1662},{"shards":"2","objects":4500}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12414","reported_seq":12700,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:00.151814+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:00.151814+0000","last_peered":"2024-09-10T15:58:00.151814+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:00.151814+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32940032,"num_objects":8042,"num_object_clones":0,"num_object_copies":16084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3751,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1645},{"shards":"2","objects":4293}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12594","reported_seq":13164,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.752712+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:59.752712+0000","last_peered":"2024-09-10T15:57:59.752712+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:59.752712+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33832960,"num_objects":8260,"num_object_clones":0,"num_object_copies":16520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9786,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4648},{"shards":"0,2","objects":1667}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12593","reported_seq":13113,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.733205+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:59.733205+0000","last_peered":"2024-09-10T15:57:59.733205+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:59.733205+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33280000,"num_objects":8125,"num_object_clones":0,"num_object_copies":16250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9011,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4386},{"shards":"0,2","objects":1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12549","reported_seq":13105,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.642703+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:59.642703+0000","last_peered":"2024-09-10T15:57:59.642703+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:59.642703+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33656832,"num_objects":8217,"num_object_clones":0,"num_object_copies":16434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9155,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4548},{"shards":"0,2","objects":1713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12812","reported_seq":13325,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.750136+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:59.750136+0000","last_peered":"2024-09-10T15:57:59.750136+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:59.750136+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33939939,"num_objects":8287,"num_object_clones":0,"num_object_copies":16574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9631,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4471},{"shards":"0,2","objects":1746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":534147579,"num_objects":130409,"num_object_clones":0,"num_object_copies":260818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37075,"num_objects_misplaced":361,"num_objects_unfound":0,"num_objects_dirty":130409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162641,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9211,"num_bytes_recovered":30724096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1189105664,"data_stored":1189021647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160820,"ondisk_log_size":160820,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626491,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633840,"kb_used_data":323168,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93738000,"statfs":{"total":96636764160,"available":95987712000,"internally_reserved":0,"allocated":330924032,"data_stored":330588873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318107310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149774,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":837572,"kb_used_data":454708,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93534268,"statfs":{"total":96636764160,"available":95779090432,"internally_reserved":0,"allocated":465620992,"data_stored":465354939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392031917},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687756,"kb_used_data":385392,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93684084,"statfs":{"total":96636764160,"available":95932502016,"internally_reserved":0,"allocated":394641408,"data_stored":394371283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":464527360,"data_stored":464523747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330469376,"data_stored":330396657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":394108928,"data_stored":394101243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:01.573 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:02.108 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:02.109 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:02.141 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-10T15:58:00.528961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":534147598,"num_objects":130411,"num_object_clones":0,"num_object_copies":260822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37075,"num_objects_misplaced":361,"num_objects_unfound":0,"num_objects_dirty":130411,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162643,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9213,"num_bytes_recovered":30724115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160823,"ondisk_log_size":160823,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2159168,"kb_used_data":1163268,"kb_used_omap":12,"kb_used_meta":995827,"kb_avail":280956352,"statfs":{"total":289910292480,"available":287699304448,"internally_reserved":0,"allocated":1191186432,"data_stored":1190315095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1019726853},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,5,7,5,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26591232,"num_objects":-6492,"num_object_clones":0,"num_object_copies":-12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-327,"num_objects_misplaced":-490,"num_objects_unfound":0,"num_objects_dirty":-6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6492,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":817,"num_bytes_recovered":2007040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004844"},"pg_stats":[{"pgid":"2.f","version":"46'12650","reported_seq":13162,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.813729+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:59.813729+0000","last_peered":"2024-09-10T15:57:59.813729+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:59.813729+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33259520,"num_objects":8120,"num_object_clones":0,"num_object_copies":16240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12650,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4365},{"shards":"0,2","objects":1708}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12786","reported_seq":12894,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:00.051826+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:00.051826+0000","last_peered":"2024-09-10T15:58:00.051826+0000","last_clean":"2024-09-10T15:58:00.051826+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:00.051826+0000","last_undegraded":"2024-09-10T15:58:00.051826+0000","last_fullsized":"2024-09-10T15:58:00.051826+0000","mapping_epoch":36,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33947648,"num_objects":8288,"num_object_clones":0,"num_object_copies":16576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11122,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12502","reported_seq":12805,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:00.140331+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:00.140331+0000","last_peered":"2024-09-10T15:58:00.140331+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:00.140331+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288216,"num_objects":8128,"num_object_clones":0,"num_object_copies":16256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3714,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8128,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10770,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1615},{"shards":"2","objects":4415}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12560","reported_seq":12640,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:00.085130+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:00.085130+0000","last_peered":"2024-09-10T15:58:00.085130+0000","last_clean":"2024-09-10T15:58:00.085130+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:00.085130+0000","last_undegraded":"2024-09-10T15:58:00.085130+0000","last_fullsized":"2024-09-10T15:58:00.085130+0000","mapping_epoch":36,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33267712,"num_objects":8122,"num_object_clones":0,"num_object_copies":16244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12560,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12344","reported_seq":12471,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:59.808645+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:59.808645+0000","last_peered":"2024-09-10T15:57:59.808645+0000","last_clean":"2024-09-10T15:57:59.808645+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:59.808645+0000","last_undegraded":"2024-09-10T15:57:59.808645+0000","last_fullsized":"2024-09-10T15:57:59.808645+0000","mapping_epoch":35,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32714752,"num_objects":7987,"num_object_clones":0,"num_object_copies":15974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9415,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12600","reported_seq":19754,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:59.818294+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:59.818294+0000","last_peered":"2024-09-10T15:57:59.818294+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:59.818294+0000","last_undegraded":"2024-09-10T15:57:59.818294+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33861632,"num_objects":8267,"num_object_clones":0,"num_object_copies":16534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":361,"num_objects_unfound":0,"num_objects_dirty":8267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9318,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3509,"num_bytes_recovered":14364672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7906},{"shards":"0,2","objects":361}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12357","reported_seq":12890,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.734232+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:59.734232+0000","last_peered":"2024-09-10T15:57:59.734232+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:59.734232+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32477184,"num_objects":7929,"num_object_clones":0,"num_object_copies":15858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9007,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4293},{"shards":"0,2","objects":1623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12702","reported_seq":12834,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:59.666983+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:59.666983+0000","last_peered":"2024-09-10T15:57:59.666983+0000","last_clean":"2024-09-10T15:57:59.666983+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:59.666983+0000","last_undegraded":"2024-09-10T15:57:59.666983+0000","last_fullsized":"2024-09-10T15:57:59.666983+0000","mapping_epoch":35,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9461,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12644","reported_seq":20772,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:59.868009+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:59.868009+0000","last_peered":"2024-09-10T15:57:59.868009+0000","last_clean":"2024-09-10T15:57:59.868009+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:59.868009+0000","last_undegraded":"2024-09-10T15:57:59.868009+0000","last_fullsized":"2024-09-10T15:57:59.868009+0000","mapping_epoch":45,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33075200,"num_objects":8075,"num_object_clones":0,"num_object_copies":16150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10621,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12448","reported_seq":13005,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.790474+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:59.790474+0000","last_peered":"2024-09-10T15:57:59.790474+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:59.790474+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33300480,"num_objects":8130,"num_object_clones":0,"num_object_copies":16260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9312,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4482},{"shards":"0,2","objects":1708}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12565","reported_seq":12914,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:00.140473+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:58:00.140473+0000","last_peered":"2024-09-10T15:58:00.140473+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:00.140473+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33669120,"num_objects":8220,"num_object_clones":0,"num_object_copies":16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3722,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10539,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1662},{"shards":"2","objects":4500}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12414","reported_seq":12700,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:00.151814+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:00.151814+0000","last_peered":"2024-09-10T15:58:00.151814+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:00.151814+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32940032,"num_objects":8042,"num_object_clones":0,"num_object_copies":16084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3751,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1645},{"shards":"2","objects":4293}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12594","reported_seq":13164,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.752712+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:59.752712+0000","last_peered":"2024-09-10T15:57:59.752712+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:59.752712+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33832960,"num_objects":8260,"num_object_clones":0,"num_object_copies":16520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9786,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4648},{"shards":"0,2","objects":1667}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12593","reported_seq":13113,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.733205+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:59.733205+0000","last_peered":"2024-09-10T15:57:59.733205+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:59.733205+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33280000,"num_objects":8125,"num_object_clones":0,"num_object_copies":16250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9011,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4386},{"shards":"0,2","objects":1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12549","reported_seq":13105,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.642703+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:59.642703+0000","last_peered":"2024-09-10T15:57:59.642703+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:59.642703+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33656832,"num_objects":8217,"num_object_clones":0,"num_object_copies":16434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9155,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4548},{"shards":"0,2","objects":1713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12812","reported_seq":13325,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.750136+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:59.750136+0000","last_peered":"2024-09-10T15:57:59.750136+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:59.750136+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33939939,"num_objects":8287,"num_object_clones":0,"num_object_copies":16574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9631,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4471},{"shards":"0,2","objects":1746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":534147579,"num_objects":130409,"num_object_clones":0,"num_object_copies":260818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37075,"num_objects_misplaced":361,"num_objects_unfound":0,"num_objects_dirty":130409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162641,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9211,"num_bytes_recovered":30724096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1189105664,"data_stored":1189021647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160820,"ondisk_log_size":160820,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626491,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633840,"kb_used_data":323168,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93738000,"statfs":{"total":96636764160,"available":95987712000,"internally_reserved":0,"allocated":330924032,"data_stored":330588873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318107310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149774,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":837572,"kb_used_data":454708,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93534268,"statfs":{"total":96636764160,"available":95779090432,"internally_reserved":0,"allocated":465620992,"data_stored":465354939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392031917},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687756,"kb_used_data":385392,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93684084,"statfs":{"total":96636764160,"available":95932502016,"internally_reserved":0,"allocated":394641408,"data_stored":394371283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":464527360,"data_stored":464523747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330469376,"data_stored":330396657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":394108928,"data_stored":394101243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:02.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:58:02.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'12650", 'reported_seq': 13162, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:59.813729+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:57:59.813729+0000', 'last_peered': '2024-09-10T15:57:59.813729+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:57:59.813729+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33259520, 'num_objects': 8120, 'num_object_clones': 0, 'num_object_copies': 16240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3755, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8120, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12650, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 218, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4365}, {'shards': '0,2', 'objects': 1708}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:02.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:58:02.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'12502", 'reported_seq': 12805, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:00.140331+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:58:00.140331+0000', 'last_peered': '2024-09-10T15:58:00.140331+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:58:00.140331+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'2500", 'ondisk_log_start': "37'2500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33288216, 'num_objects': 8128, 'num_object_clones': 0, 'num_object_copies': 16256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3714, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8128, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10770, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 89, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1615}, {'shards': '2', 'objects': 4415}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:02.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:58:02.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'12600", 'reported_seq': 19754, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:57:59.818294+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:57:59.818294+0000', 'last_peered': '2024-09-10T15:57:59.818294+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:57:59.818294+0000', 'last_undegraded': '2024-09-10T15:57:59.818294+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'2500", 'ondisk_log_start': "37'2500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33861632, 'num_objects': 8267, 'num_object_clones': 0, 'num_object_copies': 16534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 361, 'num_objects_unfound': 0, 'num_objects_dirty': 8267, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9318, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3509, 'num_bytes_recovered': 14364672, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7906}, {'shards': '0,2', 'objects': 361}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:02.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:58:02.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'12357", 'reported_seq': 12890, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:59.734232+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:57:59.734232+0000', 'last_peered': '2024-09-10T15:57:59.734232+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:57:59.734232+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'2300", 'ondisk_log_start': "37'2300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32477184, 'num_objects': 7929, 'num_object_clones': 0, 'num_object_copies': 15858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3638, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7929, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9007, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 199, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4293}, {'shards': '0,2', 'objects': 1623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:02.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:58:02.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'12448", 'reported_seq': 13005, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:59.790474+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:57:59.790474+0000', 'last_peered': '2024-09-10T15:57:59.790474+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:57:59.790474+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33300480, 'num_objects': 8130, 'num_object_clones': 0, 'num_object_copies': 16260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3650, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9312, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 215, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4482}, {'shards': '0,2', 'objects': 1708}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:02.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:58:02.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'12565", 'reported_seq': 12914, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:00.140473+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:58:00.140473+0000', 'last_peered': '2024-09-10T15:58:00.140473+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:58:00.140473+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'2500", 'ondisk_log_start': "37'2500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33669120, 'num_objects': 8220, 'num_object_clones': 0, 'num_object_copies': 16440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3722, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10539, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 112, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1662}, {'shards': '2', 'objects': 4500}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:02.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:58:02.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'12414", 'reported_seq': 12700, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:00.151814+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:58:00.151814+0000', 'last_peered': '2024-09-10T15:58:00.151814+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:58:00.151814+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32940032, 'num_objects': 8042, 'num_object_clones': 0, 'num_object_copies': 16084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3751, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8042, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10283, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 80, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1645}, {'shards': '2', 'objects': 4293}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:02.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:58:02.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'12594", 'reported_seq': 13164, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:59.752712+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:57:59.752712+0000', 'last_peered': '2024-09-10T15:57:59.752712+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:57:59.752712+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'2500", 'ondisk_log_start': "37'2500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33832960, 'num_objects': 8260, 'num_object_clones': 0, 'num_object_copies': 16520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3613, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9786, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 221, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4648}, {'shards': '0,2', 'objects': 1667}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:02.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:58:02.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'12593", 'reported_seq': 13113, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:59.733205+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:57:59.733205+0000', 'last_peered': '2024-09-10T15:57:59.733205+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:57:59.733205+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'2500", 'ondisk_log_start': "37'2500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33280000, 'num_objects': 8125, 'num_object_clones': 0, 'num_object_copies': 16250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3742, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8125, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9011, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 178, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4386}, {'shards': '0,2', 'objects': 1687}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:02.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:58:02.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'12549", 'reported_seq': 13105, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:59.642703+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:57:59.642703+0000', 'last_peered': '2024-09-10T15:57:59.642703+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:57:59.642703+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'2500", 'ondisk_log_start': "37'2500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33656832, 'num_objects': 8217, 'num_object_clones': 0, 'num_object_copies': 16434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3671, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8217, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9155, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 209, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4548}, {'shards': '0,2', 'objects': 1713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:02.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:58:02.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'12812", 'reported_seq': 13325, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:57:59.750136+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:57:59.750136+0000', 'last_peered': '2024-09-10T15:57:59.750136+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:57:59.750136+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33939939, 'num_objects': 8287, 'num_object_clones': 0, 'num_object_copies': 16574, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3819, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8287, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9631, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 187, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4471}, {'shards': '0,2', 'objects': 1746}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:02.149 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:02.852 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:02.853 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37075/260822 objects degraded (14.215%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":408,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":130411,"data_bytes":534147598,"bytes_used":2210988032,"bytes_avail":287699304448,"bytes_total":289910292480,"degraded_objects":37075,"degraded_total":260822,"degraded_ratio":0.14214675142434305,"misplaced_objects":361,"misplaced_total":260822,"misplaced_ratio":0.0013840856982923219,"recovering_objects_per_sec":47,"recovering_bytes_per_sec":113615,"recovering_keys_per_sec":0,"num_objects_recovered":565,"num_bytes_recovered":1363968,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":363},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:58:02.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:58:02.878 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:03.396 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:03.397 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:03.408 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-10T15:58:02.529801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":534147598,"num_objects":130411,"num_object_clones":0,"num_object_copies":260822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37075,"num_objects_misplaced":361,"num_objects_unfound":0,"num_objects_dirty":130411,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162643,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9213,"num_bytes_recovered":30724115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160823,"ondisk_log_size":160823,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2159168,"kb_used_data":1163268,"kb_used_omap":12,"kb_used_meta":995827,"kb_avail":280956352,"statfs":{"total":289910292480,"available":287699304448,"internally_reserved":0,"allocated":1191186432,"data_stored":1190315095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1019726853},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,5,7,5,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17870848,"num_objects":-4363,"num_object_clones":0,"num_object_copies":-8726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":-333,"num_objects_unfound":0,"num_objects_dirty":-4363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4363,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":565,"num_bytes_recovered":1363968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005114"},"pg_stats":[{"pgid":"2.f","version":"46'12650","reported_seq":13162,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.813729+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:57:59.813729+0000","last_peered":"2024-09-10T15:57:59.813729+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:57:59.813729+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33259520,"num_objects":8120,"num_object_clones":0,"num_object_copies":16240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12650,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4365},{"shards":"0,2","objects":1708}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12786","reported_seq":12894,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:00.051826+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:00.051826+0000","last_peered":"2024-09-10T15:58:00.051826+0000","last_clean":"2024-09-10T15:58:00.051826+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:00.051826+0000","last_undegraded":"2024-09-10T15:58:00.051826+0000","last_fullsized":"2024-09-10T15:58:00.051826+0000","mapping_epoch":36,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33947648,"num_objects":8288,"num_object_clones":0,"num_object_copies":16576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11122,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12502","reported_seq":12805,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:00.140331+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:00.140331+0000","last_peered":"2024-09-10T15:58:00.140331+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:00.140331+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288216,"num_objects":8128,"num_object_clones":0,"num_object_copies":16256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3714,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8128,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10770,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1615},{"shards":"2","objects":4415}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12560","reported_seq":12640,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:00.085130+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:00.085130+0000","last_peered":"2024-09-10T15:58:00.085130+0000","last_clean":"2024-09-10T15:58:00.085130+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:00.085130+0000","last_undegraded":"2024-09-10T15:58:00.085130+0000","last_fullsized":"2024-09-10T15:58:00.085130+0000","mapping_epoch":36,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33267712,"num_objects":8122,"num_object_clones":0,"num_object_copies":16244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12560,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12344","reported_seq":12471,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:59.808645+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:57:59.808645+0000","last_peered":"2024-09-10T15:57:59.808645+0000","last_clean":"2024-09-10T15:57:59.808645+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:57:59.808645+0000","last_undegraded":"2024-09-10T15:57:59.808645+0000","last_fullsized":"2024-09-10T15:57:59.808645+0000","mapping_epoch":35,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32714752,"num_objects":7987,"num_object_clones":0,"num_object_copies":15974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9415,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12600","reported_seq":19754,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:57:59.818294+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:57:59.818294+0000","last_peered":"2024-09-10T15:57:59.818294+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:57:59.818294+0000","last_undegraded":"2024-09-10T15:57:59.818294+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33861632,"num_objects":8267,"num_object_clones":0,"num_object_copies":16534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":361,"num_objects_unfound":0,"num_objects_dirty":8267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9318,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3509,"num_bytes_recovered":14364672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7906},{"shards":"0,2","objects":361}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12357","reported_seq":12890,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.734232+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:57:59.734232+0000","last_peered":"2024-09-10T15:57:59.734232+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:57:59.734232+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32477184,"num_objects":7929,"num_object_clones":0,"num_object_copies":15858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9007,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4293},{"shards":"0,2","objects":1623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12702","reported_seq":12834,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:59.666983+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:57:59.666983+0000","last_peered":"2024-09-10T15:57:59.666983+0000","last_clean":"2024-09-10T15:57:59.666983+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:57:59.666983+0000","last_undegraded":"2024-09-10T15:57:59.666983+0000","last_fullsized":"2024-09-10T15:57:59.666983+0000","mapping_epoch":35,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9461,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12644","reported_seq":20772,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:57:59.868009+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:57:59.868009+0000","last_peered":"2024-09-10T15:57:59.868009+0000","last_clean":"2024-09-10T15:57:59.868009+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:57:59.868009+0000","last_undegraded":"2024-09-10T15:57:59.868009+0000","last_fullsized":"2024-09-10T15:57:59.868009+0000","mapping_epoch":45,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33075200,"num_objects":8075,"num_object_clones":0,"num_object_copies":16150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10621,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12448","reported_seq":13005,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.790474+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:57:59.790474+0000","last_peered":"2024-09-10T15:57:59.790474+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:57:59.790474+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33300480,"num_objects":8130,"num_object_clones":0,"num_object_copies":16260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9312,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4482},{"shards":"0,2","objects":1708}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12565","reported_seq":12914,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:00.140473+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:58:00.140473+0000","last_peered":"2024-09-10T15:58:00.140473+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:00.140473+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33669120,"num_objects":8220,"num_object_clones":0,"num_object_copies":16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3722,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10539,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1662},{"shards":"2","objects":4500}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12414","reported_seq":12700,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:00.151814+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:00.151814+0000","last_peered":"2024-09-10T15:58:00.151814+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:00.151814+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32940032,"num_objects":8042,"num_object_clones":0,"num_object_copies":16084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3751,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1645},{"shards":"2","objects":4293}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12594","reported_seq":13164,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.752712+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:57:59.752712+0000","last_peered":"2024-09-10T15:57:59.752712+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:57:59.752712+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33832960,"num_objects":8260,"num_object_clones":0,"num_object_copies":16520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9786,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4648},{"shards":"0,2","objects":1667}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12593","reported_seq":13113,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.733205+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:57:59.733205+0000","last_peered":"2024-09-10T15:57:59.733205+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:57:59.733205+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33280000,"num_objects":8125,"num_object_clones":0,"num_object_copies":16250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9011,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4386},{"shards":"0,2","objects":1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12549","reported_seq":13105,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.642703+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:57:59.642703+0000","last_peered":"2024-09-10T15:57:59.642703+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:57:59.642703+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33656832,"num_objects":8217,"num_object_clones":0,"num_object_copies":16434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9155,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4548},{"shards":"0,2","objects":1713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12812","reported_seq":13325,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:57:59.750136+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:57:59.750136+0000","last_peered":"2024-09-10T15:57:59.750136+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:57:59.750136+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33939939,"num_objects":8287,"num_object_clones":0,"num_object_copies":16574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3819,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9631,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4471},{"shards":"0,2","objects":1746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":534147579,"num_objects":130409,"num_object_clones":0,"num_object_copies":260818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37075,"num_objects_misplaced":361,"num_objects_unfound":0,"num_objects_dirty":130409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162641,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9211,"num_bytes_recovered":30724096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1189105664,"data_stored":1189021647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160820,"ondisk_log_size":160820,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626491,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633840,"kb_used_data":323168,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93738000,"statfs":{"total":96636764160,"available":95987712000,"internally_reserved":0,"allocated":330924032,"data_stored":330588873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318107310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149774,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":837572,"kb_used_data":454708,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93534268,"statfs":{"total":96636764160,"available":95779090432,"internally_reserved":0,"allocated":465620992,"data_stored":465354939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392031917},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687756,"kb_used_data":385392,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93684084,"statfs":{"total":96636764160,"available":95932502016,"internally_reserved":0,"allocated":394641408,"data_stored":394371283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":464527360,"data_stored":464523747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330469376,"data_stored":330396657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":394108928,"data_stored":394101243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:06.412 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:06.943 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:06.944 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:06.966 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-10T15:58:06.531514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525177358,"num_objects":128221,"num_object_clones":0,"num_object_copies":256442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36995,"num_objects_misplaced":210,"num_objects_unfound":0,"num_objects_dirty":128221,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164833,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9444,"num_bytes_recovered":31342611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2126896,"kb_used_data":1151804,"kb_used_omap":12,"kb_used_meta":975027,"kb_avail":280988624,"statfs":{"total":289910292480,"available":287732350976,"internally_reserved":0,"allocated":1179447296,"data_stored":1178575959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":998427653},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,9,4,6,3,0,0,7],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26841088,"num_objects":-6553,"num_object_clones":0,"num_object_copies":-13106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-312,"num_objects_misplaced":-484,"num_objects_unfound":0,"num_objects_dirty":-6553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6553,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":796,"num_bytes_recovered":1982464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005545"},"pg_stats":[{"pgid":"2.f","version":"46'12800","reported_seq":13343,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.751126+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:04.751126+0000","last_peered":"2024-09-10T15:58:04.751126+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:04.751126+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32645120,"num_objects":7970,"num_object_clones":0,"num_object_copies":15940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12800,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4230},{"shards":"0,2","objects":1558}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12924","reported_seq":13032,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:05.158473+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:05.158473+0000","last_peered":"2024-09-10T15:58:05.158473+0000","last_clean":"2024-09-10T15:58:05.158473+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:05.158473+0000","last_undegraded":"2024-09-10T15:58:05.158473+0000","last_fullsized":"2024-09-10T15:58:05.158473+0000","mapping_epoch":36,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382400,"num_objects":8150,"num_object_clones":0,"num_object_copies":16300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11260,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12625","reported_seq":12938,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:05.140469+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:05.140469+0000","last_peered":"2024-09-10T15:58:05.140469+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:05.140469+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32784408,"num_objects":8005,"num_object_clones":0,"num_object_copies":16010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8005,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10893,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1492},{"shards":"2","objects":4297}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12698","reported_seq":12778,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:05.158062+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:05.158062+0000","last_peered":"2024-09-10T15:58:05.158062+0000","last_clean":"2024-09-10T15:58:05.158062+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:05.158062+0000","last_undegraded":"2024-09-10T15:58:05.158062+0000","last_fullsized":"2024-09-10T15:58:05.158062+0000","mapping_epoch":36,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32702464,"num_objects":7984,"num_object_clones":0,"num_object_copies":15968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12698,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12481","reported_seq":12608,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:04.816446+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:04.816446+0000","last_peered":"2024-09-10T15:58:04.816446+0000","last_clean":"2024-09-10T15:58:04.816446+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:04.816446+0000","last_undegraded":"2024-09-10T15:58:04.816446+0000","last_fullsized":"2024-09-10T15:58:04.816446+0000","mapping_epoch":35,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32153600,"num_objects":7850,"num_object_clones":0,"num_object_copies":15700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9552,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12744","reported_seq":20200,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:58:04.824512+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:58:04.824512+0000","last_peered":"2024-09-10T15:58:04.824512+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:58:04.824512+0000","last_undegraded":"2024-09-10T15:58:04.824512+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33271808,"num_objects":8123,"num_object_clones":0,"num_object_copies":16246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":210,"num_objects_unfound":0,"num_objects_dirty":8123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9462,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3660,"num_bytes_recovered":14983168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7913},{"shards":"0,2","objects":210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12483","reported_seq":13032,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.771507+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:04.771507+0000","last_peered":"2024-09-10T15:58:04.771507+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:04.771507+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31961088,"num_objects":7803,"num_object_clones":0,"num_object_copies":15606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9133,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4173},{"shards":"0,2","objects":1497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12846","reported_seq":12978,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:04.775982+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:04.775982+0000","last_peered":"2024-09-10T15:58:04.775982+0000","last_clean":"2024-09-10T15:58:04.775982+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:04.775982+0000","last_undegraded":"2024-09-10T15:58:04.775982+0000","last_fullsized":"2024-09-10T15:58:04.775982+0000","mapping_epoch":35,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33046528,"num_objects":8068,"num_object_clones":0,"num_object_copies":16136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9605,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12788","reported_seq":20916,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:04.804707+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:04.804707+0000","last_peered":"2024-09-10T15:58:04.804707+0000","last_clean":"2024-09-10T15:58:04.804707+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:04.804707+0000","last_undegraded":"2024-09-10T15:58:04.804707+0000","last_fullsized":"2024-09-10T15:58:04.804707+0000","mapping_epoch":45,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32485376,"num_objects":7931,"num_object_clones":0,"num_object_copies":15862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10765,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12601","reported_seq":13190,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.831151+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:04.831151+0000","last_peered":"2024-09-10T15:58:04.831151+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:04.831151+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32673792,"num_objects":7977,"num_object_clones":0,"num_object_copies":15954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3633,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9465,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4346},{"shards":"0,2","objects":1555}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12678","reported_seq":13031,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:05.070614+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:58:05.070614+0000","last_peered":"2024-09-10T15:58:05.070614+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:05.070614+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33206272,"num_objects":8107,"num_object_clones":0,"num_object_copies":16214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10652,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1549},{"shards":"2","objects":4389}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12528","reported_seq":12819,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:05.102725+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:05.102725+0000","last_peered":"2024-09-10T15:58:05.102725+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:05.102725+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32473088,"num_objects":7928,"num_object_clones":0,"num_object_copies":15856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3749,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10397,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1531},{"shards":"2","objects":4181}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12742","reported_seq":13336,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.802598+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:04.802598+0000","last_peered":"2024-09-10T15:58:04.802598+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:04.802598+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33226752,"num_objects":8112,"num_object_clones":0,"num_object_copies":16224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3601,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9934,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4512},{"shards":"0,2","objects":1519}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12730","reported_seq":13262,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.848643+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:04.848643+0000","last_peered":"2024-09-10T15:58:04.848643+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:04.848643+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32718848,"num_objects":7988,"num_object_clones":0,"num_object_copies":15976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3736,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9148,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":1550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12682","reported_seq":13248,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.742108+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:04.742108+0000","last_peered":"2024-09-10T15:58:04.742108+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:04.742108+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33112064,"num_objects":8084,"num_object_clones":0,"num_object_copies":16168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9288,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4420},{"shards":"0,2","objects":1580}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12960","reported_seq":13490,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.791541+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:04.791541+0000","last_peered":"2024-09-10T15:58:04.791541+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:04.791541+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33333731,"num_objects":8139,"num_object_clones":0,"num_object_copies":16278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9779,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4333},{"shards":"0,2","objects":1598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525177339,"num_objects":128219,"num_object_clones":0,"num_object_copies":256438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36995,"num_objects_misplaced":210,"num_objects_unfound":0,"num_objects_dirty":128219,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164831,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9442,"num_bytes_recovered":31342592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1177272320,"data_stored":1177188303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160910,"ondisk_log_size":160910,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626492,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620748,"kb_used_data":322556,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93751092,"statfs":{"total":96636764160,"available":96001118208,"internally_reserved":0,"allocated":330297344,"data_stored":329962185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305327790},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149775,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":818484,"kb_used_data":448096,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93553356,"statfs":{"total":96636764160,"available":95798636544,"internally_reserved":0,"allocated":458850304,"data_stored":458584251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":379252397},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,6,3,2,3,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687664,"kb_used_data":381152,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93684176,"statfs":{"total":96636764160,"available":95932596224,"internally_reserved":0,"allocated":390299648,"data_stored":390029523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,3,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":457584640,"data_stored":457581027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329908224,"data_stored":329835505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389779456,"data_stored":389771771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:06.968 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:07.535 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:07.535 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:07.563 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-10T15:58:06.531514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525177358,"num_objects":128221,"num_object_clones":0,"num_object_copies":256442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36995,"num_objects_misplaced":210,"num_objects_unfound":0,"num_objects_dirty":128221,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164833,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9444,"num_bytes_recovered":31342611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2126896,"kb_used_data":1151804,"kb_used_omap":12,"kb_used_meta":975027,"kb_avail":280988624,"statfs":{"total":289910292480,"available":287732350976,"internally_reserved":0,"allocated":1179447296,"data_stored":1178575959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":998427653},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,9,4,6,3,0,0,7],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26841088,"num_objects":-6553,"num_object_clones":0,"num_object_copies":-13106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-312,"num_objects_misplaced":-484,"num_objects_unfound":0,"num_objects_dirty":-6553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6553,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":796,"num_bytes_recovered":1982464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005545"},"pg_stats":[{"pgid":"2.f","version":"46'12800","reported_seq":13343,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.751126+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:04.751126+0000","last_peered":"2024-09-10T15:58:04.751126+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:04.751126+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32645120,"num_objects":7970,"num_object_clones":0,"num_object_copies":15940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12800,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4230},{"shards":"0,2","objects":1558}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12924","reported_seq":13032,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:05.158473+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:05.158473+0000","last_peered":"2024-09-10T15:58:05.158473+0000","last_clean":"2024-09-10T15:58:05.158473+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:05.158473+0000","last_undegraded":"2024-09-10T15:58:05.158473+0000","last_fullsized":"2024-09-10T15:58:05.158473+0000","mapping_epoch":36,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382400,"num_objects":8150,"num_object_clones":0,"num_object_copies":16300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11260,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12625","reported_seq":12938,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:05.140469+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:05.140469+0000","last_peered":"2024-09-10T15:58:05.140469+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:05.140469+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32784408,"num_objects":8005,"num_object_clones":0,"num_object_copies":16010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8005,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10893,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1492},{"shards":"2","objects":4297}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12698","reported_seq":12778,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:05.158062+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:05.158062+0000","last_peered":"2024-09-10T15:58:05.158062+0000","last_clean":"2024-09-10T15:58:05.158062+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:05.158062+0000","last_undegraded":"2024-09-10T15:58:05.158062+0000","last_fullsized":"2024-09-10T15:58:05.158062+0000","mapping_epoch":36,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32702464,"num_objects":7984,"num_object_clones":0,"num_object_copies":15968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12698,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12481","reported_seq":12608,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:04.816446+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:04.816446+0000","last_peered":"2024-09-10T15:58:04.816446+0000","last_clean":"2024-09-10T15:58:04.816446+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:04.816446+0000","last_undegraded":"2024-09-10T15:58:04.816446+0000","last_fullsized":"2024-09-10T15:58:04.816446+0000","mapping_epoch":35,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32153600,"num_objects":7850,"num_object_clones":0,"num_object_copies":15700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9552,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12744","reported_seq":20200,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:58:04.824512+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:58:04.824512+0000","last_peered":"2024-09-10T15:58:04.824512+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:58:04.824512+0000","last_undegraded":"2024-09-10T15:58:04.824512+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33271808,"num_objects":8123,"num_object_clones":0,"num_object_copies":16246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":210,"num_objects_unfound":0,"num_objects_dirty":8123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9462,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3660,"num_bytes_recovered":14983168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7913},{"shards":"0,2","objects":210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12483","reported_seq":13032,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.771507+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:04.771507+0000","last_peered":"2024-09-10T15:58:04.771507+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:04.771507+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31961088,"num_objects":7803,"num_object_clones":0,"num_object_copies":15606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9133,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4173},{"shards":"0,2","objects":1497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12846","reported_seq":12978,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:04.775982+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:04.775982+0000","last_peered":"2024-09-10T15:58:04.775982+0000","last_clean":"2024-09-10T15:58:04.775982+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:04.775982+0000","last_undegraded":"2024-09-10T15:58:04.775982+0000","last_fullsized":"2024-09-10T15:58:04.775982+0000","mapping_epoch":35,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33046528,"num_objects":8068,"num_object_clones":0,"num_object_copies":16136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9605,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12788","reported_seq":20916,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:04.804707+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:04.804707+0000","last_peered":"2024-09-10T15:58:04.804707+0000","last_clean":"2024-09-10T15:58:04.804707+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:04.804707+0000","last_undegraded":"2024-09-10T15:58:04.804707+0000","last_fullsized":"2024-09-10T15:58:04.804707+0000","mapping_epoch":45,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32485376,"num_objects":7931,"num_object_clones":0,"num_object_copies":15862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10765,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12601","reported_seq":13190,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.831151+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:04.831151+0000","last_peered":"2024-09-10T15:58:04.831151+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:04.831151+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32673792,"num_objects":7977,"num_object_clones":0,"num_object_copies":15954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3633,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9465,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4346},{"shards":"0,2","objects":1555}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12678","reported_seq":13031,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:05.070614+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:58:05.070614+0000","last_peered":"2024-09-10T15:58:05.070614+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:05.070614+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33206272,"num_objects":8107,"num_object_clones":0,"num_object_copies":16214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10652,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1549},{"shards":"2","objects":4389}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12528","reported_seq":12819,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:05.102725+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:05.102725+0000","last_peered":"2024-09-10T15:58:05.102725+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:05.102725+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32473088,"num_objects":7928,"num_object_clones":0,"num_object_copies":15856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3749,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10397,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1531},{"shards":"2","objects":4181}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12742","reported_seq":13336,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.802598+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:04.802598+0000","last_peered":"2024-09-10T15:58:04.802598+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:04.802598+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33226752,"num_objects":8112,"num_object_clones":0,"num_object_copies":16224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3601,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9934,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4512},{"shards":"0,2","objects":1519}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12730","reported_seq":13262,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.848643+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:04.848643+0000","last_peered":"2024-09-10T15:58:04.848643+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:04.848643+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32718848,"num_objects":7988,"num_object_clones":0,"num_object_copies":15976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3736,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9148,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":1550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12682","reported_seq":13248,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.742108+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:04.742108+0000","last_peered":"2024-09-10T15:58:04.742108+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:04.742108+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33112064,"num_objects":8084,"num_object_clones":0,"num_object_copies":16168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9288,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4420},{"shards":"0,2","objects":1580}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12960","reported_seq":13490,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.791541+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:04.791541+0000","last_peered":"2024-09-10T15:58:04.791541+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:04.791541+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33333731,"num_objects":8139,"num_object_clones":0,"num_object_copies":16278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9779,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4333},{"shards":"0,2","objects":1598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525177339,"num_objects":128219,"num_object_clones":0,"num_object_copies":256438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36995,"num_objects_misplaced":210,"num_objects_unfound":0,"num_objects_dirty":128219,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164831,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9442,"num_bytes_recovered":31342592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1177272320,"data_stored":1177188303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160910,"ondisk_log_size":160910,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626492,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620748,"kb_used_data":322556,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93751092,"statfs":{"total":96636764160,"available":96001118208,"internally_reserved":0,"allocated":330297344,"data_stored":329962185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305327790},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149775,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":818484,"kb_used_data":448096,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93553356,"statfs":{"total":96636764160,"available":95798636544,"internally_reserved":0,"allocated":458850304,"data_stored":458584251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":379252397},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,6,3,2,3,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687664,"kb_used_data":381152,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93684176,"statfs":{"total":96636764160,"available":95932596224,"internally_reserved":0,"allocated":390299648,"data_stored":390029523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,3,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":457584640,"data_stored":457581027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329908224,"data_stored":329835505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389779456,"data_stored":389771771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:07.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:58:07.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'12800", 'reported_seq': 13343, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:04.751126+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:58:04.751126+0000', 'last_peered': '2024-09-10T15:58:04.751126+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:58:04.751126+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'2700", 'ondisk_log_start': "37'2700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32645120, 'num_objects': 7970, 'num_object_clones': 0, 'num_object_copies': 15940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3740, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7970, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12800, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 233, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4230}, {'shards': '0,2', 'objects': 1558}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:07.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:58:07.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'12625", 'reported_seq': 12938, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:05.140469+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:58:05.140469+0000', 'last_peered': '2024-09-10T15:58:05.140469+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:58:05.140469+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32784408, 'num_objects': 8005, 'num_object_clones': 0, 'num_object_copies': 16010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3709, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8005, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10893, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 94, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1492}, {'shards': '2', 'objects': 4297}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:07.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:58:07.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'12744", 'reported_seq': 20200, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:58:04.824512+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:58:04.824512+0000', 'last_peered': '2024-09-10T15:58:04.824512+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:58:04.824512+0000', 'last_undegraded': '2024-09-10T15:58:04.824512+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'2700", 'ondisk_log_start': "37'2700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33271808, 'num_objects': 8123, 'num_object_clones': 0, 'num_object_copies': 16246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 210, 'num_objects_unfound': 0, 'num_objects_dirty': 8123, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9462, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3660, 'num_bytes_recovered': 14983168, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7913}, {'shards': '0,2', 'objects': 210}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:07.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:58:07.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'12483", 'reported_seq': 13032, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:04.771507+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:58:04.771507+0000', 'last_peered': '2024-09-10T15:58:04.771507+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:58:04.771507+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31961088, 'num_objects': 7803, 'num_object_clones': 0, 'num_object_copies': 15606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3632, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7803, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9133, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 205, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4173}, {'shards': '0,2', 'objects': 1497}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:07.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:58:07.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'12601", 'reported_seq': 13190, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:04.831151+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:58:04.831151+0000', 'last_peered': '2024-09-10T15:58:04.831151+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:58:04.831151+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32673792, 'num_objects': 7977, 'num_object_clones': 0, 'num_object_copies': 15954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3633, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7977, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9465, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 232, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4346}, {'shards': '0,2', 'objects': 1555}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:07.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:58:07.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'12678", 'reported_seq': 13031, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:05.070614+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:58:05.070614+0000', 'last_peered': '2024-09-10T15:58:05.070614+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:58:05.070614+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33206272, 'num_objects': 8107, 'num_object_clones': 0, 'num_object_copies': 16214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3720, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8107, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10652, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 114, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1549}, {'shards': '2', 'objects': 4389}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:07.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:58:07.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'12528", 'reported_seq': 12819, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:05.102725+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:58:05.102725+0000', 'last_peered': '2024-09-10T15:58:05.102725+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:58:05.102725+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'2500", 'ondisk_log_start': "37'2500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32473088, 'num_objects': 7928, 'num_object_clones': 0, 'num_object_copies': 15856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3749, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7928, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10397, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 82, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1531}, {'shards': '2', 'objects': 4181}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:07.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:58:07.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'12742", 'reported_seq': 13336, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:04.802598+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:58:04.802598+0000', 'last_peered': '2024-09-10T15:58:04.802598+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:58:04.802598+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'2700", 'ondisk_log_start': "37'2700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33226752, 'num_objects': 8112, 'num_object_clones': 0, 'num_object_copies': 16224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3601, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8112, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9934, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 233, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4512}, {'shards': '0,2', 'objects': 1519}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:07.566 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:58:07.566 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'12730", 'reported_seq': 13262, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:04.848643+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:58:04.848643+0000', 'last_peered': '2024-09-10T15:58:04.848643+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:58:04.848643+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'2700", 'ondisk_log_start': "37'2700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32718848, 'num_objects': 7988, 'num_object_clones': 0, 'num_object_copies': 15976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3736, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7988, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9148, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 184, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4255}, {'shards': '0,2', 'objects': 1550}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:07.566 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:58:07.566 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'12682", 'reported_seq': 13248, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:04.742108+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:58:04.742108+0000', 'last_peered': '2024-09-10T15:58:04.742108+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:58:04.742108+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33112064, 'num_objects': 8084, 'num_object_clones': 0, 'num_object_copies': 16168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3666, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8084, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9288, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 214, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4420}, {'shards': '0,2', 'objects': 1580}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:07.566 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:58:07.566 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'12960", 'reported_seq': 13490, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:04.791541+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:58:04.791541+0000', 'last_peered': '2024-09-10T15:58:04.791541+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:58:04.791541+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'2900", 'ondisk_log_start': "37'2900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33333731, 'num_objects': 8139, 'num_object_clones': 0, 'num_object_copies': 16278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3809, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8139, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9779, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 197, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4333}, {'shards': '0,2', 'objects': 1598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:07.566 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:08.251 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:08.251 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36995/256442 objects degraded (14.426%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":414,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":128221,"data_bytes":525177358,"bytes_used":2177941504,"bytes_avail":287732350976,"bytes_total":289910292480,"degraded_objects":36995,"degraded_total":256442,"degraded_ratio":0.14426264028513269,"misplaced_objects":210,"misplaced_total":256442,"misplaced_ratio":0.00081889862035080055,"recovering_objects_per_sec":66,"recovering_bytes_per_sec":165129,"recovering_keys_per_sec":0,"num_objects_recovered":796,"num_bytes_recovered":1982464,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":545},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:58:08.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:58:08.273 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:08.831 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:08.832 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:08.855 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-10T15:58:08.532228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525177358,"num_objects":128221,"num_object_clones":0,"num_object_copies":256442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36995,"num_objects_misplaced":210,"num_objects_unfound":0,"num_objects_dirty":128221,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164833,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9444,"num_bytes_recovered":31342611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2126896,"kb_used_data":1151804,"kb_used_omap":12,"kb_used_meta":975027,"kb_avail":280988624,"statfs":{"total":289910292480,"available":287732350976,"internally_reserved":0,"allocated":1179447296,"data_stored":1178575959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":998427653},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,9,4,6,3,0,0,7],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18026496,"num_objects":-4401,"num_object_clones":0,"num_object_copies":-8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-218,"num_objects_misplaced":-329,"num_objects_unfound":0,"num_objects_dirty":-4401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4401,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":1347584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005594"},"pg_stats":[{"pgid":"2.f","version":"46'12800","reported_seq":13343,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.751126+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:04.751126+0000","last_peered":"2024-09-10T15:58:04.751126+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:04.751126+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32645120,"num_objects":7970,"num_object_clones":0,"num_object_copies":15940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12800,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4230},{"shards":"0,2","objects":1558}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'12924","reported_seq":13032,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:05.158473+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:05.158473+0000","last_peered":"2024-09-10T15:58:05.158473+0000","last_clean":"2024-09-10T15:58:05.158473+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:05.158473+0000","last_undegraded":"2024-09-10T15:58:05.158473+0000","last_fullsized":"2024-09-10T15:58:05.158473+0000","mapping_epoch":36,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382400,"num_objects":8150,"num_object_clones":0,"num_object_copies":16300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11260,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12625","reported_seq":12938,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:05.140469+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:05.140469+0000","last_peered":"2024-09-10T15:58:05.140469+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:05.140469+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32784408,"num_objects":8005,"num_object_clones":0,"num_object_copies":16010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8005,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10893,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1492},{"shards":"2","objects":4297}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12698","reported_seq":12778,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:05.158062+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:05.158062+0000","last_peered":"2024-09-10T15:58:05.158062+0000","last_clean":"2024-09-10T15:58:05.158062+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:05.158062+0000","last_undegraded":"2024-09-10T15:58:05.158062+0000","last_fullsized":"2024-09-10T15:58:05.158062+0000","mapping_epoch":36,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32702464,"num_objects":7984,"num_object_clones":0,"num_object_copies":15968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12698,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12481","reported_seq":12608,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:04.816446+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:04.816446+0000","last_peered":"2024-09-10T15:58:04.816446+0000","last_clean":"2024-09-10T15:58:04.816446+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:04.816446+0000","last_undegraded":"2024-09-10T15:58:04.816446+0000","last_fullsized":"2024-09-10T15:58:04.816446+0000","mapping_epoch":35,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32153600,"num_objects":7850,"num_object_clones":0,"num_object_copies":15700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9552,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12744","reported_seq":20200,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:58:04.824512+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:58:04.824512+0000","last_peered":"2024-09-10T15:58:04.824512+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:58:04.824512+0000","last_undegraded":"2024-09-10T15:58:04.824512+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33271808,"num_objects":8123,"num_object_clones":0,"num_object_copies":16246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":210,"num_objects_unfound":0,"num_objects_dirty":8123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9462,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3660,"num_bytes_recovered":14983168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7913},{"shards":"0,2","objects":210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12483","reported_seq":13032,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.771507+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:04.771507+0000","last_peered":"2024-09-10T15:58:04.771507+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:04.771507+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31961088,"num_objects":7803,"num_object_clones":0,"num_object_copies":15606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9133,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4173},{"shards":"0,2","objects":1497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12846","reported_seq":12978,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:04.775982+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:04.775982+0000","last_peered":"2024-09-10T15:58:04.775982+0000","last_clean":"2024-09-10T15:58:04.775982+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:04.775982+0000","last_undegraded":"2024-09-10T15:58:04.775982+0000","last_fullsized":"2024-09-10T15:58:04.775982+0000","mapping_epoch":35,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33046528,"num_objects":8068,"num_object_clones":0,"num_object_copies":16136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9605,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12788","reported_seq":20916,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:04.804707+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:04.804707+0000","last_peered":"2024-09-10T15:58:04.804707+0000","last_clean":"2024-09-10T15:58:04.804707+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:04.804707+0000","last_undegraded":"2024-09-10T15:58:04.804707+0000","last_fullsized":"2024-09-10T15:58:04.804707+0000","mapping_epoch":45,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32485376,"num_objects":7931,"num_object_clones":0,"num_object_copies":15862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10765,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12601","reported_seq":13190,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.831151+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:04.831151+0000","last_peered":"2024-09-10T15:58:04.831151+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:04.831151+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32673792,"num_objects":7977,"num_object_clones":0,"num_object_copies":15954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3633,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9465,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4346},{"shards":"0,2","objects":1555}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12678","reported_seq":13031,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:05.070614+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:58:05.070614+0000","last_peered":"2024-09-10T15:58:05.070614+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:05.070614+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33206272,"num_objects":8107,"num_object_clones":0,"num_object_copies":16214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10652,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1549},{"shards":"2","objects":4389}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12528","reported_seq":12819,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:05.102725+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:05.102725+0000","last_peered":"2024-09-10T15:58:05.102725+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:05.102725+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32473088,"num_objects":7928,"num_object_clones":0,"num_object_copies":15856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3749,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10397,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1531},{"shards":"2","objects":4181}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12742","reported_seq":13336,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.802598+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:04.802598+0000","last_peered":"2024-09-10T15:58:04.802598+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:04.802598+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33226752,"num_objects":8112,"num_object_clones":0,"num_object_copies":16224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3601,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9934,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4512},{"shards":"0,2","objects":1519}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12730","reported_seq":13262,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.848643+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:04.848643+0000","last_peered":"2024-09-10T15:58:04.848643+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:04.848643+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32718848,"num_objects":7988,"num_object_clones":0,"num_object_copies":15976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3736,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9148,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":1550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12682","reported_seq":13248,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.742108+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:04.742108+0000","last_peered":"2024-09-10T15:58:04.742108+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:04.742108+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33112064,"num_objects":8084,"num_object_clones":0,"num_object_copies":16168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9288,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4420},{"shards":"0,2","objects":1580}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'12960","reported_seq":13490,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:04.791541+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:04.791541+0000","last_peered":"2024-09-10T15:58:04.791541+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:04.791541+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33333731,"num_objects":8139,"num_object_clones":0,"num_object_copies":16278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9779,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4333},{"shards":"0,2","objects":1598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525177339,"num_objects":128219,"num_object_clones":0,"num_object_copies":256438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36995,"num_objects_misplaced":210,"num_objects_unfound":0,"num_objects_dirty":128219,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164831,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9442,"num_bytes_recovered":31342592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1177272320,"data_stored":1177188303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160910,"ondisk_log_size":160910,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626492,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620748,"kb_used_data":322556,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93751092,"statfs":{"total":96636764160,"available":96001118208,"internally_reserved":0,"allocated":330297344,"data_stored":329962185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305327790},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149775,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":818484,"kb_used_data":448096,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93553356,"statfs":{"total":96636764160,"available":95798636544,"internally_reserved":0,"allocated":458850304,"data_stored":458584251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":379252397},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,6,3,2,3,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687664,"kb_used_data":381152,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93684176,"statfs":{"total":96636764160,"available":95932596224,"internally_reserved":0,"allocated":390299648,"data_stored":390029523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,3,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":457584640,"data_stored":457581027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329908224,"data_stored":329835505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389779456,"data_stored":389771771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:11.857 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:12.341 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:12.341 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:12.356 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-10T15:58:11.735040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":515867150,"num_objects":125948,"num_object_clones":0,"num_object_copies":251896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36899,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":125948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167106,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9721,"num_bytes_recovered":32083987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160786,"ondisk_log_size":160786,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2135704,"kb_used_data":1139816,"kb_used_omap":12,"kb_used_meta":995827,"kb_avail":280979816,"statfs":{"total":289910292480,"available":287723331584,"internally_reserved":0,"allocated":1167171584,"data_stored":1166300247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1019726853},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,5,3,6,6,2,0,0,3,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18280448,"num_objects":-4463,"num_object_clones":0,"num_object_copies":-8926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-176,"num_objects_misplaced":-332,"num_objects_unfound":0,"num_objects_dirty":-4463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4463,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":508,"num_bytes_recovered":1359872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.206078"},"pg_stats":[{"pgid":"2.f","version":"46'12944","reported_seq":13505,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.838043+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:09.838043+0000","last_peered":"2024-09-10T15:58:09.838043+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:09.838043+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32055296,"num_objects":7826,"num_object_clones":0,"num_object_copies":15652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4095},{"shards":"0,2","objects":1414}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'13088","reported_seq":13196,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:10.151164+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:10.151164+0000","last_peered":"2024-09-10T15:58:10.151164+0000","last_clean":"2024-09-10T15:58:10.151164+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:10.151164+0000","last_undegraded":"2024-09-10T15:58:10.151164+0000","last_fullsized":"2024-09-10T15:58:10.151164+0000","mapping_epoch":36,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32710656,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11424,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12751","reported_seq":13090,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:10.132156+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:10.132156+0000","last_peered":"2024-09-10T15:58:10.132156+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:10.132156+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32268312,"num_objects":7879,"num_object_clones":0,"num_object_copies":15758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3696,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7879,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11019,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1366},{"shards":"2","objects":4184}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12851","reported_seq":12931,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:10.157447+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:10.157447+0000","last_peered":"2024-09-10T15:58:10.157447+0000","last_clean":"2024-09-10T15:58:10.157447+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:10.157447+0000","last_undegraded":"2024-09-10T15:58:10.157447+0000","last_fullsized":"2024-09-10T15:58:10.157447+0000","mapping_epoch":36,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32075776,"num_objects":7831,"num_object_clones":0,"num_object_copies":15662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12851,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12610","reported_seq":12737,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:09.818705+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:09.818705+0000","last_peered":"2024-09-10T15:58:09.818705+0000","last_clean":"2024-09-10T15:58:09.818705+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:09.818705+0000","last_undegraded":"2024-09-10T15:58:09.818705+0000","last_fullsized":"2024-09-10T15:58:09.818705+0000","mapping_epoch":35,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31625216,"num_objects":7721,"num_object_clones":0,"num_object_copies":15442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9681,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12873","reported_seq":20691,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:58:09.854159+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:58:09.854159+0000","last_peered":"2024-09-10T15:58:09.854159+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:58:09.854159+0000","last_undegraded":"2024-09-10T15:58:09.854159+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32743424,"num_objects":7994,"num_object_clones":0,"num_object_copies":15988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":7994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9591,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3841,"num_bytes_recovered":15724544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7965},{"shards":"0,2","objects":29}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12620","reported_seq":13187,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.862391+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:09.862391+0000","last_peered":"2024-09-10T15:58:09.862391+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:09.862391+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31399936,"num_objects":7666,"num_object_clones":0,"num_object_copies":15332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3623,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9270,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4045},{"shards":"0,2","objects":1360}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12984","reported_seq":13116,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:09.820931+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:09.820931+0000","last_peered":"2024-09-10T15:58:09.820931+0000","last_clean":"2024-09-10T15:58:09.820931+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:09.820931+0000","last_undegraded":"2024-09-10T15:58:09.820931+0000","last_fullsized":"2024-09-10T15:58:09.820931+0000","mapping_epoch":35,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32481280,"num_objects":7930,"num_object_clones":0,"num_object_copies":15860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9743,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12941","reported_seq":21068,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:09.968635+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:09.968635+0000","last_peered":"2024-09-10T15:58:09.968635+0000","last_clean":"2024-09-10T15:58:09.968635+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:09.968635+0000","last_undegraded":"2024-09-10T15:58:09.968635+0000","last_fullsized":"2024-09-10T15:58:09.968635+0000","mapping_epoch":45,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31858688,"num_objects":7778,"num_object_clones":0,"num_object_copies":15556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10918,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12735","reported_seq":13360,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.830283+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:09.830283+0000","last_peered":"2024-09-10T15:58:09.830283+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:09.830283+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32124928,"num_objects":7843,"num_object_clones":0,"num_object_copies":15686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4229},{"shards":"0,2","objects":1421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12825","reported_seq":13188,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:10.144679+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:58:10.144679+0000","last_peered":"2024-09-10T15:58:10.144679+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:10.144679+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32604160,"num_objects":7960,"num_object_clones":0,"num_object_copies":15920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10799,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1402},{"shards":"2","objects":4247}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12681","reported_seq":12984,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:10.114587+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:10.114587+0000","last_peered":"2024-09-10T15:58:10.114587+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:10.114587+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31846400,"num_objects":7775,"num_object_clones":0,"num_object_copies":15550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3743,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10550,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1378},{"shards":"2","objects":4034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12873","reported_seq":13483,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.869132+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:09.869132+0000","last_peered":"2024-09-10T15:58:09.869132+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:09.869132+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32690176,"num_objects":7981,"num_object_clones":0,"num_object_copies":15962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10065,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4389},{"shards":"0,2","objects":1388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12874","reported_seq":13426,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.858801+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:09.858801+0000","last_peered":"2024-09-10T15:58:09.858801+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:09.858801+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129024,"num_objects":7844,"num_object_clones":0,"num_object_copies":15688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9292,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4121},{"shards":"0,2","objects":1406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12820","reported_seq":13403,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.793449+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:09.793449+0000","last_peered":"2024-09-10T15:58:09.793449+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:09.793449+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32546816,"num_objects":7946,"num_object_clones":0,"num_object_copies":15892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3656,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9426,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4292},{"shards":"0,2","objects":1442}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'13113","reported_seq":13661,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.793780+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:09.793780+0000","last_peered":"2024-09-10T15:58:09.793780+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:09.793780+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32707043,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9932,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4189},{"shards":"0,2","objects":1445}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":515867131,"num_objects":125946,"num_object_clones":0,"num_object_copies":251892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36899,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":125946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167104,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9719,"num_bytes_recovered":32083968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1165123584,"data_stored":1165039567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626493,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628672,"kb_used_data":322160,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93743168,"statfs":{"total":96636764160,"available":95993004032,"internally_reserved":0,"allocated":329891840,"data_stored":329556681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313847470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149776,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819888,"kb_used_data":441184,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93551952,"statfs":{"total":96636764160,"available":95797198848,"internally_reserved":0,"allocated":451772416,"data_stored":451506363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":387772077},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,4,1,0,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687144,"kb_used_data":376472,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93684696,"statfs":{"total":96636764160,"available":95933128704,"internally_reserved":0,"allocated":385507328,"data_stored":385237203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,1,0,2,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":450617344,"data_stored":450613731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329486336,"data_stored":329413617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385019904,"data_stored":385012219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:12.358 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:12.715 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:12.716 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:12.727 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-10T15:58:12.533168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":515867150,"num_objects":125948,"num_object_clones":0,"num_object_copies":251896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36899,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":125948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167106,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9721,"num_bytes_recovered":32083987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160786,"ondisk_log_size":160786,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2135704,"kb_used_data":1139816,"kb_used_omap":12,"kb_used_meta":995827,"kb_avail":280979816,"statfs":{"total":289910292480,"available":287723331584,"internally_reserved":0,"allocated":1167171584,"data_stored":1166300247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1019726853},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,5,3,6,6,2,0,0,3,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18280448,"num_objects":-4463,"num_object_clones":0,"num_object_copies":-8926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-176,"num_objects_misplaced":-332,"num_objects_unfound":0,"num_objects_dirty":-4463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4463,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":508,"num_bytes_recovered":1359872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003366"},"pg_stats":[{"pgid":"2.f","version":"46'12944","reported_seq":13505,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.838043+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:09.838043+0000","last_peered":"2024-09-10T15:58:09.838043+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:09.838043+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32055296,"num_objects":7826,"num_object_clones":0,"num_object_copies":15652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4095},{"shards":"0,2","objects":1414}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'13088","reported_seq":13196,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:10.151164+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:10.151164+0000","last_peered":"2024-09-10T15:58:10.151164+0000","last_clean":"2024-09-10T15:58:10.151164+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:10.151164+0000","last_undegraded":"2024-09-10T15:58:10.151164+0000","last_fullsized":"2024-09-10T15:58:10.151164+0000","mapping_epoch":36,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32710656,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11424,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12751","reported_seq":13090,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:10.132156+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:10.132156+0000","last_peered":"2024-09-10T15:58:10.132156+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:10.132156+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32268312,"num_objects":7879,"num_object_clones":0,"num_object_copies":15758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3696,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7879,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11019,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1366},{"shards":"2","objects":4184}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12851","reported_seq":12931,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:10.157447+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:10.157447+0000","last_peered":"2024-09-10T15:58:10.157447+0000","last_clean":"2024-09-10T15:58:10.157447+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:10.157447+0000","last_undegraded":"2024-09-10T15:58:10.157447+0000","last_fullsized":"2024-09-10T15:58:10.157447+0000","mapping_epoch":36,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32075776,"num_objects":7831,"num_object_clones":0,"num_object_copies":15662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12851,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12610","reported_seq":12737,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:09.818705+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:09.818705+0000","last_peered":"2024-09-10T15:58:09.818705+0000","last_clean":"2024-09-10T15:58:09.818705+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:09.818705+0000","last_undegraded":"2024-09-10T15:58:09.818705+0000","last_fullsized":"2024-09-10T15:58:09.818705+0000","mapping_epoch":35,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31625216,"num_objects":7721,"num_object_clones":0,"num_object_copies":15442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9681,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12873","reported_seq":20691,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:58:09.854159+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:58:09.854159+0000","last_peered":"2024-09-10T15:58:09.854159+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:58:09.854159+0000","last_undegraded":"2024-09-10T15:58:09.854159+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32743424,"num_objects":7994,"num_object_clones":0,"num_object_copies":15988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":7994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9591,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3841,"num_bytes_recovered":15724544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7965},{"shards":"0,2","objects":29}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12620","reported_seq":13187,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.862391+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:09.862391+0000","last_peered":"2024-09-10T15:58:09.862391+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:09.862391+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31399936,"num_objects":7666,"num_object_clones":0,"num_object_copies":15332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3623,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9270,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4045},{"shards":"0,2","objects":1360}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12984","reported_seq":13116,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:09.820931+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:09.820931+0000","last_peered":"2024-09-10T15:58:09.820931+0000","last_clean":"2024-09-10T15:58:09.820931+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:09.820931+0000","last_undegraded":"2024-09-10T15:58:09.820931+0000","last_fullsized":"2024-09-10T15:58:09.820931+0000","mapping_epoch":35,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32481280,"num_objects":7930,"num_object_clones":0,"num_object_copies":15860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9743,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12941","reported_seq":21068,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:09.968635+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:09.968635+0000","last_peered":"2024-09-10T15:58:09.968635+0000","last_clean":"2024-09-10T15:58:09.968635+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:09.968635+0000","last_undegraded":"2024-09-10T15:58:09.968635+0000","last_fullsized":"2024-09-10T15:58:09.968635+0000","mapping_epoch":45,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31858688,"num_objects":7778,"num_object_clones":0,"num_object_copies":15556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10918,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12735","reported_seq":13360,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.830283+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:09.830283+0000","last_peered":"2024-09-10T15:58:09.830283+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:09.830283+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32124928,"num_objects":7843,"num_object_clones":0,"num_object_copies":15686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4229},{"shards":"0,2","objects":1421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12825","reported_seq":13188,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:10.144679+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:58:10.144679+0000","last_peered":"2024-09-10T15:58:10.144679+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:10.144679+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32604160,"num_objects":7960,"num_object_clones":0,"num_object_copies":15920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10799,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1402},{"shards":"2","objects":4247}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12681","reported_seq":12984,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:10.114587+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:10.114587+0000","last_peered":"2024-09-10T15:58:10.114587+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:10.114587+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31846400,"num_objects":7775,"num_object_clones":0,"num_object_copies":15550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3743,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10550,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1378},{"shards":"2","objects":4034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12873","reported_seq":13483,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.869132+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:09.869132+0000","last_peered":"2024-09-10T15:58:09.869132+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:09.869132+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32690176,"num_objects":7981,"num_object_clones":0,"num_object_copies":15962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10065,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4389},{"shards":"0,2","objects":1388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12874","reported_seq":13426,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.858801+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:09.858801+0000","last_peered":"2024-09-10T15:58:09.858801+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:09.858801+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129024,"num_objects":7844,"num_object_clones":0,"num_object_copies":15688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9292,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4121},{"shards":"0,2","objects":1406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12820","reported_seq":13403,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.793449+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:09.793449+0000","last_peered":"2024-09-10T15:58:09.793449+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:09.793449+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32546816,"num_objects":7946,"num_object_clones":0,"num_object_copies":15892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3656,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9426,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4292},{"shards":"0,2","objects":1442}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'13113","reported_seq":13661,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.793780+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:09.793780+0000","last_peered":"2024-09-10T15:58:09.793780+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:09.793780+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32707043,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9932,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4189},{"shards":"0,2","objects":1445}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":515867131,"num_objects":125946,"num_object_clones":0,"num_object_copies":251892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36899,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":125946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167104,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9719,"num_bytes_recovered":32083968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1165123584,"data_stored":1165039567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626493,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628672,"kb_used_data":322160,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93743168,"statfs":{"total":96636764160,"available":95993004032,"internally_reserved":0,"allocated":329891840,"data_stored":329556681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313847470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149776,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819888,"kb_used_data":441184,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93551952,"statfs":{"total":96636764160,"available":95797198848,"internally_reserved":0,"allocated":451772416,"data_stored":451506363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":387772077},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,4,1,0,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687144,"kb_used_data":376472,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93684696,"statfs":{"total":96636764160,"available":95933128704,"internally_reserved":0,"allocated":385507328,"data_stored":385237203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,1,0,2,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":450617344,"data_stored":450613731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329486336,"data_stored":329413617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385019904,"data_stored":385012219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:12.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:58:12.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'12944", 'reported_seq': 13505, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:09.838043+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:58:09.838043+0000', 'last_peered': '2024-09-10T15:58:09.838043+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:58:09.838043+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'2900", 'ondisk_log_start': "37'2900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32055296, 'num_objects': 7826, 'num_object_clones': 0, 'num_object_copies': 15652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3731, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12944, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 242, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4095}, {'shards': '0,2', 'objects': 1414}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:12.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:58:12.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "46'12751", 'reported_seq': 13090, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:10.132156+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:58:10.132156+0000', 'last_peered': '2024-09-10T15:58:10.132156+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:58:10.132156+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'2700", 'ondisk_log_start': "37'2700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32268312, 'num_objects': 7879, 'num_object_clones': 0, 'num_object_copies': 15758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3696, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7879, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11019, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 107, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1366}, {'shards': '2', 'objects': 4184}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:12.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-10T15:58:12.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'12873", 'reported_seq': 20691, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-10T15:58:09.854159+0000', 'last_change': '2024-09-10T15:55:19.440231+0000', 'last_active': '2024-09-10T15:58:09.854159+0000', 'last_peered': '2024-09-10T15:58:09.854159+0000', 'last_clean': '2024-09-10T15:53:17.303422+0000', 'last_became_active': '2024-09-10T15:53:19.892346+0000', 'last_became_peered': '2024-09-10T15:53:19.892346+0000', 'last_unstale': '2024-09-10T15:58:09.854159+0000', 'last_undegraded': '2024-09-10T15:58:09.854159+0000', 'last_fullsized': '2024-09-10T15:53:19.468212+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3327", 'last_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:07.938168+0000', 'objects_scrubbed': 3277, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1368, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32743424, 'num_objects': 7994, 'num_object_clones': 0, 'num_object_copies': 15988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 29, 'num_objects_unfound': 0, 'num_objects_dirty': 7994, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9591, 'num_write_kb': 28616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3841, 'num_bytes_recovered': 15724544, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7965}, {'shards': '0,2', 'objects': 29}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:12.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:58:12.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'12620", 'reported_seq': 13187, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:09.862391+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:58:09.862391+0000', 'last_peered': '2024-09-10T15:58:09.862391+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:58:09.862391+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31399936, 'num_objects': 7666, 'num_object_clones': 0, 'num_object_copies': 15332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3623, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7666, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9270, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 214, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4045}, {'shards': '0,2', 'objects': 1360}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:12.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:58:12.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'12735", 'reported_seq': 13360, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:09.830283+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:58:09.830283+0000', 'last_peered': '2024-09-10T15:58:09.830283+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:58:09.830283+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'2700", 'ondisk_log_start': "37'2700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32124928, 'num_objects': 7843, 'num_object_clones': 0, 'num_object_copies': 15686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9599, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 249, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4229}, {'shards': '0,2', 'objects': 1421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:12.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:58:12.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'12825", 'reported_seq': 13188, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:10.144679+0000', 'last_change': '2024-09-10T15:53:19.935547+0000', 'last_active': '2024-09-10T15:58:10.144679+0000', 'last_peered': '2024-09-10T15:58:10.144679+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:58:10.144679+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32604160, 'num_objects': 7960, 'num_object_clones': 0, 'num_object_copies': 15920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3715, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7960, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10799, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 119, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1402}, {'shards': '2', 'objects': 4247}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:12.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:58:12.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'12681", 'reported_seq': 12984, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:10.114587+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:58:10.114587+0000', 'last_peered': '2024-09-10T15:58:10.114587+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:58:10.114587+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31846400, 'num_objects': 7775, 'num_object_clones': 0, 'num_object_copies': 15550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3743, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7775, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10550, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 88, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1378}, {'shards': '2', 'objects': 4034}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:12.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:58:12.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'12873", 'reported_seq': 13483, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:09.869132+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:58:09.869132+0000', 'last_peered': '2024-09-10T15:58:09.869132+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:58:09.869132+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32690176, 'num_objects': 7981, 'num_object_clones': 0, 'num_object_copies': 15962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3593, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7981, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10065, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 241, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4389}, {'shards': '0,2', 'objects': 1388}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:12.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:58:12.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'12874", 'reported_seq': 13426, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:09.858801+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:58:09.858801+0000', 'last_peered': '2024-09-10T15:58:09.858801+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:58:09.858801+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32129024, 'num_objects': 7844, 'num_object_clones': 0, 'num_object_copies': 15688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3726, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7844, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9292, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 194, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4121}, {'shards': '0,2', 'objects': 1406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:12.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:58:12.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'12820", 'reported_seq': 13403, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:09.793449+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:58:09.793449+0000', 'last_peered': '2024-09-10T15:58:09.793449+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:58:09.793449+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32546816, 'num_objects': 7946, 'num_object_clones': 0, 'num_object_copies': 15892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3656, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7946, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9426, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 224, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4292}, {'shards': '0,2', 'objects': 1442}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:12.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:58:12.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'13113", 'reported_seq': 13661, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:09.793780+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:58:09.793780+0000', 'last_peered': '2024-09-10T15:58:09.793780+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:58:09.793780+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'3100", 'ondisk_log_start': "37'3100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32707043, 'num_objects': 7986, 'num_object_clones': 0, 'num_object_copies': 15972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3800, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7986, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9932, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 206, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4189}, {'shards': '0,2', 'objects': 1445}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:12.731 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:13.351 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:13.351 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36899/251896 objects degraded (14.649%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":419,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":125948,"data_bytes":515867150,"bytes_used":2186960896,"bytes_avail":287723331584,"bytes_total":289910292480,"degraded_objects":36899,"degraded_total":251896,"degraded_ratio":0.14648505732524533,"misplaced_objects":29,"misplaced_total":251896,"misplaced_ratio":0.00011512687775907518,"recovering_objects_per_sec":50,"recovering_bytes_per_sec":135941,"recovering_keys_per_sec":0,"num_objects_recovered":508,"num_bytes_recovered":1359872,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":446},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:58:13.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:58:13.373 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:13.952 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:13.953 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:13.979 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-10T15:58:12.743275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":515867150,"num_objects":125948,"num_object_clones":0,"num_object_copies":251896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36899,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":125948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167106,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9721,"num_bytes_recovered":32083987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160786,"ondisk_log_size":160786,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2135704,"kb_used_data":1139816,"kb_used_omap":12,"kb_used_meta":995827,"kb_avail":280979816,"statfs":{"total":289910292480,"available":287723331584,"internally_reserved":0,"allocated":1167171584,"data_stored":1166300247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1019726853},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,5,3,6,6,2,0,0,3,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18280448,"num_objects":-4463,"num_object_clones":0,"num_object_copies":-8926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-176,"num_objects_misplaced":-332,"num_objects_unfound":0,"num_objects_dirty":-4463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4463,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":508,"num_bytes_recovered":1359872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.212423"},"pg_stats":[{"pgid":"2.f","version":"46'12944","reported_seq":13505,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.838043+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:09.838043+0000","last_peered":"2024-09-10T15:58:09.838043+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:09.838043+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32055296,"num_objects":7826,"num_object_clones":0,"num_object_copies":15652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4095},{"shards":"0,2","objects":1414}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'13088","reported_seq":13196,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:10.151164+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:10.151164+0000","last_peered":"2024-09-10T15:58:10.151164+0000","last_clean":"2024-09-10T15:58:10.151164+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:10.151164+0000","last_undegraded":"2024-09-10T15:58:10.151164+0000","last_fullsized":"2024-09-10T15:58:10.151164+0000","mapping_epoch":36,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32710656,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11424,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'12751","reported_seq":13090,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:10.132156+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:10.132156+0000","last_peered":"2024-09-10T15:58:10.132156+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:10.132156+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32268312,"num_objects":7879,"num_object_clones":0,"num_object_copies":15758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3696,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7879,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11019,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1366},{"shards":"2","objects":4184}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'12851","reported_seq":12931,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:10.157447+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:10.157447+0000","last_peered":"2024-09-10T15:58:10.157447+0000","last_clean":"2024-09-10T15:58:10.157447+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:10.157447+0000","last_undegraded":"2024-09-10T15:58:10.157447+0000","last_fullsized":"2024-09-10T15:58:10.157447+0000","mapping_epoch":36,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32075776,"num_objects":7831,"num_object_clones":0,"num_object_copies":15662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12851,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'12610","reported_seq":12737,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:09.818705+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:09.818705+0000","last_peered":"2024-09-10T15:58:09.818705+0000","last_clean":"2024-09-10T15:58:09.818705+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:09.818705+0000","last_undegraded":"2024-09-10T15:58:09.818705+0000","last_fullsized":"2024-09-10T15:58:09.818705+0000","mapping_epoch":35,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31625216,"num_objects":7721,"num_object_clones":0,"num_object_copies":15442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9681,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'12873","reported_seq":20691,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-10T15:58:09.854159+0000","last_change":"2024-09-10T15:55:19.440231+0000","last_active":"2024-09-10T15:58:09.854159+0000","last_peered":"2024-09-10T15:58:09.854159+0000","last_clean":"2024-09-10T15:53:17.303422+0000","last_became_active":"2024-09-10T15:53:19.892346+0000","last_became_peered":"2024-09-10T15:53:19.892346+0000","last_unstale":"2024-09-10T15:58:09.854159+0000","last_undegraded":"2024-09-10T15:58:09.854159+0000","last_fullsized":"2024-09-10T15:53:19.468212+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32743424,"num_objects":7994,"num_object_clones":0,"num_object_copies":15988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":7994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9591,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3841,"num_bytes_recovered":15724544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7965},{"shards":"0,2","objects":29}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'12620","reported_seq":13187,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.862391+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:09.862391+0000","last_peered":"2024-09-10T15:58:09.862391+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:09.862391+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31399936,"num_objects":7666,"num_object_clones":0,"num_object_copies":15332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3623,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9270,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4045},{"shards":"0,2","objects":1360}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'12984","reported_seq":13116,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:09.820931+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:09.820931+0000","last_peered":"2024-09-10T15:58:09.820931+0000","last_clean":"2024-09-10T15:58:09.820931+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:09.820931+0000","last_undegraded":"2024-09-10T15:58:09.820931+0000","last_fullsized":"2024-09-10T15:58:09.820931+0000","mapping_epoch":35,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32481280,"num_objects":7930,"num_object_clones":0,"num_object_copies":15860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9743,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671986+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:55:20.671986+0000","last_peered":"2024-09-10T15:55:20.671986+0000","last_clean":"2024-09-10T15:55:20.671986+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:55:20.671986+0000","last_undegraded":"2024-09-10T15:55:20.671986+0000","last_fullsized":"2024-09-10T15:55:20.671986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'12941","reported_seq":21068,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:58:09.968635+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:09.968635+0000","last_peered":"2024-09-10T15:58:09.968635+0000","last_clean":"2024-09-10T15:58:09.968635+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:09.968635+0000","last_undegraded":"2024-09-10T15:58:09.968635+0000","last_fullsized":"2024-09-10T15:58:09.968635+0000","mapping_epoch":45,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31858688,"num_objects":7778,"num_object_clones":0,"num_object_copies":15556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10918,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680016+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:55:20.680016+0000","last_peered":"2024-09-10T15:55:20.680016+0000","last_clean":"2024-09-10T15:55:20.680016+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:55:20.680016+0000","last_undegraded":"2024-09-10T15:55:20.680016+0000","last_fullsized":"2024-09-10T15:55:20.680016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'12735","reported_seq":13360,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.830283+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:09.830283+0000","last_peered":"2024-09-10T15:58:09.830283+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:09.830283+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32124928,"num_objects":7843,"num_object_clones":0,"num_object_copies":15686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4229},{"shards":"0,2","objects":1421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.671807+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:55:20.671807+0000","last_peered":"2024-09-10T15:55:20.671807+0000","last_clean":"2024-09-10T15:55:20.671807+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:55:20.671807+0000","last_undegraded":"2024-09-10T15:55:20.671807+0000","last_fullsized":"2024-09-10T15:55:20.671807+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'12825","reported_seq":13188,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:10.144679+0000","last_change":"2024-09-10T15:53:19.935547+0000","last_active":"2024-09-10T15:58:10.144679+0000","last_peered":"2024-09-10T15:58:10.144679+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:10.144679+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32604160,"num_objects":7960,"num_object_clones":0,"num_object_copies":15920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10799,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1402},{"shards":"2","objects":4247}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635422+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:55:23.635422+0000","last_peered":"2024-09-10T15:55:23.635422+0000","last_clean":"2024-09-10T15:55:23.635422+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:55:23.635422+0000","last_undegraded":"2024-09-10T15:55:23.635422+0000","last_fullsized":"2024-09-10T15:55:23.635422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'12681","reported_seq":12984,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:10.114587+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:10.114587+0000","last_peered":"2024-09-10T15:58:10.114587+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:10.114587+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31846400,"num_objects":7775,"num_object_clones":0,"num_object_copies":15550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3743,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10550,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1378},{"shards":"2","objects":4034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'12873","reported_seq":13483,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.869132+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:09.869132+0000","last_peered":"2024-09-10T15:58:09.869132+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:09.869132+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32690176,"num_objects":7981,"num_object_clones":0,"num_object_copies":15962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10065,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4389},{"shards":"0,2","objects":1388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635453+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:55:23.635453+0000","last_peered":"2024-09-10T15:55:23.635453+0000","last_clean":"2024-09-10T15:55:23.635453+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:55:23.635453+0000","last_undegraded":"2024-09-10T15:55:23.635453+0000","last_fullsized":"2024-09-10T15:55:23.635453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'12874","reported_seq":13426,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.858801+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:09.858801+0000","last_peered":"2024-09-10T15:58:09.858801+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:09.858801+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129024,"num_objects":7844,"num_object_clones":0,"num_object_copies":15688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9292,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4121},{"shards":"0,2","objects":1406}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":94,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:20.680140+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:55:20.680140+0000","last_peered":"2024-09-10T15:55:20.680140+0000","last_clean":"2024-09-10T15:55:20.680140+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:55:20.680140+0000","last_undegraded":"2024-09-10T15:55:20.680140+0000","last_fullsized":"2024-09-10T15:55:20.680140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'12820","reported_seq":13403,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.793449+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:09.793449+0000","last_peered":"2024-09-10T15:58:09.793449+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:09.793449+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32546816,"num_objects":7946,"num_object_clones":0,"num_object_copies":15892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3656,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9426,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4292},{"shards":"0,2","objects":1442}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635463+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:55:23.635463+0000","last_peered":"2024-09-10T15:55:23.635463+0000","last_clean":"2024-09-10T15:55:23.635463+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:55:23.635463+0000","last_undegraded":"2024-09-10T15:55:23.635463+0000","last_fullsized":"2024-09-10T15:55:23.635463+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T15:55:23.635473+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:55:23.635473+0000","last_peered":"2024-09-10T15:55:23.635473+0000","last_clean":"2024-09-10T15:55:23.635473+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:55:23.635473+0000","last_undegraded":"2024-09-10T15:55:23.635473+0000","last_fullsized":"2024-09-10T15:55:23.635473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'13113","reported_seq":13661,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:09.793780+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:09.793780+0000","last_peered":"2024-09-10T15:58:09.793780+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:09.793780+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32707043,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3800,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9932,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4189},{"shards":"0,2","objects":1445}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":515867131,"num_objects":125946,"num_object_clones":0,"num_object_copies":251892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36899,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":125946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167104,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9719,"num_bytes_recovered":32083968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1165123584,"data_stored":1165039567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626493,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628672,"kb_used_data":322160,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93743168,"statfs":{"total":96636764160,"available":95993004032,"internally_reserved":0,"allocated":329891840,"data_stored":329556681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313847470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149776,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819888,"kb_used_data":441184,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93551952,"statfs":{"total":96636764160,"available":95797198848,"internally_reserved":0,"allocated":451772416,"data_stored":451506363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":387772077},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,4,1,0,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687144,"kb_used_data":376472,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93684696,"statfs":{"total":96636764160,"available":95933128704,"internally_reserved":0,"allocated":385507328,"data_stored":385237203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,1,0,2,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":450617344,"data_stored":450613731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":329486336,"data_stored":329413617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385019904,"data_stored":385012219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:16.982 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:17.519 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:17.522 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:17.540 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-10T15:58:16.534278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":507429390,"num_objects":123888,"num_object_clones":0,"num_object_copies":247776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35480,"num_objects_misplaced":1216,"num_objects_unfound":0,"num_objects_dirty":123888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169166,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9954,"num_bytes_recovered":32792595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2138924,"kb_used_data":1129132,"kb_used_omap":12,"kb_used_meta":1009715,"kb_avail":280976596,"statfs":{"total":289910292480,"available":287720034304,"internally_reserved":0,"allocated":1156231168,"data_stored":1155335722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1033948165},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,1,1,2,18,1,3,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17747968,"num_objects":-4333,"num_object_clones":0,"num_object_copies":-8666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1515,"num_objects_misplaced":1006,"num_objects_unfound":0,"num_objects_dirty":-4333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4333,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":510,"num_bytes_recovered":1449984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002050"},"pg_stats":[{"pgid":"2.f","version":"48'13068","reported_seq":13649,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.863261+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:14.863261+0000","last_peered":"2024-09-10T15:58:14.863261+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:14.863261+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31547392,"num_objects":7702,"num_object_clones":0,"num_object_copies":15404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3723,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13068,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3979},{"shards":"0,2","objects":1290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13206","reported_seq":13318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:15.094243+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:15.094243+0000","last_peered":"2024-09-10T15:58:15.094243+0000","last_clean":"2024-09-10T15:58:15.094243+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:15.094243+0000","last_undegraded":"2024-09-10T15:58:15.094243+0000","last_fullsized":"2024-09-10T15:58:15.094243+0000","mapping_epoch":36,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32227328,"num_objects":7868,"num_object_clones":0,"num_object_copies":15736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11542,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'12900","reported_seq":13256,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:15.076402+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:15.076402+0000","last_peered":"2024-09-10T15:58:15.076402+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:15.076402+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31658008,"num_objects":7730,"num_object_clones":0,"num_object_copies":15460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7730,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11168,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1217},{"shards":"2","objects":4042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'12975","reported_seq":13059,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:15.097884+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:15.097884+0000","last_peered":"2024-09-10T15:58:15.097884+0000","last_clean":"2024-09-10T15:58:15.097884+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:15.097884+0000","last_undegraded":"2024-09-10T15:58:15.097884+0000","last_fullsized":"2024-09-10T15:58:15.097884+0000","mapping_epoch":36,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31567872,"num_objects":7707,"num_object_clones":0,"num_object_copies":15414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12975,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'12735","reported_seq":12866,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:14.897725+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:14.897725+0000","last_peered":"2024-09-10T15:58:14.897725+0000","last_clean":"2024-09-10T15:58:14.897725+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:14.897725+0000","last_undegraded":"2024-09-10T15:58:14.897725+0000","last_fullsized":"2024-09-10T15:58:14.897725+0000","mapping_epoch":35,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31113216,"num_objects":7596,"num_object_clones":0,"num_object_copies":15192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9806,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13001","reported_seq":20892,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:14.941796+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:14.941796+0000","last_peered":"2024-09-10T15:58:14.941796+0000","last_clean":"2024-09-10T15:58:14.941796+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:14.941796+0000","last_undegraded":"2024-09-10T15:58:14.941796+0000","last_fullsized":"2024-09-10T15:58:14.941796+0000","mapping_epoch":47,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32219136,"num_objects":7866,"num_object_clones":0,"num_object_copies":15732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9719,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'12754","reported_seq":13332,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.838178+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:14.838178+0000","last_peered":"2024-09-10T15:58:14.838178+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:14.838178+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30851072,"num_objects":7532,"num_object_clones":0,"num_object_copies":15064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9404,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3915},{"shards":"0,2","objects":1226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13105","reported_seq":13241,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:14.869810+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:14.869810+0000","last_peered":"2024-09-10T15:58:14.869810+0000","last_clean":"2024-09-10T15:58:14.869810+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:14.869810+0000","last_undegraded":"2024-09-10T15:58:14.869810+0000","last_fullsized":"2024-09-10T15:58:14.869810+0000","mapping_epoch":35,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31985664,"num_objects":7809,"num_object_clones":0,"num_object_copies":15618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9864,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13070","reported_seq":21202,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:15.031080+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:15.031080+0000","last_peered":"2024-09-10T15:58:15.031080+0000","last_clean":"2024-09-10T15:58:15.031080+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:15.031080+0000","last_undegraded":"2024-09-10T15:58:15.031080+0000","last_fullsized":"2024-09-10T15:58:15.031080+0000","mapping_epoch":45,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31330304,"num_objects":7649,"num_object_clones":0,"num_object_copies":15298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11047,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'12869","reported_seq":13516,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.904039+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:14.904039+0000","last_peered":"2024-09-10T15:58:14.904039+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:14.904039+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9733,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4104},{"shards":"0,2","objects":1287}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753134+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753134+0000","last_peered":"2024-09-10T15:58:12.753134+0000","last_clean":"2024-09-10T15:58:12.753134+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753134+0000","last_undegraded":"2024-09-10T15:58:12.753134+0000","last_fullsized":"2024-09-10T15:58:12.753134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'12950","reported_seq":13612,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:15.181379+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:15.181379+0000","last_peered":"2024-09-10T15:58:15.181379+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:15.181379+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32092160,"num_objects":7835,"num_object_clones":0,"num_object_copies":15670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2352,"num_objects_misplaced":1216,"num_objects_unfound":0,"num_objects_dirty":7835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10924,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1216},{"shards":"2","objects":4267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'12820","reported_seq":13133,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:15.123058+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:15.123058+0000","last_peered":"2024-09-10T15:58:15.123058+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:15.123058+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31277056,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10689,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1239},{"shards":"2","objects":3898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13011","reported_seq":13639,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.894630+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:14.894630+0000","last_peered":"2024-09-10T15:58:14.894630+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:14.894630+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32124928,"num_objects":7843,"num_object_clones":0,"num_object_copies":15686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10203,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4258},{"shards":"0,2","objects":1250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'12992","reported_seq":13562,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.845823+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:14.845823+0000","last_peered":"2024-09-10T15:58:14.845823+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:14.845823+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31645696,"num_objects":7726,"num_object_clones":0,"num_object_copies":15452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3719,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9410,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4010},{"shards":"0,2","objects":1288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'12937","reported_seq":13537,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.764460+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:14.764460+0000","last_peered":"2024-09-10T15:58:14.764460+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:14.764460+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32067584,"num_objects":7829,"num_object_clones":0,"num_object_copies":15658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9543,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4180},{"shards":"0,2","objects":1325}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13250","reported_seq":13814,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.880481+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:14.880481+0000","last_peered":"2024-09-10T15:58:14.880481+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:14.880481+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32145891,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10069,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4058},{"shards":"0,2","objects":1308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":507429371,"num_objects":123886,"num_object_clones":0,"num_object_copies":247772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35480,"num_objects_misplaced":1216,"num_objects_unfound":0,"num_objects_dirty":123886,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169164,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9952,"num_bytes_recovered":32792576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1154064384,"data_stored":1153987593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160743,"ondisk_log_size":160743,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626494,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632172,"kb_used_data":321496,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93739668,"statfs":{"total":96636764160,"available":95989420032,"internally_reserved":0,"allocated":329211904,"data_stored":328873526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318107310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149777,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":836208,"kb_used_data":435292,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93535632,"statfs":{"total":96636764160,"available":95780487168,"internally_reserved":0,"allocated":445739008,"data_stored":445462510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":410513069},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,0,16,1,1,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670544,"kb_used_data":372344,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93701296,"statfs":{"total":96636764160,"available":95950127104,"internally_reserved":0,"allocated":381280256,"data_stored":380999686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444387328,"data_stored":444383715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328802304,"data_stored":328736811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380874752,"data_stored":380867067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:17.541 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:18.095 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:18.096 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:18.127 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-10T15:58:16.534278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":507429390,"num_objects":123888,"num_object_clones":0,"num_object_copies":247776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35480,"num_objects_misplaced":1216,"num_objects_unfound":0,"num_objects_dirty":123888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169166,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9954,"num_bytes_recovered":32792595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2138924,"kb_used_data":1129132,"kb_used_omap":12,"kb_used_meta":1009715,"kb_avail":280976596,"statfs":{"total":289910292480,"available":287720034304,"internally_reserved":0,"allocated":1156231168,"data_stored":1155335722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1033948165},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,1,1,2,18,1,3,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17747968,"num_objects":-4333,"num_object_clones":0,"num_object_copies":-8666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1515,"num_objects_misplaced":1006,"num_objects_unfound":0,"num_objects_dirty":-4333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4333,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":510,"num_bytes_recovered":1449984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002050"},"pg_stats":[{"pgid":"2.f","version":"48'13068","reported_seq":13649,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.863261+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:14.863261+0000","last_peered":"2024-09-10T15:58:14.863261+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:14.863261+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31547392,"num_objects":7702,"num_object_clones":0,"num_object_copies":15404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3723,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13068,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3979},{"shards":"0,2","objects":1290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13206","reported_seq":13318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:15.094243+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:15.094243+0000","last_peered":"2024-09-10T15:58:15.094243+0000","last_clean":"2024-09-10T15:58:15.094243+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:15.094243+0000","last_undegraded":"2024-09-10T15:58:15.094243+0000","last_fullsized":"2024-09-10T15:58:15.094243+0000","mapping_epoch":36,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32227328,"num_objects":7868,"num_object_clones":0,"num_object_copies":15736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11542,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'12900","reported_seq":13256,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:15.076402+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:15.076402+0000","last_peered":"2024-09-10T15:58:15.076402+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:15.076402+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31658008,"num_objects":7730,"num_object_clones":0,"num_object_copies":15460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7730,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11168,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1217},{"shards":"2","objects":4042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'12975","reported_seq":13059,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:15.097884+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:15.097884+0000","last_peered":"2024-09-10T15:58:15.097884+0000","last_clean":"2024-09-10T15:58:15.097884+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:15.097884+0000","last_undegraded":"2024-09-10T15:58:15.097884+0000","last_fullsized":"2024-09-10T15:58:15.097884+0000","mapping_epoch":36,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31567872,"num_objects":7707,"num_object_clones":0,"num_object_copies":15414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12975,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'12735","reported_seq":12866,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:14.897725+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:14.897725+0000","last_peered":"2024-09-10T15:58:14.897725+0000","last_clean":"2024-09-10T15:58:14.897725+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:14.897725+0000","last_undegraded":"2024-09-10T15:58:14.897725+0000","last_fullsized":"2024-09-10T15:58:14.897725+0000","mapping_epoch":35,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31113216,"num_objects":7596,"num_object_clones":0,"num_object_copies":15192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9806,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13001","reported_seq":20892,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:14.941796+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:14.941796+0000","last_peered":"2024-09-10T15:58:14.941796+0000","last_clean":"2024-09-10T15:58:14.941796+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:14.941796+0000","last_undegraded":"2024-09-10T15:58:14.941796+0000","last_fullsized":"2024-09-10T15:58:14.941796+0000","mapping_epoch":47,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32219136,"num_objects":7866,"num_object_clones":0,"num_object_copies":15732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9719,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'12754","reported_seq":13332,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.838178+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:14.838178+0000","last_peered":"2024-09-10T15:58:14.838178+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:14.838178+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30851072,"num_objects":7532,"num_object_clones":0,"num_object_copies":15064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9404,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3915},{"shards":"0,2","objects":1226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13105","reported_seq":13241,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:14.869810+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:14.869810+0000","last_peered":"2024-09-10T15:58:14.869810+0000","last_clean":"2024-09-10T15:58:14.869810+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:14.869810+0000","last_undegraded":"2024-09-10T15:58:14.869810+0000","last_fullsized":"2024-09-10T15:58:14.869810+0000","mapping_epoch":35,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31985664,"num_objects":7809,"num_object_clones":0,"num_object_copies":15618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9864,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13070","reported_seq":21202,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:15.031080+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:15.031080+0000","last_peered":"2024-09-10T15:58:15.031080+0000","last_clean":"2024-09-10T15:58:15.031080+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:15.031080+0000","last_undegraded":"2024-09-10T15:58:15.031080+0000","last_fullsized":"2024-09-10T15:58:15.031080+0000","mapping_epoch":45,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31330304,"num_objects":7649,"num_object_clones":0,"num_object_copies":15298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11047,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'12869","reported_seq":13516,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.904039+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:14.904039+0000","last_peered":"2024-09-10T15:58:14.904039+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:14.904039+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9733,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4104},{"shards":"0,2","objects":1287}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753134+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753134+0000","last_peered":"2024-09-10T15:58:12.753134+0000","last_clean":"2024-09-10T15:58:12.753134+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753134+0000","last_undegraded":"2024-09-10T15:58:12.753134+0000","last_fullsized":"2024-09-10T15:58:12.753134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'12950","reported_seq":13612,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:15.181379+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:15.181379+0000","last_peered":"2024-09-10T15:58:15.181379+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:15.181379+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32092160,"num_objects":7835,"num_object_clones":0,"num_object_copies":15670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2352,"num_objects_misplaced":1216,"num_objects_unfound":0,"num_objects_dirty":7835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10924,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1216},{"shards":"2","objects":4267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'12820","reported_seq":13133,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:15.123058+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:15.123058+0000","last_peered":"2024-09-10T15:58:15.123058+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:15.123058+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31277056,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10689,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1239},{"shards":"2","objects":3898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13011","reported_seq":13639,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.894630+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:14.894630+0000","last_peered":"2024-09-10T15:58:14.894630+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:14.894630+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32124928,"num_objects":7843,"num_object_clones":0,"num_object_copies":15686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10203,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4258},{"shards":"0,2","objects":1250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'12992","reported_seq":13562,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.845823+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:14.845823+0000","last_peered":"2024-09-10T15:58:14.845823+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:14.845823+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31645696,"num_objects":7726,"num_object_clones":0,"num_object_copies":15452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3719,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9410,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4010},{"shards":"0,2","objects":1288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'12937","reported_seq":13537,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.764460+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:14.764460+0000","last_peered":"2024-09-10T15:58:14.764460+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:14.764460+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32067584,"num_objects":7829,"num_object_clones":0,"num_object_copies":15658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9543,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4180},{"shards":"0,2","objects":1325}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13250","reported_seq":13814,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.880481+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:14.880481+0000","last_peered":"2024-09-10T15:58:14.880481+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:14.880481+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32145891,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10069,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4058},{"shards":"0,2","objects":1308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":507429371,"num_objects":123886,"num_object_clones":0,"num_object_copies":247772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35480,"num_objects_misplaced":1216,"num_objects_unfound":0,"num_objects_dirty":123886,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169164,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9952,"num_bytes_recovered":32792576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1154064384,"data_stored":1153987593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160743,"ondisk_log_size":160743,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626494,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632172,"kb_used_data":321496,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93739668,"statfs":{"total":96636764160,"available":95989420032,"internally_reserved":0,"allocated":329211904,"data_stored":328873526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318107310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149777,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":836208,"kb_used_data":435292,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93535632,"statfs":{"total":96636764160,"available":95780487168,"internally_reserved":0,"allocated":445739008,"data_stored":445462510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":410513069},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,0,16,1,1,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670544,"kb_used_data":372344,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93701296,"statfs":{"total":96636764160,"available":95950127104,"internally_reserved":0,"allocated":381280256,"data_stored":380999686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444387328,"data_stored":444383715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328802304,"data_stored":328736811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380874752,"data_stored":380867067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:18.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:58:18.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'13068", 'reported_seq': 13649, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:14.863261+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:58:14.863261+0000', 'last_peered': '2024-09-10T15:58:14.863261+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:58:14.863261+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'3000", 'ondisk_log_start': "37'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31547392, 'num_objects': 7702, 'num_object_clones': 0, 'num_object_copies': 15404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3723, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7702, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13068, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 250, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3979}, {'shards': '0,2', 'objects': 1290}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:18.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:58:18.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'12900", 'reported_seq': 13256, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:15.076402+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:58:15.076402+0000', 'last_peered': '2024-09-10T15:58:15.076402+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:58:15.076402+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31658008, 'num_objects': 7730, 'num_object_clones': 0, 'num_object_copies': 15460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3689, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7730, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11168, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 114, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1217}, {'shards': '2', 'objects': 4042}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:18.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:58:18.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'12754", 'reported_seq': 13332, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:14.838178+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:58:14.838178+0000', 'last_peered': '2024-09-10T15:58:14.838178+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:58:14.838178+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'2700", 'ondisk_log_start': "37'2700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30851072, 'num_objects': 7532, 'num_object_clones': 0, 'num_object_copies': 15064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3619, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7532, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9404, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 218, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3915}, {'shards': '0,2', 'objects': 1226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:18.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:58:18.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'12869", 'reported_seq': 13516, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:14.904039+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:58:14.904039+0000', 'last_peered': '2024-09-10T15:58:14.904039+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:58:14.904039+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31576064, 'num_objects': 7709, 'num_object_clones': 0, 'num_object_copies': 15418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3607, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7709, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9733, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 258, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4104}, {'shards': '0,2', 'objects': 1287}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:18.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:58:18.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'12950", 'reported_seq': 13612, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:15.181379+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:58:15.181379+0000', 'last_peered': '2024-09-10T15:58:15.181379+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:58:15.181379+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'2900", 'ondisk_log_start': "37'2900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32092160, 'num_objects': 7835, 'num_object_clones': 0, 'num_object_copies': 15670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2352, 'num_objects_misplaced': 1216, 'num_objects_unfound': 0, 'num_objects_dirty': 7835, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10924, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 266, 'num_bytes_recovered': 589824, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1216}, {'shards': '2', 'objects': 4267}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:18.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:58:18.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'12820", 'reported_seq': 13133, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:15.123058+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:58:15.123058+0000', 'last_peered': '2024-09-10T15:58:15.123058+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:58:15.123058+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31277056, 'num_objects': 7636, 'num_object_clones': 0, 'num_object_copies': 15272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3740, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7636, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10689, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 91, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1239}, {'shards': '2', 'objects': 3898}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:18.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:58:18.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'13011", 'reported_seq': 13639, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:14.894630+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:58:14.894630+0000', 'last_peered': '2024-09-10T15:58:14.894630+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:58:14.894630+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'3000", 'ondisk_log_start': "37'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32124928, 'num_objects': 7843, 'num_object_clones': 0, 'num_object_copies': 15686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3586, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10203, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 248, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4258}, {'shards': '0,2', 'objects': 1250}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:18.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:58:18.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'12992", 'reported_seq': 13562, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:14.845823+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:58:14.845823+0000', 'last_peered': '2024-09-10T15:58:14.845823+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:58:14.845823+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'2900", 'ondisk_log_start': "37'2900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31645696, 'num_objects': 7726, 'num_object_clones': 0, 'num_object_copies': 15452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3719, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9410, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 201, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4010}, {'shards': '0,2', 'objects': 1288}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:18.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:58:18.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'12937", 'reported_seq': 13537, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:14.764460+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:58:14.764460+0000', 'last_peered': '2024-09-10T15:58:14.764460+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:58:14.764460+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'2900", 'ondisk_log_start': "37'2900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32067584, 'num_objects': 7829, 'num_object_clones': 0, 'num_object_copies': 15658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3651, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7829, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9543, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 229, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4180}, {'shards': '0,2', 'objects': 1325}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:18.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:58:18.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'13250", 'reported_seq': 13814, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:14.880481+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:58:14.880481+0000', 'last_peered': '2024-09-10T15:58:14.880481+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:58:14.880481+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'3200", 'ondisk_log_start': "37'3200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32145891, 'num_objects': 7849, 'num_object_clones': 0, 'num_object_copies': 15698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3794, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7849, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10069, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 212, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4058}, {'shards': '0,2', 'objects': 1308}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:18.130 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:18.789 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:18.789 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35480/247776 objects degraded (14.319%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":424,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":123888,"data_bytes":507429390,"bytes_used":2190258176,"bytes_avail":287720034304,"bytes_total":289910292480,"degraded_objects":35480,"degraded_total":247776,"degraded_ratio":0.14319385251194627,"misplaced_objects":1216,"misplaced_total":247776,"misplaced_ratio":0.0049076585302854194,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":88552,"recovering_keys_per_sec":0,"num_objects_recovered":233,"num_bytes_recovered":708608,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":257},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:58:18.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:58:18.812 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:19.336 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:19.339 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:19.363 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-10T15:58:18.534795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":507429390,"num_objects":123888,"num_object_clones":0,"num_object_copies":247776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35480,"num_objects_misplaced":1216,"num_objects_unfound":0,"num_objects_dirty":123888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169166,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9954,"num_bytes_recovered":32792595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2138924,"kb_used_data":1129132,"kb_used_omap":12,"kb_used_meta":1009715,"kb_avail":280976596,"statfs":{"total":289910292480,"available":287720034304,"internally_reserved":0,"allocated":1156231168,"data_stored":1155335722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1033948165},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,1,1,2,18,1,3,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8437760,"num_objects":-2060,"num_object_clones":0,"num_object_copies":-4120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1419,"num_objects_misplaced":1187,"num_objects_unfound":0,"num_objects_dirty":-2060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2060,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002137"},"pg_stats":[{"pgid":"2.f","version":"48'13068","reported_seq":13649,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.863261+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:14.863261+0000","last_peered":"2024-09-10T15:58:14.863261+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:14.863261+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31547392,"num_objects":7702,"num_object_clones":0,"num_object_copies":15404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3723,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13068,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3979},{"shards":"0,2","objects":1290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13206","reported_seq":13318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:15.094243+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:15.094243+0000","last_peered":"2024-09-10T15:58:15.094243+0000","last_clean":"2024-09-10T15:58:15.094243+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:15.094243+0000","last_undegraded":"2024-09-10T15:58:15.094243+0000","last_fullsized":"2024-09-10T15:58:15.094243+0000","mapping_epoch":36,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32227328,"num_objects":7868,"num_object_clones":0,"num_object_copies":15736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11542,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'12900","reported_seq":13256,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:15.076402+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:15.076402+0000","last_peered":"2024-09-10T15:58:15.076402+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:15.076402+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31658008,"num_objects":7730,"num_object_clones":0,"num_object_copies":15460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7730,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11168,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1217},{"shards":"2","objects":4042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'12975","reported_seq":13059,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:15.097884+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:15.097884+0000","last_peered":"2024-09-10T15:58:15.097884+0000","last_clean":"2024-09-10T15:58:15.097884+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:15.097884+0000","last_undegraded":"2024-09-10T15:58:15.097884+0000","last_fullsized":"2024-09-10T15:58:15.097884+0000","mapping_epoch":36,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31567872,"num_objects":7707,"num_object_clones":0,"num_object_copies":15414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12975,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'12735","reported_seq":12866,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:14.897725+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:14.897725+0000","last_peered":"2024-09-10T15:58:14.897725+0000","last_clean":"2024-09-10T15:58:14.897725+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:14.897725+0000","last_undegraded":"2024-09-10T15:58:14.897725+0000","last_fullsized":"2024-09-10T15:58:14.897725+0000","mapping_epoch":35,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31113216,"num_objects":7596,"num_object_clones":0,"num_object_copies":15192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9806,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13001","reported_seq":20892,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:14.941796+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:14.941796+0000","last_peered":"2024-09-10T15:58:14.941796+0000","last_clean":"2024-09-10T15:58:14.941796+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:14.941796+0000","last_undegraded":"2024-09-10T15:58:14.941796+0000","last_fullsized":"2024-09-10T15:58:14.941796+0000","mapping_epoch":47,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32219136,"num_objects":7866,"num_object_clones":0,"num_object_copies":15732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9719,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'12754","reported_seq":13332,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.838178+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:14.838178+0000","last_peered":"2024-09-10T15:58:14.838178+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:14.838178+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30851072,"num_objects":7532,"num_object_clones":0,"num_object_copies":15064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9404,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3915},{"shards":"0,2","objects":1226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13105","reported_seq":13241,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:14.869810+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:14.869810+0000","last_peered":"2024-09-10T15:58:14.869810+0000","last_clean":"2024-09-10T15:58:14.869810+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:14.869810+0000","last_undegraded":"2024-09-10T15:58:14.869810+0000","last_fullsized":"2024-09-10T15:58:14.869810+0000","mapping_epoch":35,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31985664,"num_objects":7809,"num_object_clones":0,"num_object_copies":15618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9864,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13070","reported_seq":21202,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:15.031080+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:15.031080+0000","last_peered":"2024-09-10T15:58:15.031080+0000","last_clean":"2024-09-10T15:58:15.031080+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:15.031080+0000","last_undegraded":"2024-09-10T15:58:15.031080+0000","last_fullsized":"2024-09-10T15:58:15.031080+0000","mapping_epoch":45,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31330304,"num_objects":7649,"num_object_clones":0,"num_object_copies":15298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11047,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'12869","reported_seq":13516,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.904039+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:14.904039+0000","last_peered":"2024-09-10T15:58:14.904039+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:14.904039+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9733,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4104},{"shards":"0,2","objects":1287}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753134+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753134+0000","last_peered":"2024-09-10T15:58:12.753134+0000","last_clean":"2024-09-10T15:58:12.753134+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753134+0000","last_undegraded":"2024-09-10T15:58:12.753134+0000","last_fullsized":"2024-09-10T15:58:12.753134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'12950","reported_seq":13612,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:15.181379+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:15.181379+0000","last_peered":"2024-09-10T15:58:15.181379+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:15.181379+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32092160,"num_objects":7835,"num_object_clones":0,"num_object_copies":15670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2352,"num_objects_misplaced":1216,"num_objects_unfound":0,"num_objects_dirty":7835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10924,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1216},{"shards":"2","objects":4267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'12820","reported_seq":13133,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:15.123058+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:15.123058+0000","last_peered":"2024-09-10T15:58:15.123058+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:15.123058+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31277056,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10689,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1239},{"shards":"2","objects":3898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13011","reported_seq":13639,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.894630+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:14.894630+0000","last_peered":"2024-09-10T15:58:14.894630+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:14.894630+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32124928,"num_objects":7843,"num_object_clones":0,"num_object_copies":15686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10203,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4258},{"shards":"0,2","objects":1250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'12992","reported_seq":13562,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.845823+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:14.845823+0000","last_peered":"2024-09-10T15:58:14.845823+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:14.845823+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31645696,"num_objects":7726,"num_object_clones":0,"num_object_copies":15452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3719,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9410,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4010},{"shards":"0,2","objects":1288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'12937","reported_seq":13537,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.764460+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:14.764460+0000","last_peered":"2024-09-10T15:58:14.764460+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:14.764460+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32067584,"num_objects":7829,"num_object_clones":0,"num_object_copies":15658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9543,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4180},{"shards":"0,2","objects":1325}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13250","reported_seq":13814,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:14.880481+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:14.880481+0000","last_peered":"2024-09-10T15:58:14.880481+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:14.880481+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32145891,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10069,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4058},{"shards":"0,2","objects":1308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":507429371,"num_objects":123886,"num_object_clones":0,"num_object_copies":247772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35480,"num_objects_misplaced":1216,"num_objects_unfound":0,"num_objects_dirty":123886,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169164,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9952,"num_bytes_recovered":32792576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1154064384,"data_stored":1153987593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160743,"ondisk_log_size":160743,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626494,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632172,"kb_used_data":321496,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93739668,"statfs":{"total":96636764160,"available":95989420032,"internally_reserved":0,"allocated":329211904,"data_stored":328873526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318107310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149777,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":836208,"kb_used_data":435292,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93535632,"statfs":{"total":96636764160,"available":95780487168,"internally_reserved":0,"allocated":445739008,"data_stored":445462510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":410513069},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,0,16,1,1,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670544,"kb_used_data":372344,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93701296,"statfs":{"total":96636764160,"available":95950127104,"internally_reserved":0,"allocated":381280256,"data_stored":380999686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":444387328,"data_stored":444383715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328802304,"data_stored":328736811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380874752,"data_stored":380867067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:22.365 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:22.902 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:22.902 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:22.928 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-10T15:58:22.536230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":498328078,"num_objects":121666,"num_object_clones":0,"num_object_copies":243332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35392,"num_objects_misplaced":1100,"num_objects_unfound":0,"num_objects_dirty":121666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171388,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10158,"num_bytes_recovered":33267731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160668,"ondisk_log_size":160668,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2111032,"kb_used_data":1117876,"kb_used_omap":12,"kb_used_meta":993075,"kb_avail":281004488,"statfs":{"total":289910292480,"available":287748595712,"internally_reserved":0,"allocated":1144705024,"data_stored":1143809578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1016908805},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,13,5,2,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17539072,"num_objects":-4282,"num_object_clones":0,"num_object_copies":-8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1507,"num_objects_misplaced":1071,"num_objects_unfound":0,"num_objects_dirty":-4282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4282,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003062"},"pg_stats":[{"pgid":"2.f","version":"48'13228","reported_seq":13841,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.919594+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:19.919594+0000","last_peered":"2024-09-10T15:58:19.919594+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:19.919594+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30892032,"num_objects":7542,"num_object_clones":0,"num_object_copies":15084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13228,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3835},{"shards":"0,2","objects":1130}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13359","reported_seq":13471,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:20.039038+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:20.039038+0000","last_peered":"2024-09-10T15:58:20.039038+0000","last_clean":"2024-09-10T15:58:20.039038+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:20.039038+0000","last_undegraded":"2024-09-10T15:58:20.039038+0000","last_fullsized":"2024-09-10T15:58:20.039038+0000","mapping_epoch":36,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31600640,"num_objects":7715,"num_object_clones":0,"num_object_copies":15430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11695,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13034","reported_seq":13400,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:20.182946+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:20.182946+0000","last_peered":"2024-09-10T15:58:20.182946+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:20.182946+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31109144,"num_objects":7596,"num_object_clones":0,"num_object_copies":15192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3684,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11302,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1083},{"shards":"2","objects":3913}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13102","reported_seq":13186,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:20.131687+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:20.131687+0000","last_peered":"2024-09-10T15:58:20.131687+0000","last_clean":"2024-09-10T15:58:20.131687+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:20.131687+0000","last_undegraded":"2024-09-10T15:58:20.131687+0000","last_fullsized":"2024-09-10T15:58:20.131687+0000","mapping_epoch":36,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31047680,"num_objects":7580,"num_object_clones":0,"num_object_copies":15160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13102,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'12859","reported_seq":12990,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:19.846700+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:19.846700+0000","last_peered":"2024-09-10T15:58:19.846700+0000","last_clean":"2024-09-10T15:58:19.846700+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:19.846700+0000","last_undegraded":"2024-09-10T15:58:19.846700+0000","last_fullsized":"2024-09-10T15:58:19.846700+0000","mapping_epoch":35,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9930,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13149","reported_seq":21042,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:19.834747+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:19.834747+0000","last_peered":"2024-09-10T15:58:19.834747+0000","last_clean":"2024-09-10T15:58:19.834747+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:19.834747+0000","last_undegraded":"2024-09-10T15:58:19.834747+0000","last_fullsized":"2024-09-10T15:58:19.834747+0000","mapping_epoch":47,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31612928,"num_objects":7718,"num_object_clones":0,"num_object_copies":15436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9867,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'12885","reported_seq":13476,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.952682+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:19.952682+0000","last_peered":"2024-09-10T15:58:19.952682+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:19.952682+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314496,"num_objects":7401,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9535,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3790},{"shards":"0,2","objects":1095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13227","reported_seq":13363,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:19.876925+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:19.876925+0000","last_peered":"2024-09-10T15:58:19.876925+0000","last_clean":"2024-09-10T15:58:19.876925+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:19.876925+0000","last_undegraded":"2024-09-10T15:58:19.876925+0000","last_fullsized":"2024-09-10T15:58:19.876925+0000","mapping_epoch":35,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31485952,"num_objects":7687,"num_object_clones":0,"num_object_copies":15374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9986,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13210","reported_seq":21342,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:20.021742+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:20.021742+0000","last_peered":"2024-09-10T15:58:20.021742+0000","last_clean":"2024-09-10T15:58:20.021742+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:20.021742+0000","last_undegraded":"2024-09-10T15:58:20.021742+0000","last_fullsized":"2024-09-10T15:58:20.021742+0000","mapping_epoch":45,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30756864,"num_objects":7509,"num_object_clones":0,"num_object_copies":15018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11187,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13008","reported_seq":13675,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.903687+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:19.903687+0000","last_peered":"2024-09-10T15:58:19.903687+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:19.903687+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3597,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9872,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3975},{"shards":"0,2","objects":1148}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753134+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753134+0000","last_peered":"2024-09-10T15:58:12.753134+0000","last_clean":"2024-09-10T15:58:12.753134+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753134+0000","last_undegraded":"2024-09-10T15:58:12.753134+0000","last_fullsized":"2024-09-10T15:58:12.753134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13081","reported_seq":13975,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:20.202135+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:20.202135+0000","last_peered":"2024-09-10T15:58:20.202135+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:20.202135+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31555584,"num_objects":7704,"num_object_clones":0,"num_object_copies":15408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2352,"num_objects_misplaced":1100,"num_objects_unfound":0,"num_objects_dirty":7704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11055,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":1064960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1100},{"shards":"2","objects":4252}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'12961","reported_seq":13283,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:20.177332+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:20.177332+0000","last_peered":"2024-09-10T15:58:20.177332+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:20.177332+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30699520,"num_objects":7495,"num_object_clones":0,"num_object_copies":14990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10830,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1098},{"shards":"2","objects":3762}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13125","reported_seq":13775,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.922190+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:19.922190+0000","last_peered":"2024-09-10T15:58:19.922190+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:19.922190+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31657984,"num_objects":7729,"num_object_clones":0,"num_object_copies":15458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3575,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10317,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4155},{"shards":"0,2","objects":1136}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13137","reported_seq":13735,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.855162+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:19.855162+0000","last_peered":"2024-09-10T15:58:19.855162+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:19.855162+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31051776,"num_objects":7581,"num_object_clones":0,"num_object_copies":15162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3705,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9555,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3879},{"shards":"0,2","objects":1143}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13087","reported_seq":13709,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.924602+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:19.924602+0000","last_peered":"2024-09-10T15:58:19.924602+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:19.924602+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31453184,"num_objects":7679,"num_object_clones":0,"num_object_copies":15358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9693,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4042},{"shards":"0,2","objects":1175}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13413","reported_seq":13995,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.950519+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:19.950519+0000","last_peered":"2024-09-10T15:58:19.950519+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:19.950519+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31478243,"num_objects":7686,"num_object_clones":0,"num_object_copies":15372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10232,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3904},{"shards":"0,2","objects":1145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":498328059,"num_objects":121664,"num_object_clones":0,"num_object_copies":243328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35392,"num_objects_misplaced":1100,"num_objects_unfound":0,"num_objects_dirty":121664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171386,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10156,"num_bytes_recovered":33267712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1141444608,"data_stored":1141367817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160665,"ondisk_log_size":160665,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626495,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622612,"kb_used_data":320252,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93749228,"statfs":{"total":96636764160,"available":95999209472,"internally_reserved":0,"allocated":327938048,"data_stored":327599670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309587630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149778,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":817144,"kb_used_data":428704,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93554696,"statfs":{"total":96636764160,"available":95800008704,"internally_reserved":0,"allocated":438992896,"data_stored":438716398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":397733549},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,8,3,2,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671276,"kb_used_data":368920,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93700564,"statfs":{"total":96636764160,"available":95949377536,"internally_reserved":0,"allocated":377774080,"data_stored":377493510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":437473280,"data_stored":437469667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327536640,"data_stored":327471147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376434688,"data_stored":376427003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:22.930 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:23.426 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:23.426 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:23.448 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-10T15:58:22.536230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":498328078,"num_objects":121666,"num_object_clones":0,"num_object_copies":243332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35392,"num_objects_misplaced":1100,"num_objects_unfound":0,"num_objects_dirty":121666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171388,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10158,"num_bytes_recovered":33267731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160668,"ondisk_log_size":160668,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2111032,"kb_used_data":1117876,"kb_used_omap":12,"kb_used_meta":993075,"kb_avail":281004488,"statfs":{"total":289910292480,"available":287748595712,"internally_reserved":0,"allocated":1144705024,"data_stored":1143809578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1016908805},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,13,5,2,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17539072,"num_objects":-4282,"num_object_clones":0,"num_object_copies":-8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1507,"num_objects_misplaced":1071,"num_objects_unfound":0,"num_objects_dirty":-4282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4282,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003062"},"pg_stats":[{"pgid":"2.f","version":"48'13228","reported_seq":13841,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.919594+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:19.919594+0000","last_peered":"2024-09-10T15:58:19.919594+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:19.919594+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30892032,"num_objects":7542,"num_object_clones":0,"num_object_copies":15084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13228,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3835},{"shards":"0,2","objects":1130}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13359","reported_seq":13471,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:20.039038+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:20.039038+0000","last_peered":"2024-09-10T15:58:20.039038+0000","last_clean":"2024-09-10T15:58:20.039038+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:20.039038+0000","last_undegraded":"2024-09-10T15:58:20.039038+0000","last_fullsized":"2024-09-10T15:58:20.039038+0000","mapping_epoch":36,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31600640,"num_objects":7715,"num_object_clones":0,"num_object_copies":15430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11695,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13034","reported_seq":13400,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:20.182946+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:20.182946+0000","last_peered":"2024-09-10T15:58:20.182946+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:20.182946+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31109144,"num_objects":7596,"num_object_clones":0,"num_object_copies":15192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3684,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11302,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1083},{"shards":"2","objects":3913}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13102","reported_seq":13186,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:20.131687+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:20.131687+0000","last_peered":"2024-09-10T15:58:20.131687+0000","last_clean":"2024-09-10T15:58:20.131687+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:20.131687+0000","last_undegraded":"2024-09-10T15:58:20.131687+0000","last_fullsized":"2024-09-10T15:58:20.131687+0000","mapping_epoch":36,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31047680,"num_objects":7580,"num_object_clones":0,"num_object_copies":15160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13102,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'12859","reported_seq":12990,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:19.846700+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:19.846700+0000","last_peered":"2024-09-10T15:58:19.846700+0000","last_clean":"2024-09-10T15:58:19.846700+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:19.846700+0000","last_undegraded":"2024-09-10T15:58:19.846700+0000","last_fullsized":"2024-09-10T15:58:19.846700+0000","mapping_epoch":35,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9930,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13149","reported_seq":21042,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:19.834747+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:19.834747+0000","last_peered":"2024-09-10T15:58:19.834747+0000","last_clean":"2024-09-10T15:58:19.834747+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:19.834747+0000","last_undegraded":"2024-09-10T15:58:19.834747+0000","last_fullsized":"2024-09-10T15:58:19.834747+0000","mapping_epoch":47,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31612928,"num_objects":7718,"num_object_clones":0,"num_object_copies":15436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9867,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'12885","reported_seq":13476,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.952682+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:19.952682+0000","last_peered":"2024-09-10T15:58:19.952682+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:19.952682+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314496,"num_objects":7401,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9535,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3790},{"shards":"0,2","objects":1095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13227","reported_seq":13363,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:19.876925+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:19.876925+0000","last_peered":"2024-09-10T15:58:19.876925+0000","last_clean":"2024-09-10T15:58:19.876925+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:19.876925+0000","last_undegraded":"2024-09-10T15:58:19.876925+0000","last_fullsized":"2024-09-10T15:58:19.876925+0000","mapping_epoch":35,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31485952,"num_objects":7687,"num_object_clones":0,"num_object_copies":15374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9986,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13210","reported_seq":21342,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:20.021742+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:20.021742+0000","last_peered":"2024-09-10T15:58:20.021742+0000","last_clean":"2024-09-10T15:58:20.021742+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:20.021742+0000","last_undegraded":"2024-09-10T15:58:20.021742+0000","last_fullsized":"2024-09-10T15:58:20.021742+0000","mapping_epoch":45,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30756864,"num_objects":7509,"num_object_clones":0,"num_object_copies":15018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11187,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13008","reported_seq":13675,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.903687+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:19.903687+0000","last_peered":"2024-09-10T15:58:19.903687+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:19.903687+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3597,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9872,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3975},{"shards":"0,2","objects":1148}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753134+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753134+0000","last_peered":"2024-09-10T15:58:12.753134+0000","last_clean":"2024-09-10T15:58:12.753134+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753134+0000","last_undegraded":"2024-09-10T15:58:12.753134+0000","last_fullsized":"2024-09-10T15:58:12.753134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13081","reported_seq":13975,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:20.202135+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:20.202135+0000","last_peered":"2024-09-10T15:58:20.202135+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:20.202135+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31555584,"num_objects":7704,"num_object_clones":0,"num_object_copies":15408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2352,"num_objects_misplaced":1100,"num_objects_unfound":0,"num_objects_dirty":7704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11055,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":1064960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1100},{"shards":"2","objects":4252}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'12961","reported_seq":13283,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:20.177332+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:20.177332+0000","last_peered":"2024-09-10T15:58:20.177332+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:20.177332+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30699520,"num_objects":7495,"num_object_clones":0,"num_object_copies":14990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10830,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1098},{"shards":"2","objects":3762}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13125","reported_seq":13775,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.922190+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:19.922190+0000","last_peered":"2024-09-10T15:58:19.922190+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:19.922190+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31657984,"num_objects":7729,"num_object_clones":0,"num_object_copies":15458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3575,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10317,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4155},{"shards":"0,2","objects":1136}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13137","reported_seq":13735,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.855162+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:19.855162+0000","last_peered":"2024-09-10T15:58:19.855162+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:19.855162+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31051776,"num_objects":7581,"num_object_clones":0,"num_object_copies":15162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3705,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9555,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3879},{"shards":"0,2","objects":1143}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13087","reported_seq":13709,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.924602+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:19.924602+0000","last_peered":"2024-09-10T15:58:19.924602+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:19.924602+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31453184,"num_objects":7679,"num_object_clones":0,"num_object_copies":15358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9693,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4042},{"shards":"0,2","objects":1175}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13413","reported_seq":13995,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.950519+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:19.950519+0000","last_peered":"2024-09-10T15:58:19.950519+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:19.950519+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31478243,"num_objects":7686,"num_object_clones":0,"num_object_copies":15372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10232,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3904},{"shards":"0,2","objects":1145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":498328059,"num_objects":121664,"num_object_clones":0,"num_object_copies":243328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35392,"num_objects_misplaced":1100,"num_objects_unfound":0,"num_objects_dirty":121664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171386,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10156,"num_bytes_recovered":33267712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1141444608,"data_stored":1141367817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160665,"ondisk_log_size":160665,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626495,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622612,"kb_used_data":320252,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93749228,"statfs":{"total":96636764160,"available":95999209472,"internally_reserved":0,"allocated":327938048,"data_stored":327599670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309587630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149778,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":817144,"kb_used_data":428704,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93554696,"statfs":{"total":96636764160,"available":95800008704,"internally_reserved":0,"allocated":438992896,"data_stored":438716398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":397733549},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,8,3,2,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671276,"kb_used_data":368920,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93700564,"statfs":{"total":96636764160,"available":95949377536,"internally_reserved":0,"allocated":377774080,"data_stored":377493510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":437473280,"data_stored":437469667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327536640,"data_stored":327471147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376434688,"data_stored":376427003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:23.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:58:23.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'13228", 'reported_seq': 13841, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:19.919594+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:58:19.919594+0000', 'last_peered': '2024-09-10T15:58:19.919594+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:58:19.919594+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'3200", 'ondisk_log_start': "37'3200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30892032, 'num_objects': 7542, 'num_object_clones': 0, 'num_object_copies': 15084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3707, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7542, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13228, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 266, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3835}, {'shards': '0,2', 'objects': 1130}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:23.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:58:23.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'13034", 'reported_seq': 13400, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:20.182946+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:58:20.182946+0000', 'last_peered': '2024-09-10T15:58:20.182946+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:58:20.182946+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'3000", 'ondisk_log_start': "37'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31109144, 'num_objects': 7596, 'num_object_clones': 0, 'num_object_copies': 15192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3684, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7596, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11302, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 119, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1083}, {'shards': '2', 'objects': 3913}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:23.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:58:23.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'12885", 'reported_seq': 13476, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:19.952682+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:58:19.952682+0000', 'last_peered': '2024-09-10T15:58:19.952682+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:58:19.952682+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30314496, 'num_objects': 7401, 'num_object_clones': 0, 'num_object_copies': 14802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3613, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7401, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9535, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 224, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3790}, {'shards': '0,2', 'objects': 1095}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:58:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'13008", 'reported_seq': 13675, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:19.903687+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:58:19.903687+0000', 'last_peered': '2024-09-10T15:58:19.903687+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:58:19.903687+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'3000", 'ondisk_log_start': "37'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31006720, 'num_objects': 7570, 'num_object_clones': 0, 'num_object_copies': 15140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3597, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9872, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 268, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3975}, {'shards': '0,2', 'objects': 1148}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:58:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'13081", 'reported_seq': 13975, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:20.202135+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:58:20.202135+0000', 'last_peered': '2024-09-10T15:58:20.202135+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:58:20.202135+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'3000", 'ondisk_log_start': "37'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31555584, 'num_objects': 7704, 'num_object_clones': 0, 'num_object_copies': 15408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2352, 'num_objects_misplaced': 1100, 'num_objects_unfound': 0, 'num_objects_dirty': 7704, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11055, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 382, 'num_bytes_recovered': 1064960, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1100}, {'shards': '2', 'objects': 4252}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:58:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'12961", 'reported_seq': 13283, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:20.177332+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:58:20.177332+0000', 'last_peered': '2024-09-10T15:58:20.177332+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:58:20.177332+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'2900", 'ondisk_log_start': "37'2900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30699520, 'num_objects': 7495, 'num_object_clones': 0, 'num_object_copies': 14990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3735, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7495, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10830, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 96, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1098}, {'shards': '2', 'objects': 3762}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:58:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'13125", 'reported_seq': 13775, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:19.922190+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:58:19.922190+0000', 'last_peered': '2024-09-10T15:58:19.922190+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:58:19.922190+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'3100", 'ondisk_log_start': "37'3100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31657984, 'num_objects': 7729, 'num_object_clones': 0, 'num_object_copies': 15458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3575, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7729, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10317, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 259, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4155}, {'shards': '0,2', 'objects': 1136}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:58:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'13137", 'reported_seq': 13735, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:19.855162+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:58:19.855162+0000', 'last_peered': '2024-09-10T15:58:19.855162+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:58:19.855162+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'3100", 'ondisk_log_start': "37'3100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31051776, 'num_objects': 7581, 'num_object_clones': 0, 'num_object_copies': 15162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3705, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7581, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9555, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 215, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3879}, {'shards': '0,2', 'objects': 1143}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:58:23.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'13087", 'reported_seq': 13709, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:19.924602+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:58:19.924602+0000', 'last_peered': '2024-09-10T15:58:19.924602+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:58:19.924602+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'3000", 'ondisk_log_start': "37'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31453184, 'num_objects': 7679, 'num_object_clones': 0, 'num_object_copies': 15358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3639, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7679, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9693, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 241, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4042}, {'shards': '0,2', 'objects': 1175}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:23.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:58:23.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'13413", 'reported_seq': 13995, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:19.950519+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:58:19.950519+0000', 'last_peered': '2024-09-10T15:58:19.950519+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:58:19.950519+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'3400", 'ondisk_log_start': "37'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31478243, 'num_objects': 7686, 'num_object_clones': 0, 'num_object_copies': 15372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3785, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10232, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 221, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3904}, {'shards': '0,2', 'objects': 1145}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:23.451 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:24.149 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:24.149 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35392/243332 objects degraded (14.545%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":430,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":121666,"data_bytes":498328078,"bytes_used":2161696768,"bytes_avail":287748595712,"bytes_total":289910292480,"degraded_objects":35392,"degraded_total":243332,"degraded_ratio":0.14544737231436886,"misplaced_objects":1100,"misplaced_total":243332,"misplaced_ratio":0.0045205727154669344,"recovering_objects_per_sec":43,"recovering_bytes_per_sec":118338,"recovering_keys_per_sec":0,"num_objects_recovered":437,"num_bytes_recovered":1183744,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":428},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:58:24.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:58:24.178 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:24.725 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:24.726 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:24.751 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-10T15:58:24.536767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":498328078,"num_objects":121666,"num_object_clones":0,"num_object_copies":243332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35392,"num_objects_misplaced":1100,"num_objects_unfound":0,"num_objects_dirty":121666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171388,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10158,"num_bytes_recovered":33267731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160668,"ondisk_log_size":160668,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2111032,"kb_used_data":1117876,"kb_used_omap":12,"kb_used_meta":993075,"kb_avail":281004488,"statfs":{"total":289910292480,"available":287748595712,"internally_reserved":0,"allocated":1144705024,"data_stored":1143809578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1016908805},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,13,5,2,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17539072,"num_objects":-4282,"num_object_clones":0,"num_object_copies":-8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1507,"num_objects_misplaced":1071,"num_objects_unfound":0,"num_objects_dirty":-4282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4282,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.793491"},"pg_stats":[{"pgid":"2.f","version":"48'13228","reported_seq":13841,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.919594+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:19.919594+0000","last_peered":"2024-09-10T15:58:19.919594+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:19.919594+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30892032,"num_objects":7542,"num_object_clones":0,"num_object_copies":15084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13228,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3835},{"shards":"0,2","objects":1130}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13359","reported_seq":13471,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:20.039038+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:20.039038+0000","last_peered":"2024-09-10T15:58:20.039038+0000","last_clean":"2024-09-10T15:58:20.039038+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:20.039038+0000","last_undegraded":"2024-09-10T15:58:20.039038+0000","last_fullsized":"2024-09-10T15:58:20.039038+0000","mapping_epoch":36,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31600640,"num_objects":7715,"num_object_clones":0,"num_object_copies":15430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11695,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13034","reported_seq":13400,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:20.182946+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:20.182946+0000","last_peered":"2024-09-10T15:58:20.182946+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:20.182946+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31109144,"num_objects":7596,"num_object_clones":0,"num_object_copies":15192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3684,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11302,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1083},{"shards":"2","objects":3913}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13102","reported_seq":13186,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:20.131687+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:20.131687+0000","last_peered":"2024-09-10T15:58:20.131687+0000","last_clean":"2024-09-10T15:58:20.131687+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:20.131687+0000","last_undegraded":"2024-09-10T15:58:20.131687+0000","last_fullsized":"2024-09-10T15:58:20.131687+0000","mapping_epoch":36,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31047680,"num_objects":7580,"num_object_clones":0,"num_object_copies":15160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13102,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'12859","reported_seq":12990,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:19.846700+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:19.846700+0000","last_peered":"2024-09-10T15:58:19.846700+0000","last_clean":"2024-09-10T15:58:19.846700+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:19.846700+0000","last_undegraded":"2024-09-10T15:58:19.846700+0000","last_fullsized":"2024-09-10T15:58:19.846700+0000","mapping_epoch":35,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9930,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13149","reported_seq":21042,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:19.834747+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:19.834747+0000","last_peered":"2024-09-10T15:58:19.834747+0000","last_clean":"2024-09-10T15:58:19.834747+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:19.834747+0000","last_undegraded":"2024-09-10T15:58:19.834747+0000","last_fullsized":"2024-09-10T15:58:19.834747+0000","mapping_epoch":47,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31612928,"num_objects":7718,"num_object_clones":0,"num_object_copies":15436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9867,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'12885","reported_seq":13476,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.952682+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:19.952682+0000","last_peered":"2024-09-10T15:58:19.952682+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:19.952682+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314496,"num_objects":7401,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9535,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3790},{"shards":"0,2","objects":1095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13227","reported_seq":13363,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:19.876925+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:19.876925+0000","last_peered":"2024-09-10T15:58:19.876925+0000","last_clean":"2024-09-10T15:58:19.876925+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:19.876925+0000","last_undegraded":"2024-09-10T15:58:19.876925+0000","last_fullsized":"2024-09-10T15:58:19.876925+0000","mapping_epoch":35,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31485952,"num_objects":7687,"num_object_clones":0,"num_object_copies":15374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9986,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13210","reported_seq":21342,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:20.021742+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:20.021742+0000","last_peered":"2024-09-10T15:58:20.021742+0000","last_clean":"2024-09-10T15:58:20.021742+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:20.021742+0000","last_undegraded":"2024-09-10T15:58:20.021742+0000","last_fullsized":"2024-09-10T15:58:20.021742+0000","mapping_epoch":45,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30756864,"num_objects":7509,"num_object_clones":0,"num_object_copies":15018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11187,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13008","reported_seq":13675,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.903687+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:19.903687+0000","last_peered":"2024-09-10T15:58:19.903687+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:19.903687+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3597,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9872,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3975},{"shards":"0,2","objects":1148}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753134+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753134+0000","last_peered":"2024-09-10T15:58:12.753134+0000","last_clean":"2024-09-10T15:58:12.753134+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753134+0000","last_undegraded":"2024-09-10T15:58:12.753134+0000","last_fullsized":"2024-09-10T15:58:12.753134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13081","reported_seq":13975,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:20.202135+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:20.202135+0000","last_peered":"2024-09-10T15:58:20.202135+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:20.202135+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31555584,"num_objects":7704,"num_object_clones":0,"num_object_copies":15408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2352,"num_objects_misplaced":1100,"num_objects_unfound":0,"num_objects_dirty":7704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11055,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":1064960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1100},{"shards":"2","objects":4252}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'12961","reported_seq":13283,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:20.177332+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:20.177332+0000","last_peered":"2024-09-10T15:58:20.177332+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:20.177332+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30699520,"num_objects":7495,"num_object_clones":0,"num_object_copies":14990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10830,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1098},{"shards":"2","objects":3762}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13125","reported_seq":13775,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.922190+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:19.922190+0000","last_peered":"2024-09-10T15:58:19.922190+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:19.922190+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31657984,"num_objects":7729,"num_object_clones":0,"num_object_copies":15458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3575,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10317,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4155},{"shards":"0,2","objects":1136}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13137","reported_seq":13735,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.855162+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:19.855162+0000","last_peered":"2024-09-10T15:58:19.855162+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:19.855162+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31051776,"num_objects":7581,"num_object_clones":0,"num_object_copies":15162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3705,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9555,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3879},{"shards":"0,2","objects":1143}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13087","reported_seq":13709,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.924602+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:19.924602+0000","last_peered":"2024-09-10T15:58:19.924602+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:19.924602+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31453184,"num_objects":7679,"num_object_clones":0,"num_object_copies":15358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9693,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4042},{"shards":"0,2","objects":1175}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13413","reported_seq":13995,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:19.950519+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:19.950519+0000","last_peered":"2024-09-10T15:58:19.950519+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:19.950519+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31478243,"num_objects":7686,"num_object_clones":0,"num_object_copies":15372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10232,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3904},{"shards":"0,2","objects":1145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":498328059,"num_objects":121664,"num_object_clones":0,"num_object_copies":243328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35392,"num_objects_misplaced":1100,"num_objects_unfound":0,"num_objects_dirty":121664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171386,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10156,"num_bytes_recovered":33267712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1141444608,"data_stored":1141367817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160665,"ondisk_log_size":160665,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626495,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622612,"kb_used_data":320252,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93749228,"statfs":{"total":96636764160,"available":95999209472,"internally_reserved":0,"allocated":327938048,"data_stored":327599670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309587630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149778,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":817144,"kb_used_data":428704,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93554696,"statfs":{"total":96636764160,"available":95800008704,"internally_reserved":0,"allocated":438992896,"data_stored":438716398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":397733549},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,8,3,2,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671276,"kb_used_data":368920,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93700564,"statfs":{"total":96636764160,"available":95949377536,"internally_reserved":0,"allocated":377774080,"data_stored":377493510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":437473280,"data_stored":437469667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327536640,"data_stored":327471147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376434688,"data_stored":376427003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:27.753 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:28.258 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:28.259 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:28.283 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-10T15:58:26.537570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":489316878,"num_objects":119466,"num_object_clones":0,"num_object_copies":238932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35305,"num_objects_misplaced":990,"num_objects_unfound":0,"num_objects_dirty":119466,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173588,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10355,"num_bytes_recovered":33706003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110668,"kb_used_data":1105052,"kb_used_omap":12,"kb_used_meta":1005555,"kb_avail":281004852,"statfs":{"total":289910292480,"available":287748968448,"internally_reserved":0,"allocated":1131573248,"data_stored":1130677802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1029688325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,16,6,5,5,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26550272,"num_objects":-6482,"num_object_clones":0,"num_object_copies":-12964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1594,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":-6482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6482,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":634,"num_bytes_recovered":1622016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003889"},"pg_stats":[{"pgid":"2.f","version":"48'13352","reported_seq":13999,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.953058+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:24.953058+0000","last_peered":"2024-09-10T15:58:24.953058+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:24.953058+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30384128,"num_objects":7418,"num_object_clones":0,"num_object_copies":14836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13352,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3728},{"shards":"0,2","objects":1006}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13505","reported_seq":13617,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.195479+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:25.195479+0000","last_peered":"2024-09-10T15:58:25.195479+0000","last_clean":"2024-09-10T15:58:25.195479+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:25.195479+0000","last_undegraded":"2024-09-10T15:58:25.195479+0000","last_fullsized":"2024-09-10T15:58:25.195479+0000","mapping_epoch":36,"log_start":"37'3500","ondisk_log_start":"37'3500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31002624,"num_objects":7569,"num_object_clones":0,"num_object_copies":15138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11841,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13162","reported_seq":13541,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:25.178964+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:25.178964+0000","last_peered":"2024-09-10T15:58:25.178964+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:25.178964+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30584856,"num_objects":7468,"num_object_clones":0,"num_object_copies":14936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3678,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7468,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11430,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":955},{"shards":"2","objects":3791}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13241","reported_seq":13325,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.205702+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:25.205702+0000","last_peered":"2024-09-10T15:58:25.205702+0000","last_clean":"2024-09-10T15:58:25.205702+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:25.205702+0000","last_undegraded":"2024-09-10T15:58:25.205702+0000","last_fullsized":"2024-09-10T15:58:25.205702+0000","mapping_epoch":36,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30478336,"num_objects":7441,"num_object_clones":0,"num_object_copies":14882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13241,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'12998","reported_seq":13129,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.921024+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:24.921024+0000","last_peered":"2024-09-10T15:58:24.921024+0000","last_clean":"2024-09-10T15:58:24.921024+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:24.921024+0000","last_undegraded":"2024-09-10T15:58:24.921024+0000","last_fullsized":"2024-09-10T15:58:24.921024+0000","mapping_epoch":35,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30035968,"num_objects":7333,"num_object_clones":0,"num_object_copies":14666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10069,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13296","reported_seq":21189,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.994740+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:24.994740+0000","last_peered":"2024-09-10T15:58:24.994740+0000","last_clean":"2024-09-10T15:58:24.994740+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:24.994740+0000","last_undegraded":"2024-09-10T15:58:24.994740+0000","last_fullsized":"2024-09-10T15:58:24.994740+0000","mapping_epoch":47,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31010816,"num_objects":7571,"num_object_clones":0,"num_object_copies":15142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10014,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13001","reported_seq":13610,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.988962+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:24.988962+0000","last_peered":"2024-09-10T15:58:24.988962+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:24.988962+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839360,"num_objects":7285,"num_object_clones":0,"num_object_copies":14570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9651,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3683},{"shards":"0,2","objects":979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13358","reported_seq":13494,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.919033+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:24.919033+0000","last_peered":"2024-09-10T15:58:24.919033+0000","last_clean":"2024-09-10T15:58:24.919033+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:24.919033+0000","last_undegraded":"2024-09-10T15:58:24.919033+0000","last_fullsized":"2024-09-10T15:58:24.919033+0000","mapping_epoch":35,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30949376,"num_objects":7556,"num_object_clones":0,"num_object_copies":15112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10117,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13344","reported_seq":21476,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.800498+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:24.800498+0000","last_peered":"2024-09-10T15:58:24.800498+0000","last_clean":"2024-09-10T15:58:24.800498+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:24.800498+0000","last_undegraded":"2024-09-10T15:58:24.800498+0000","last_fullsized":"2024-09-10T15:58:24.800498+0000","mapping_epoch":45,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30208000,"num_objects":7375,"num_object_clones":0,"num_object_copies":14750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11321,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13172","reported_seq":13857,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.984694+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:24.984694+0000","last_peered":"2024-09-10T15:58:24.984694+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:24.984694+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10036,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3820},{"shards":"0,2","objects":984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13236","reported_seq":14344,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:25.236620+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:25.236620+0000","last_peered":"2024-09-10T15:58:25.236620+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:25.236620+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30920704,"num_objects":7549,"num_object_clones":0,"num_object_copies":15098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2355,"num_objects_misplaced":990,"num_objects_unfound":0,"num_objects_dirty":7549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11210,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":1503232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":990},{"shards":"2","objects":4204}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13081","reported_seq":13407,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:25.204792+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:25.204792+0000","last_peered":"2024-09-10T15:58:25.204792+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:25.204792+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30208000,"num_objects":7375,"num_object_clones":0,"num_object_copies":14750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10950,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":978},{"shards":"2","objects":3645}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13249","reported_seq":13923,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.880034+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:24.880034+0000","last_peered":"2024-09-10T15:58:24.880034+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:24.880034+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31150080,"num_objects":7605,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10441,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4043},{"shards":"0,2","objects":1012}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13278","reported_seq":13892,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.966078+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:24.966078+0000","last_peered":"2024-09-10T15:58:24.966078+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:24.966078+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30474240,"num_objects":7440,"num_object_clones":0,"num_object_copies":14880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9696,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3746},{"shards":"0,2","objects":1002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13231","reported_seq":13883,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.972486+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:24.972486+0000","last_peered":"2024-09-10T15:58:24.972486+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:24.972486+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30863360,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9837,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3912},{"shards":"0,2","objects":1031}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13561","reported_seq":14170,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.956025+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:24.956025+0000","last_peered":"2024-09-10T15:58:24.956025+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:24.956025+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3500","ondisk_log_start":"37'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30872035,"num_objects":7538,"num_object_clones":0,"num_object_copies":15076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10380,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3768},{"shards":"0,2","objects":997}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":489316859,"num_objects":119464,"num_object_clones":0,"num_object_copies":238928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35305,"num_objects_misplaced":990,"num_objects_unfound":0,"num_objects_dirty":119464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173586,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10353,"num_bytes_recovered":33705984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1128914944,"data_stored":1128838153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626496,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625736,"kb_used_data":319224,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93746104,"statfs":{"total":96636764160,"available":95996010496,"internally_reserved":0,"allocated":326885376,"data_stored":326546998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313847470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149779,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":814720,"kb_used_data":422128,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93557120,"statfs":{"total":96636764160,"available":95802490880,"internally_reserved":0,"allocated":432259072,"data_stored":431982574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":401993389},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,14,1,2,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670212,"kb_used_data":363700,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93701628,"statfs":{"total":96636764160,"available":95950467072,"internally_reserved":0,"allocated":372428800,"data_stored":372148230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":430706688,"data_stored":430703075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326225920,"data_stored":326160427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":371982336,"data_stored":371974651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:28.285 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:28.814 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:28.815 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:28.840 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-09-10T15:58:28.538433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":489316878,"num_objects":119466,"num_object_clones":0,"num_object_copies":238932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35305,"num_objects_misplaced":990,"num_objects_unfound":0,"num_objects_dirty":119466,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173588,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10355,"num_bytes_recovered":33706003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110668,"kb_used_data":1105052,"kb_used_omap":12,"kb_used_meta":1005555,"kb_avail":281004852,"statfs":{"total":289910292480,"available":287748968448,"internally_reserved":0,"allocated":1131573248,"data_stored":1130677802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1029688325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,16,6,5,5,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18112512,"num_objects":-4422,"num_object_clones":0,"num_object_copies":-8844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-175,"num_objects_misplaced":-226,"num_objects_unfound":0,"num_objects_dirty":-4422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4422,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004155"},"pg_stats":[{"pgid":"2.f","version":"48'13352","reported_seq":13999,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.953058+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:24.953058+0000","last_peered":"2024-09-10T15:58:24.953058+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:24.953058+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30384128,"num_objects":7418,"num_object_clones":0,"num_object_copies":14836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13352,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3728},{"shards":"0,2","objects":1006}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13505","reported_seq":13617,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.195479+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:25.195479+0000","last_peered":"2024-09-10T15:58:25.195479+0000","last_clean":"2024-09-10T15:58:25.195479+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:25.195479+0000","last_undegraded":"2024-09-10T15:58:25.195479+0000","last_fullsized":"2024-09-10T15:58:25.195479+0000","mapping_epoch":36,"log_start":"37'3500","ondisk_log_start":"37'3500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31002624,"num_objects":7569,"num_object_clones":0,"num_object_copies":15138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11841,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13162","reported_seq":13541,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:25.178964+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:25.178964+0000","last_peered":"2024-09-10T15:58:25.178964+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:25.178964+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30584856,"num_objects":7468,"num_object_clones":0,"num_object_copies":14936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3678,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7468,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11430,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":955},{"shards":"2","objects":3791}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13241","reported_seq":13325,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.205702+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:25.205702+0000","last_peered":"2024-09-10T15:58:25.205702+0000","last_clean":"2024-09-10T15:58:25.205702+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:25.205702+0000","last_undegraded":"2024-09-10T15:58:25.205702+0000","last_fullsized":"2024-09-10T15:58:25.205702+0000","mapping_epoch":36,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30478336,"num_objects":7441,"num_object_clones":0,"num_object_copies":14882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13241,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'12998","reported_seq":13129,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.921024+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:24.921024+0000","last_peered":"2024-09-10T15:58:24.921024+0000","last_clean":"2024-09-10T15:58:24.921024+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:24.921024+0000","last_undegraded":"2024-09-10T15:58:24.921024+0000","last_fullsized":"2024-09-10T15:58:24.921024+0000","mapping_epoch":35,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30035968,"num_objects":7333,"num_object_clones":0,"num_object_copies":14666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10069,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13296","reported_seq":21189,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.994740+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:24.994740+0000","last_peered":"2024-09-10T15:58:24.994740+0000","last_clean":"2024-09-10T15:58:24.994740+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:24.994740+0000","last_undegraded":"2024-09-10T15:58:24.994740+0000","last_fullsized":"2024-09-10T15:58:24.994740+0000","mapping_epoch":47,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31010816,"num_objects":7571,"num_object_clones":0,"num_object_copies":15142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10014,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13001","reported_seq":13610,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.988962+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:24.988962+0000","last_peered":"2024-09-10T15:58:24.988962+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:24.988962+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839360,"num_objects":7285,"num_object_clones":0,"num_object_copies":14570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9651,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3683},{"shards":"0,2","objects":979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13358","reported_seq":13494,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.919033+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:24.919033+0000","last_peered":"2024-09-10T15:58:24.919033+0000","last_clean":"2024-09-10T15:58:24.919033+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:24.919033+0000","last_undegraded":"2024-09-10T15:58:24.919033+0000","last_fullsized":"2024-09-10T15:58:24.919033+0000","mapping_epoch":35,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30949376,"num_objects":7556,"num_object_clones":0,"num_object_copies":15112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10117,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13344","reported_seq":21476,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.800498+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:24.800498+0000","last_peered":"2024-09-10T15:58:24.800498+0000","last_clean":"2024-09-10T15:58:24.800498+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:24.800498+0000","last_undegraded":"2024-09-10T15:58:24.800498+0000","last_fullsized":"2024-09-10T15:58:24.800498+0000","mapping_epoch":45,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30208000,"num_objects":7375,"num_object_clones":0,"num_object_copies":14750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11321,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13172","reported_seq":13857,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.984694+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:24.984694+0000","last_peered":"2024-09-10T15:58:24.984694+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:24.984694+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10036,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3820},{"shards":"0,2","objects":984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13236","reported_seq":14344,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:25.236620+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:25.236620+0000","last_peered":"2024-09-10T15:58:25.236620+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:25.236620+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30920704,"num_objects":7549,"num_object_clones":0,"num_object_copies":15098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2355,"num_objects_misplaced":990,"num_objects_unfound":0,"num_objects_dirty":7549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11210,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":1503232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":990},{"shards":"2","objects":4204}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13081","reported_seq":13407,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:25.204792+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:25.204792+0000","last_peered":"2024-09-10T15:58:25.204792+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:25.204792+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30208000,"num_objects":7375,"num_object_clones":0,"num_object_copies":14750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10950,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":978},{"shards":"2","objects":3645}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13249","reported_seq":13923,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.880034+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:24.880034+0000","last_peered":"2024-09-10T15:58:24.880034+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:24.880034+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31150080,"num_objects":7605,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10441,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4043},{"shards":"0,2","objects":1012}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13278","reported_seq":13892,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.966078+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:24.966078+0000","last_peered":"2024-09-10T15:58:24.966078+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:24.966078+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30474240,"num_objects":7440,"num_object_clones":0,"num_object_copies":14880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9696,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3746},{"shards":"0,2","objects":1002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13231","reported_seq":13883,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.972486+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:24.972486+0000","last_peered":"2024-09-10T15:58:24.972486+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:24.972486+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30863360,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9837,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3912},{"shards":"0,2","objects":1031}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13561","reported_seq":14170,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.956025+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:24.956025+0000","last_peered":"2024-09-10T15:58:24.956025+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:24.956025+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3500","ondisk_log_start":"37'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30872035,"num_objects":7538,"num_object_clones":0,"num_object_copies":15076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10380,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3768},{"shards":"0,2","objects":997}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":489316859,"num_objects":119464,"num_object_clones":0,"num_object_copies":238928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35305,"num_objects_misplaced":990,"num_objects_unfound":0,"num_objects_dirty":119464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173586,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10353,"num_bytes_recovered":33705984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1128914944,"data_stored":1128838153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626496,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625736,"kb_used_data":319224,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93746104,"statfs":{"total":96636764160,"available":95996010496,"internally_reserved":0,"allocated":326885376,"data_stored":326546998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313847470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149779,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":814720,"kb_used_data":422128,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93557120,"statfs":{"total":96636764160,"available":95802490880,"internally_reserved":0,"allocated":432259072,"data_stored":431982574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":401993389},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,14,1,2,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670212,"kb_used_data":363700,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93701628,"statfs":{"total":96636764160,"available":95950467072,"internally_reserved":0,"allocated":372428800,"data_stored":372148230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":430706688,"data_stored":430703075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326225920,"data_stored":326160427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":371982336,"data_stored":371974651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:28.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:58:28.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'13352", 'reported_seq': 13999, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:24.953058+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:58:24.953058+0000', 'last_peered': '2024-09-10T15:58:24.953058+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:58:24.953058+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'3300", 'ondisk_log_start': "37'3300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30384128, 'num_objects': 7418, 'num_object_clones': 0, 'num_object_copies': 14836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3690, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7418, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13352, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 283, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3728}, {'shards': '0,2', 'objects': 1006}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:28.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:58:28.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'13162", 'reported_seq': 13541, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:25.178964+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:58:25.178964+0000', 'last_peered': '2024-09-10T15:58:25.178964+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:58:25.178964+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'3100", 'ondisk_log_start': "37'3100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30584856, 'num_objects': 7468, 'num_object_clones': 0, 'num_object_copies': 14936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3678, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7468, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11430, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 125, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 955}, {'shards': '2', 'objects': 3791}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:28.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:58:28.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'13001", 'reported_seq': 13610, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:24.988962+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:58:24.988962+0000', 'last_peered': '2024-09-10T15:58:24.988962+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:58:24.988962+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'3000", 'ondisk_log_start': "37'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29839360, 'num_objects': 7285, 'num_object_clones': 0, 'num_object_copies': 14570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3604, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7285, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9651, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 233, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3683}, {'shards': '0,2', 'objects': 979}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:28.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:58:28.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'13172", 'reported_seq': 13857, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:24.984694+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:58:24.984694+0000', 'last_peered': '2024-09-10T15:58:24.984694+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:58:24.984694+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'3100", 'ondisk_log_start': "37'3100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30334976, 'num_objects': 7406, 'num_object_clones': 0, 'num_object_copies': 14812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3588, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10036, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 277, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3820}, {'shards': '0,2', 'objects': 984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:28.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:58:28.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'13236", 'reported_seq': 14344, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:25.236620+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:58:25.236620+0000', 'last_peered': '2024-09-10T15:58:25.236620+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:58:25.236620+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'3200", 'ondisk_log_start': "37'3200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30920704, 'num_objects': 7549, 'num_object_clones': 0, 'num_object_copies': 15098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2355, 'num_objects_misplaced': 990, 'num_objects_unfound': 0, 'num_objects_dirty': 7549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11210, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 489, 'num_bytes_recovered': 1503232, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 990}, {'shards': '2', 'objects': 4204}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:28.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:58:28.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'13081", 'reported_seq': 13407, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:25.204792+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:58:25.204792+0000', 'last_peered': '2024-09-10T15:58:25.204792+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:58:25.204792+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'3000", 'ondisk_log_start': "37'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30208000, 'num_objects': 7375, 'num_object_clones': 0, 'num_object_copies': 14750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3732, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7375, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10950, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 99, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 978}, {'shards': '2', 'objects': 3645}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:28.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:58:28.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'13249", 'reported_seq': 13923, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:24.880034+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:58:24.880034+0000', 'last_peered': '2024-09-10T15:58:24.880034+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:58:24.880034+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'3200", 'ondisk_log_start': "37'3200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31150080, 'num_objects': 7605, 'num_object_clones': 0, 'num_object_copies': 15210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7605, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10441, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 271, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4043}, {'shards': '0,2', 'objects': 1012}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:28.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:58:28.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'13278", 'reported_seq': 13892, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:24.966078+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:58:24.966078+0000', 'last_peered': '2024-09-10T15:58:24.966078+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:58:24.966078+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'3200", 'ondisk_log_start': "37'3200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30474240, 'num_objects': 7440, 'num_object_clones': 0, 'num_object_copies': 14880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7440, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9696, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 223, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3746}, {'shards': '0,2', 'objects': 1002}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:28.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:58:28.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'13231", 'reported_seq': 13883, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:24.972486+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:58:24.972486+0000', 'last_peered': '2024-09-10T15:58:24.972486+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:58:24.972486+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'3200", 'ondisk_log_start': "37'3200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30863360, 'num_objects': 7535, 'num_object_clones': 0, 'num_object_copies': 15070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3625, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7535, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9837, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 255, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3912}, {'shards': '0,2', 'objects': 1031}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:28.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:58:28.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'13561", 'reported_seq': 14170, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:24.956025+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:58:24.956025+0000', 'last_peered': '2024-09-10T15:58:24.956025+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:58:24.956025+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'3500", 'ondisk_log_start': "37'3500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30872035, 'num_objects': 7538, 'num_object_clones': 0, 'num_object_copies': 15076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3773, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7538, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10380, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 233, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3768}, {'shards': '0,2', 'objects': 997}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:28.843 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:29.495 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:29.495 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35305/238932 objects degraded (14.776%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":435,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":119466,"data_bytes":489316878,"bytes_used":2161324032,"bytes_avail":287748968448,"bytes_total":289910292480,"degraded_objects":35305,"degraded_total":238932,"degraded_ratio":0.14776170625952154,"misplaced_objects":990,"misplaced_total":238932,"misplaced_ratio":0.0041434383004369446,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":76090,"recovering_keys_per_sec":0,"num_objects_recovered":401,"num_bytes_recovered":913408,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":368},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:58:29.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:58:29.522 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:30.084 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:30.086 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:30.110 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-09-10T15:58:28.538433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":489316878,"num_objects":119466,"num_object_clones":0,"num_object_copies":238932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35305,"num_objects_misplaced":990,"num_objects_unfound":0,"num_objects_dirty":119466,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173588,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10355,"num_bytes_recovered":33706003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110668,"kb_used_data":1105052,"kb_used_omap":12,"kb_used_meta":1005555,"kb_avail":281004852,"statfs":{"total":289910292480,"available":287748968448,"internally_reserved":0,"allocated":1131573248,"data_stored":1130677802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1029688325},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,16,6,5,5,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18112512,"num_objects":-4422,"num_object_clones":0,"num_object_copies":-8844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-175,"num_objects_misplaced":-226,"num_objects_unfound":0,"num_objects_dirty":-4422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4422,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004155"},"pg_stats":[{"pgid":"2.f","version":"48'13352","reported_seq":13999,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.953058+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:24.953058+0000","last_peered":"2024-09-10T15:58:24.953058+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:24.953058+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30384128,"num_objects":7418,"num_object_clones":0,"num_object_copies":14836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13352,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3728},{"shards":"0,2","objects":1006}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13505","reported_seq":13617,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.195479+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:25.195479+0000","last_peered":"2024-09-10T15:58:25.195479+0000","last_clean":"2024-09-10T15:58:25.195479+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:25.195479+0000","last_undegraded":"2024-09-10T15:58:25.195479+0000","last_fullsized":"2024-09-10T15:58:25.195479+0000","mapping_epoch":36,"log_start":"37'3500","ondisk_log_start":"37'3500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31002624,"num_objects":7569,"num_object_clones":0,"num_object_copies":15138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11841,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13162","reported_seq":13541,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:25.178964+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:25.178964+0000","last_peered":"2024-09-10T15:58:25.178964+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:25.178964+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30584856,"num_objects":7468,"num_object_clones":0,"num_object_copies":14936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3678,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7468,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11430,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":955},{"shards":"2","objects":3791}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13241","reported_seq":13325,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:25.205702+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:25.205702+0000","last_peered":"2024-09-10T15:58:25.205702+0000","last_clean":"2024-09-10T15:58:25.205702+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:25.205702+0000","last_undegraded":"2024-09-10T15:58:25.205702+0000","last_fullsized":"2024-09-10T15:58:25.205702+0000","mapping_epoch":36,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30478336,"num_objects":7441,"num_object_clones":0,"num_object_copies":14882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13241,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'12998","reported_seq":13129,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.921024+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:24.921024+0000","last_peered":"2024-09-10T15:58:24.921024+0000","last_clean":"2024-09-10T15:58:24.921024+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:24.921024+0000","last_undegraded":"2024-09-10T15:58:24.921024+0000","last_fullsized":"2024-09-10T15:58:24.921024+0000","mapping_epoch":35,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30035968,"num_objects":7333,"num_object_clones":0,"num_object_copies":14666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10069,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13296","reported_seq":21189,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.994740+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:24.994740+0000","last_peered":"2024-09-10T15:58:24.994740+0000","last_clean":"2024-09-10T15:58:24.994740+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:24.994740+0000","last_undegraded":"2024-09-10T15:58:24.994740+0000","last_fullsized":"2024-09-10T15:58:24.994740+0000","mapping_epoch":47,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31010816,"num_objects":7571,"num_object_clones":0,"num_object_copies":15142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10014,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13001","reported_seq":13610,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.988962+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:24.988962+0000","last_peered":"2024-09-10T15:58:24.988962+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:24.988962+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839360,"num_objects":7285,"num_object_clones":0,"num_object_copies":14570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9651,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3683},{"shards":"0,2","objects":979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13358","reported_seq":13494,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.919033+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:24.919033+0000","last_peered":"2024-09-10T15:58:24.919033+0000","last_clean":"2024-09-10T15:58:24.919033+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:24.919033+0000","last_undegraded":"2024-09-10T15:58:24.919033+0000","last_fullsized":"2024-09-10T15:58:24.919033+0000","mapping_epoch":35,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30949376,"num_objects":7556,"num_object_clones":0,"num_object_copies":15112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10117,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13344","reported_seq":21476,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.800498+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:24.800498+0000","last_peered":"2024-09-10T15:58:24.800498+0000","last_clean":"2024-09-10T15:58:24.800498+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:24.800498+0000","last_undegraded":"2024-09-10T15:58:24.800498+0000","last_fullsized":"2024-09-10T15:58:24.800498+0000","mapping_epoch":45,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30208000,"num_objects":7375,"num_object_clones":0,"num_object_copies":14750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11321,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13172","reported_seq":13857,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.984694+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:24.984694+0000","last_peered":"2024-09-10T15:58:24.984694+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:24.984694+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30334976,"num_objects":7406,"num_object_clones":0,"num_object_copies":14812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10036,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3820},{"shards":"0,2","objects":984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13236","reported_seq":14344,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:25.236620+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:25.236620+0000","last_peered":"2024-09-10T15:58:25.236620+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:25.236620+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30920704,"num_objects":7549,"num_object_clones":0,"num_object_copies":15098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2355,"num_objects_misplaced":990,"num_objects_unfound":0,"num_objects_dirty":7549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11210,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":1503232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":990},{"shards":"2","objects":4204}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13081","reported_seq":13407,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:25.204792+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:25.204792+0000","last_peered":"2024-09-10T15:58:25.204792+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:25.204792+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'3000","ondisk_log_start":"37'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30208000,"num_objects":7375,"num_object_clones":0,"num_object_copies":14750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10950,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":978},{"shards":"2","objects":3645}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13249","reported_seq":13923,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.880034+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:24.880034+0000","last_peered":"2024-09-10T15:58:24.880034+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:24.880034+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31150080,"num_objects":7605,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10441,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4043},{"shards":"0,2","objects":1012}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13278","reported_seq":13892,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.966078+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:24.966078+0000","last_peered":"2024-09-10T15:58:24.966078+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:24.966078+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30474240,"num_objects":7440,"num_object_clones":0,"num_object_copies":14880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9696,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3746},{"shards":"0,2","objects":1002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13231","reported_seq":13883,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.972486+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:24.972486+0000","last_peered":"2024-09-10T15:58:24.972486+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:24.972486+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30863360,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9837,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3912},{"shards":"0,2","objects":1031}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13561","reported_seq":14170,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:24.956025+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:24.956025+0000","last_peered":"2024-09-10T15:58:24.956025+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:24.956025+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3500","ondisk_log_start":"37'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30872035,"num_objects":7538,"num_object_clones":0,"num_object_copies":15076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10380,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3768},{"shards":"0,2","objects":997}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":489316859,"num_objects":119464,"num_object_clones":0,"num_object_copies":238928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35305,"num_objects_misplaced":990,"num_objects_unfound":0,"num_objects_dirty":119464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173586,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10353,"num_bytes_recovered":33705984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1128914944,"data_stored":1128838153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626496,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625736,"kb_used_data":319224,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93746104,"statfs":{"total":96636764160,"available":95996010496,"internally_reserved":0,"allocated":326885376,"data_stored":326546998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313847470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149779,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":814720,"kb_used_data":422128,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93557120,"statfs":{"total":96636764160,"available":95802490880,"internally_reserved":0,"allocated":432259072,"data_stored":431982574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":401993389},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,14,1,2,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670212,"kb_used_data":363700,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93701628,"statfs":{"total":96636764160,"available":95950467072,"internally_reserved":0,"allocated":372428800,"data_stored":372148230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":430706688,"data_stored":430703075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326225920,"data_stored":326160427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":371982336,"data_stored":371974651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:33.112 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:33.559 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:33.561 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:33.584 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-10T15:58:32.540153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":480383502,"num_objects":117285,"num_object_clones":0,"num_object_copies":234570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35263,"num_objects_misplaced":828,"num_objects_unfound":0,"num_objects_dirty":117285,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":175769,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10559,"num_bytes_recovered":34234387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160949,"ondisk_log_size":160949,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2120768,"kb_used_data":1092928,"kb_used_omap":12,"kb_used_meta":1027763,"kb_avail":280994752,"statfs":{"total":289910292480,"available":287738626048,"internally_reserved":0,"allocated":1119158272,"data_stored":1118262826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1052429317},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,3,3,8,5,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17944576,"num_objects":-4381,"num_object_clones":0,"num_object_copies":-8762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-129,"num_objects_misplaced":-272,"num_objects_unfound":0,"num_objects_dirty":-4381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4381,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":966656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004619"},"pg_stats":[{"pgid":"2.f","version":"48'13474","reported_seq":14149,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.940637+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:29.940637+0000","last_peered":"2024-09-10T15:58:29.940637+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:29.940637+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13474,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3620},{"shards":"0,2","objects":884}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13641","reported_seq":13753,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:30.183627+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:30.183627+0000","last_peered":"2024-09-10T15:58:30.183627+0000","last_clean":"2024-09-10T15:58:30.183627+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:30.183627+0000","last_undegraded":"2024-09-10T15:58:30.183627+0000","last_fullsized":"2024-09-10T15:58:30.183627+0000","mapping_epoch":36,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445568,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11977,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13293","reported_seq":13684,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:30.243680+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:30.243680+0000","last_peered":"2024-09-10T15:58:30.243680+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:30.243680+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30048280,"num_objects":7337,"num_object_clones":0,"num_object_copies":14674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11561,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":824},{"shards":"2","objects":3666}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13387","reported_seq":13471,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:30.246735+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:30.246735+0000","last_peered":"2024-09-10T15:58:30.246735+0000","last_clean":"2024-09-10T15:58:30.246735+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:30.246735+0000","last_undegraded":"2024-09-10T15:58:30.246735+0000","last_fullsized":"2024-09-10T15:58:30.246735+0000","mapping_epoch":36,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29880320,"num_objects":7295,"num_object_clones":0,"num_object_copies":14590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13387,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13129","reported_seq":13260,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:29.940263+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:29.940263+0000","last_peered":"2024-09-10T15:58:29.940263+0000","last_clean":"2024-09-10T15:58:29.940263+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:29.940263+0000","last_undegraded":"2024-09-10T15:58:29.940263+0000","last_fullsized":"2024-09-10T15:58:29.940263+0000","mapping_epoch":35,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29499392,"num_objects":7202,"num_object_clones":0,"num_object_copies":14404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10200,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13425","reported_seq":21318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:30.067370+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:30.067370+0000","last_peered":"2024-09-10T15:58:30.067370+0000","last_clean":"2024-09-10T15:58:30.067370+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:30.067370+0000","last_undegraded":"2024-09-10T15:58:30.067370+0000","last_fullsized":"2024-09-10T15:58:30.067370+0000","mapping_epoch":47,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30482432,"num_objects":7442,"num_object_clones":0,"num_object_copies":14884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10143,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13120","reported_seq":13741,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.935350+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:29.935350+0000","last_peered":"2024-09-10T15:58:29.935350+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:29.935350+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29351936,"num_objects":7166,"num_object_clones":0,"num_object_copies":14332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9770,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3570},{"shards":"0,2","objects":860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13492","reported_seq":13628,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:29.912977+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:29.912977+0000","last_peered":"2024-09-10T15:58:29.912977+0000","last_clean":"2024-09-10T15:58:29.912977+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:29.912977+0000","last_undegraded":"2024-09-10T15:58:29.912977+0000","last_fullsized":"2024-09-10T15:58:29.912977+0000","mapping_epoch":35,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30400512,"num_objects":7422,"num_object_clones":0,"num_object_copies":14844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10251,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13491","reported_seq":21623,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:30.066025+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:30.066025+0000","last_peered":"2024-09-10T15:58:30.066025+0000","last_clean":"2024-09-10T15:58:30.066025+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:30.066025+0000","last_undegraded":"2024-09-10T15:58:30.066025+0000","last_fullsized":"2024-09-10T15:58:30.066025+0000","mapping_epoch":45,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29605888,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11468,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13314","reported_seq":14020,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.915344+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:29.915344+0000","last_peered":"2024-09-10T15:58:29.915344+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:29.915344+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29753344,"num_objects":7264,"num_object_clones":0,"num_object_copies":14528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10178,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3688},{"shards":"0,2","objects":842}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13385","reported_seq":14755,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:30.251494+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:30.251494+0000","last_peered":"2024-09-10T15:58:30.251494+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:30.251494+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30310400,"num_objects":7400,"num_object_clones":0,"num_object_copies":14800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2384,"num_objects_misplaced":828,"num_objects_unfound":0,"num_objects_dirty":7400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11359,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":622,"num_bytes_recovered":2031616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":828},{"shards":"2","objects":4188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13218","reported_seq":13551,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:30.171442+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:30.171442+0000","last_peered":"2024-09-10T15:58:30.171442+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:30.171442+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29646848,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11087,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":841},{"shards":"2","objects":3510}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13370","reported_seq":14054,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.880441+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:29.880441+0000","last_peered":"2024-09-10T15:58:29.880441+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:29.880441+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30654464,"num_objects":7484,"num_object_clones":0,"num_object_copies":14968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10562,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13436","reported_seq":14072,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.903502+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:29.903502+0000","last_peered":"2024-09-10T15:58:29.903502+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:29.903502+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29827072,"num_objects":7282,"num_object_clones":0,"num_object_copies":14564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9854,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3599},{"shards":"0,2","objects":844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13374","reported_seq":14046,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.880609+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:29.880609+0000","last_peered":"2024-09-10T15:58:29.880609+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:29.880609+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30277632,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9980,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3779},{"shards":"0,2","objects":888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13697","reported_seq":14320,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.937755+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:29.937755+0000","last_peered":"2024-09-10T15:58:29.937755+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:29.937755+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3600","ondisk_log_start":"37'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314979,"num_objects":7402,"num_object_clones":0,"num_object_copies":14804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3766,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10516,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3639},{"shards":"0,2","objects":861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":480383483,"num_objects":117283,"num_object_clones":0,"num_object_copies":234566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35263,"num_objects_misplaced":828,"num_objects_unfound":0,"num_objects_dirty":117283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":175767,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10557,"num_bytes_recovered":34234368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1116684288,"data_stored":1116607497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160946,"ondisk_log_size":160946,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626497,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634192,"kb_used_data":317944,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93737648,"statfs":{"total":96636764160,"available":95987351552,"internally_reserved":0,"allocated":325574656,"data_stored":325236278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323808942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149780,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":816560,"kb_used_data":415644,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93555280,"statfs":{"total":96636764160,"available":95800606720,"internally_reserved":0,"allocated":425619456,"data_stored":425342958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":410513069},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,3,3,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670016,"kb_used_data":359340,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93701824,"statfs":{"total":96636764160,"available":95950667776,"internally_reserved":0,"allocated":367964160,"data_stored":367683590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,2,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":424108032,"data_stored":424104419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325160960,"data_stored":325095467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":367415296,"data_stored":367407611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:33.585 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:34.793 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:34.796 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:34.814 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-10T15:58:34.540837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":480383502,"num_objects":117285,"num_object_clones":0,"num_object_copies":234570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35263,"num_objects_misplaced":828,"num_objects_unfound":0,"num_objects_dirty":117285,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":175769,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10559,"num_bytes_recovered":34234387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160949,"ondisk_log_size":160949,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2120768,"kb_used_data":1092928,"kb_used_omap":12,"kb_used_meta":1027763,"kb_avail":280994752,"statfs":{"total":289910292480,"available":287738626048,"internally_reserved":0,"allocated":1119158272,"data_stored":1118262826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1052429317},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,3,3,8,5,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17944576,"num_objects":-4381,"num_object_clones":0,"num_object_copies":-8762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-129,"num_objects_misplaced":-272,"num_objects_unfound":0,"num_objects_dirty":-4381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4381,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":966656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004606"},"pg_stats":[{"pgid":"2.f","version":"48'13474","reported_seq":14149,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.940637+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:29.940637+0000","last_peered":"2024-09-10T15:58:29.940637+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:29.940637+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13474,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3620},{"shards":"0,2","objects":884}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13641","reported_seq":13753,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:30.183627+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:30.183627+0000","last_peered":"2024-09-10T15:58:30.183627+0000","last_clean":"2024-09-10T15:58:30.183627+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:30.183627+0000","last_undegraded":"2024-09-10T15:58:30.183627+0000","last_fullsized":"2024-09-10T15:58:30.183627+0000","mapping_epoch":36,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445568,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11977,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13293","reported_seq":13684,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:30.243680+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:30.243680+0000","last_peered":"2024-09-10T15:58:30.243680+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:30.243680+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30048280,"num_objects":7337,"num_object_clones":0,"num_object_copies":14674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11561,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":824},{"shards":"2","objects":3666}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13387","reported_seq":13471,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:30.246735+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:30.246735+0000","last_peered":"2024-09-10T15:58:30.246735+0000","last_clean":"2024-09-10T15:58:30.246735+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:30.246735+0000","last_undegraded":"2024-09-10T15:58:30.246735+0000","last_fullsized":"2024-09-10T15:58:30.246735+0000","mapping_epoch":36,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29880320,"num_objects":7295,"num_object_clones":0,"num_object_copies":14590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13387,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13129","reported_seq":13260,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:29.940263+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:29.940263+0000","last_peered":"2024-09-10T15:58:29.940263+0000","last_clean":"2024-09-10T15:58:29.940263+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:29.940263+0000","last_undegraded":"2024-09-10T15:58:29.940263+0000","last_fullsized":"2024-09-10T15:58:29.940263+0000","mapping_epoch":35,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29499392,"num_objects":7202,"num_object_clones":0,"num_object_copies":14404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10200,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13425","reported_seq":21318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:30.067370+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:30.067370+0000","last_peered":"2024-09-10T15:58:30.067370+0000","last_clean":"2024-09-10T15:58:30.067370+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:30.067370+0000","last_undegraded":"2024-09-10T15:58:30.067370+0000","last_fullsized":"2024-09-10T15:58:30.067370+0000","mapping_epoch":47,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30482432,"num_objects":7442,"num_object_clones":0,"num_object_copies":14884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10143,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13120","reported_seq":13741,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.935350+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:29.935350+0000","last_peered":"2024-09-10T15:58:29.935350+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:29.935350+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29351936,"num_objects":7166,"num_object_clones":0,"num_object_copies":14332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9770,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3570},{"shards":"0,2","objects":860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13492","reported_seq":13628,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:29.912977+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:29.912977+0000","last_peered":"2024-09-10T15:58:29.912977+0000","last_clean":"2024-09-10T15:58:29.912977+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:29.912977+0000","last_undegraded":"2024-09-10T15:58:29.912977+0000","last_fullsized":"2024-09-10T15:58:29.912977+0000","mapping_epoch":35,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30400512,"num_objects":7422,"num_object_clones":0,"num_object_copies":14844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10251,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13491","reported_seq":21623,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:30.066025+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:30.066025+0000","last_peered":"2024-09-10T15:58:30.066025+0000","last_clean":"2024-09-10T15:58:30.066025+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:30.066025+0000","last_undegraded":"2024-09-10T15:58:30.066025+0000","last_fullsized":"2024-09-10T15:58:30.066025+0000","mapping_epoch":45,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29605888,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11468,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13314","reported_seq":14020,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.915344+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:29.915344+0000","last_peered":"2024-09-10T15:58:29.915344+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:29.915344+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29753344,"num_objects":7264,"num_object_clones":0,"num_object_copies":14528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10178,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3688},{"shards":"0,2","objects":842}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13385","reported_seq":14755,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:30.251494+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:30.251494+0000","last_peered":"2024-09-10T15:58:30.251494+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:30.251494+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30310400,"num_objects":7400,"num_object_clones":0,"num_object_copies":14800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2384,"num_objects_misplaced":828,"num_objects_unfound":0,"num_objects_dirty":7400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11359,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":622,"num_bytes_recovered":2031616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":828},{"shards":"2","objects":4188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13218","reported_seq":13551,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:30.171442+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:30.171442+0000","last_peered":"2024-09-10T15:58:30.171442+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:30.171442+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29646848,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11087,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":841},{"shards":"2","objects":3510}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13370","reported_seq":14054,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.880441+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:29.880441+0000","last_peered":"2024-09-10T15:58:29.880441+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:29.880441+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30654464,"num_objects":7484,"num_object_clones":0,"num_object_copies":14968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10562,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13436","reported_seq":14072,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.903502+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:29.903502+0000","last_peered":"2024-09-10T15:58:29.903502+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:29.903502+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29827072,"num_objects":7282,"num_object_clones":0,"num_object_copies":14564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9854,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3599},{"shards":"0,2","objects":844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13374","reported_seq":14046,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.880609+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:29.880609+0000","last_peered":"2024-09-10T15:58:29.880609+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:29.880609+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30277632,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9980,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3779},{"shards":"0,2","objects":888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13697","reported_seq":14320,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.937755+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:29.937755+0000","last_peered":"2024-09-10T15:58:29.937755+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:29.937755+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3600","ondisk_log_start":"37'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314979,"num_objects":7402,"num_object_clones":0,"num_object_copies":14804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3766,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10516,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3639},{"shards":"0,2","objects":861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":480383483,"num_objects":117283,"num_object_clones":0,"num_object_copies":234566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35263,"num_objects_misplaced":828,"num_objects_unfound":0,"num_objects_dirty":117283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":175767,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10557,"num_bytes_recovered":34234368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1116684288,"data_stored":1116607497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160946,"ondisk_log_size":160946,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626497,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634192,"kb_used_data":317944,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93737648,"statfs":{"total":96636764160,"available":95987351552,"internally_reserved":0,"allocated":325574656,"data_stored":325236278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323808942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149780,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":816560,"kb_used_data":415644,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93555280,"statfs":{"total":96636764160,"available":95800606720,"internally_reserved":0,"allocated":425619456,"data_stored":425342958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":410513069},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,3,3,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670016,"kb_used_data":359340,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93701824,"statfs":{"total":96636764160,"available":95950667776,"internally_reserved":0,"allocated":367964160,"data_stored":367683590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,2,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":424108032,"data_stored":424104419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325160960,"data_stored":325095467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":367415296,"data_stored":367407611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:34.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:58:34.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'13474", 'reported_seq': 14149, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:29.940637+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:58:29.940637+0000', 'last_peered': '2024-09-10T15:58:29.940637+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:58:29.940637+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "37'3400", 'ondisk_log_start': "37'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29884416, 'num_objects': 7296, 'num_object_clones': 0, 'num_object_copies': 14592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3676, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13474, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 297, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3620}, {'shards': '0,2', 'objects': 884}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:34.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:58:34.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'13293", 'reported_seq': 13684, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:30.243680+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:58:30.243680+0000', 'last_peered': '2024-09-10T15:58:30.243680+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:58:30.243680+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'3200", 'ondisk_log_start': "37'3200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30048280, 'num_objects': 7337, 'num_object_clones': 0, 'num_object_copies': 14674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3672, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7337, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11561, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 131, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 824}, {'shards': '2', 'objects': 3666}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:34.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:58:34.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'13120", 'reported_seq': 13741, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:29.935350+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:58:29.935350+0000', 'last_peered': '2024-09-10T15:58:29.935350+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:58:29.935350+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'3100", 'ondisk_log_start': "37'3100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29351936, 'num_objects': 7166, 'num_object_clones': 0, 'num_object_copies': 14332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3598, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7166, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9770, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 239, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3570}, {'shards': '0,2', 'objects': 860}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:34.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:58:34.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'13314", 'reported_seq': 14020, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:29.915344+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:58:29.915344+0000', 'last_peered': '2024-09-10T15:58:29.915344+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:58:29.915344+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'3300", 'ondisk_log_start': "37'3300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29753344, 'num_objects': 7264, 'num_object_clones': 0, 'num_object_copies': 14528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3578, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7264, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10178, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 287, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3688}, {'shards': '0,2', 'objects': 842}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:34.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:58:34.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'13385", 'reported_seq': 14755, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:30.251494+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:58:30.251494+0000', 'last_peered': '2024-09-10T15:58:30.251494+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:58:30.251494+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "37'3300", 'ondisk_log_start': "37'3300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30310400, 'num_objects': 7400, 'num_object_clones': 0, 'num_object_copies': 14800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2384, 'num_objects_misplaced': 828, 'num_objects_unfound': 0, 'num_objects_dirty': 7400, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11359, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 622, 'num_bytes_recovered': 2031616, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 828}, {'shards': '2', 'objects': 4188}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:34.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:58:34.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'13218", 'reported_seq': 13551, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:30.171442+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:58:30.171442+0000', 'last_peered': '2024-09-10T15:58:30.171442+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:58:30.171442+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'3200", 'ondisk_log_start': "37'3200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29646848, 'num_objects': 7238, 'num_object_clones': 0, 'num_object_copies': 14476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3730, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11087, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 101, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 841}, {'shards': '2', 'objects': 3510}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:34.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:58:34.818 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'13370", 'reported_seq': 14054, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:29.880441+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:58:29.880441+0000', 'last_peered': '2024-09-10T15:58:29.880441+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:58:29.880441+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "37'3300", 'ondisk_log_start': "37'3300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30654464, 'num_objects': 7484, 'num_object_clones': 0, 'num_object_copies': 14968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3558, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7484, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10562, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 276, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3927}, {'shards': '0,2', 'objects': 891}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:34.818 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:58:34.818 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'13436", 'reported_seq': 14072, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:29.903502+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:58:29.903502+0000', 'last_peered': '2024-09-10T15:58:29.903502+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:58:29.903502+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'3400", 'ondisk_log_start': "37'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29827072, 'num_objects': 7282, 'num_object_clones': 0, 'num_object_copies': 14564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3686, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7282, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9854, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 234, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3599}, {'shards': '0,2', 'objects': 844}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:34.818 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:58:34.818 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'13374", 'reported_seq': 14046, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:29.880609+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:58:29.880609+0000', 'last_peered': '2024-09-10T15:58:29.880609+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:58:29.880609+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "37'3300", 'ondisk_log_start': "37'3300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30277632, 'num_objects': 7392, 'num_object_clones': 0, 'num_object_copies': 14784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3615, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9980, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 265, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3779}, {'shards': '0,2', 'objects': 888}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:34.818 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:58:34.818 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'13697", 'reported_seq': 14320, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:29.937755+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:58:29.937755+0000', 'last_peered': '2024-09-10T15:58:29.937755+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:58:29.937755+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "37'3600", 'ondisk_log_start': "37'3600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30314979, 'num_objects': 7402, 'num_object_clones': 0, 'num_object_copies': 14804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3766, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7402, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10516, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 240, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3639}, {'shards': '0,2', 'objects': 861}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:34.819 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:35.511 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:35.511 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35263/234570 objects degraded (15.033%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":441,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":117285,"data_bytes":480383502,"bytes_used":2171666432,"bytes_avail":287738626048,"bytes_total":289910292480,"degraded_objects":35263,"degraded_total":234570,"degraded_ratio":0.15033039178070512,"misplaced_objects":828,"misplaced_total":234570,"misplaced_ratio":0.003529863153855992,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":80523,"recovering_keys_per_sec":0,"num_objects_recovered":401,"num_bytes_recovered":966656,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":364},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:58:35.541 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:58:35.541 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:36.070 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:36.070 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:36.101 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-10T15:58:34.540837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":480383502,"num_objects":117285,"num_object_clones":0,"num_object_copies":234570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35263,"num_objects_misplaced":828,"num_objects_unfound":0,"num_objects_dirty":117285,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":175769,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10559,"num_bytes_recovered":34234387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160949,"ondisk_log_size":160949,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2120768,"kb_used_data":1092928,"kb_used_omap":12,"kb_used_meta":1027763,"kb_avail":280994752,"statfs":{"total":289910292480,"available":287738626048,"internally_reserved":0,"allocated":1119158272,"data_stored":1118262826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1052429317},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,3,3,8,5,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17944576,"num_objects":-4381,"num_object_clones":0,"num_object_copies":-8762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-129,"num_objects_misplaced":-272,"num_objects_unfound":0,"num_objects_dirty":-4381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4381,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":966656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004606"},"pg_stats":[{"pgid":"2.f","version":"48'13474","reported_seq":14149,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.940637+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:29.940637+0000","last_peered":"2024-09-10T15:58:29.940637+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:29.940637+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13474,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3620},{"shards":"0,2","objects":884}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13641","reported_seq":13753,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:30.183627+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:30.183627+0000","last_peered":"2024-09-10T15:58:30.183627+0000","last_clean":"2024-09-10T15:58:30.183627+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:30.183627+0000","last_undegraded":"2024-09-10T15:58:30.183627+0000","last_fullsized":"2024-09-10T15:58:30.183627+0000","mapping_epoch":36,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445568,"num_objects":7433,"num_object_clones":0,"num_object_copies":14866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11977,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13293","reported_seq":13684,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:30.243680+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:30.243680+0000","last_peered":"2024-09-10T15:58:30.243680+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:30.243680+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30048280,"num_objects":7337,"num_object_clones":0,"num_object_copies":14674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11561,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":824},{"shards":"2","objects":3666}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13387","reported_seq":13471,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:30.246735+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:30.246735+0000","last_peered":"2024-09-10T15:58:30.246735+0000","last_clean":"2024-09-10T15:58:30.246735+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:30.246735+0000","last_undegraded":"2024-09-10T15:58:30.246735+0000","last_fullsized":"2024-09-10T15:58:30.246735+0000","mapping_epoch":36,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29880320,"num_objects":7295,"num_object_clones":0,"num_object_copies":14590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13387,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13129","reported_seq":13260,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:29.940263+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:29.940263+0000","last_peered":"2024-09-10T15:58:29.940263+0000","last_clean":"2024-09-10T15:58:29.940263+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:29.940263+0000","last_undegraded":"2024-09-10T15:58:29.940263+0000","last_fullsized":"2024-09-10T15:58:29.940263+0000","mapping_epoch":35,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29499392,"num_objects":7202,"num_object_clones":0,"num_object_copies":14404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10200,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13425","reported_seq":21318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:30.067370+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:30.067370+0000","last_peered":"2024-09-10T15:58:30.067370+0000","last_clean":"2024-09-10T15:58:30.067370+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:30.067370+0000","last_undegraded":"2024-09-10T15:58:30.067370+0000","last_fullsized":"2024-09-10T15:58:30.067370+0000","mapping_epoch":47,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30482432,"num_objects":7442,"num_object_clones":0,"num_object_copies":14884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10143,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13120","reported_seq":13741,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.935350+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:29.935350+0000","last_peered":"2024-09-10T15:58:29.935350+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:29.935350+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'3100","ondisk_log_start":"37'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29351936,"num_objects":7166,"num_object_clones":0,"num_object_copies":14332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9770,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3570},{"shards":"0,2","objects":860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13492","reported_seq":13628,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:29.912977+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:29.912977+0000","last_peered":"2024-09-10T15:58:29.912977+0000","last_clean":"2024-09-10T15:58:29.912977+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:29.912977+0000","last_undegraded":"2024-09-10T15:58:29.912977+0000","last_fullsized":"2024-09-10T15:58:29.912977+0000","mapping_epoch":35,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30400512,"num_objects":7422,"num_object_clones":0,"num_object_copies":14844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10251,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13491","reported_seq":21623,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:30.066025+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:30.066025+0000","last_peered":"2024-09-10T15:58:30.066025+0000","last_clean":"2024-09-10T15:58:30.066025+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:30.066025+0000","last_undegraded":"2024-09-10T15:58:30.066025+0000","last_fullsized":"2024-09-10T15:58:30.066025+0000","mapping_epoch":45,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29605888,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11468,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13314","reported_seq":14020,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.915344+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:29.915344+0000","last_peered":"2024-09-10T15:58:29.915344+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:29.915344+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29753344,"num_objects":7264,"num_object_clones":0,"num_object_copies":14528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3578,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10178,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3688},{"shards":"0,2","objects":842}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13385","reported_seq":14755,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:30.251494+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:30.251494+0000","last_peered":"2024-09-10T15:58:30.251494+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:30.251494+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30310400,"num_objects":7400,"num_object_clones":0,"num_object_copies":14800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2384,"num_objects_misplaced":828,"num_objects_unfound":0,"num_objects_dirty":7400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11359,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":622,"num_bytes_recovered":2031616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":828},{"shards":"2","objects":4188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13218","reported_seq":13551,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:30.171442+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:30.171442+0000","last_peered":"2024-09-10T15:58:30.171442+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:30.171442+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29646848,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11087,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":841},{"shards":"2","objects":3510}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13370","reported_seq":14054,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.880441+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:29.880441+0000","last_peered":"2024-09-10T15:58:29.880441+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:29.880441+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30654464,"num_objects":7484,"num_object_clones":0,"num_object_copies":14968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10562,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3927},{"shards":"0,2","objects":891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13436","reported_seq":14072,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.903502+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:29.903502+0000","last_peered":"2024-09-10T15:58:29.903502+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:29.903502+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29827072,"num_objects":7282,"num_object_clones":0,"num_object_copies":14564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9854,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3599},{"shards":"0,2","objects":844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13374","reported_seq":14046,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.880609+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:29.880609+0000","last_peered":"2024-09-10T15:58:29.880609+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:29.880609+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30277632,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9980,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3779},{"shards":"0,2","objects":888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13697","reported_seq":14320,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:29.937755+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:29.937755+0000","last_peered":"2024-09-10T15:58:29.937755+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:29.937755+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"37'3600","ondisk_log_start":"37'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30314979,"num_objects":7402,"num_object_clones":0,"num_object_copies":14804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3766,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10516,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3639},{"shards":"0,2","objects":861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":480383483,"num_objects":117283,"num_object_clones":0,"num_object_copies":234566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35263,"num_objects_misplaced":828,"num_objects_unfound":0,"num_objects_dirty":117283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":175767,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10557,"num_bytes_recovered":34234368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1116684288,"data_stored":1116607497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160946,"ondisk_log_size":160946,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626497,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634192,"kb_used_data":317944,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93737648,"statfs":{"total":96636764160,"available":95987351552,"internally_reserved":0,"allocated":325574656,"data_stored":325236278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323808942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149780,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":816560,"kb_used_data":415644,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93555280,"statfs":{"total":96636764160,"available":95800606720,"internally_reserved":0,"allocated":425619456,"data_stored":425342958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":410513069},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,3,3,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670016,"kb_used_data":359340,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93701824,"statfs":{"total":96636764160,"available":95950667776,"internally_reserved":0,"allocated":367964160,"data_stored":367683590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,2,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":424108032,"data_stored":424104419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325160960,"data_stored":325095467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":367415296,"data_stored":367407611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:39.105 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:39.658 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:39.659 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:39.681 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-09-10T15:58:38.542596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471237134,"num_objects":115052,"num_object_clones":0,"num_object_copies":230104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35152,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":115052,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178002,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10813,"num_bytes_recovered":34820115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160682,"ondisk_log_size":160682,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087864,"kb_used_data":1080836,"kb_used_omap":12,"kb_used_meta":1006963,"kb_avail":281027656,"statfs":{"total":289910292480,"available":287772319744,"internally_reserved":0,"allocated":1106776064,"data_stored":1105880618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1031130117},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,16,2,8,16,1,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18079744,"num_objects":-4414,"num_object_clones":0,"num_object_copies":-8828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-153,"num_objects_misplaced":-305,"num_objects_unfound":0,"num_objects_dirty":-4414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4414,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005026"},"pg_stats":[{"pgid":"2.f","version":"48'13616","reported_seq":14327,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:34.955407+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:34.955407+0000","last_peered":"2024-09-10T15:58:34.955407+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:34.955407+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29302784,"num_objects":7154,"num_object_clones":0,"num_object_copies":14308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13616,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3496},{"shards":"0,2","objects":742}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13770","reported_seq":13882,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:35.258211+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:35.258211+0000","last_peered":"2024-09-10T15:58:35.258211+0000","last_clean":"2024-09-10T15:58:35.258211+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:35.258211+0000","last_undegraded":"2024-09-10T15:58:35.258211+0000","last_fullsized":"2024-09-10T15:58:35.258211+0000","mapping_epoch":36,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29917184,"num_objects":7304,"num_object_clones":0,"num_object_copies":14608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12106,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13414","reported_seq":13816,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:35.287735+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:35.287735+0000","last_peered":"2024-09-10T15:58:35.287735+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:35.287735+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29552664,"num_objects":7216,"num_object_clones":0,"num_object_copies":14432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11682,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":703},{"shards":"2","objects":3552}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13522","reported_seq":13606,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:35.189086+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:35.189086+0000","last_peered":"2024-09-10T15:58:35.189086+0000","last_clean":"2024-09-10T15:58:35.189086+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:35.189086+0000","last_undegraded":"2024-09-10T15:58:35.189086+0000","last_fullsized":"2024-09-10T15:58:35.189086+0000","mapping_epoch":36,"log_start":"37'3500","ondisk_log_start":"37'3500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29327360,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13522,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13283","reported_seq":13414,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:35.016670+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:35.016670+0000","last_peered":"2024-09-10T15:58:35.016670+0000","last_clean":"2024-09-10T15:58:35.016670+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:35.016670+0000","last_undegraded":"2024-09-10T15:58:35.016670+0000","last_fullsized":"2024-09-10T15:58:35.016670+0000","mapping_epoch":35,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28868608,"num_objects":7048,"num_object_clones":0,"num_object_copies":14096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10354,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13576","reported_seq":21469,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:35.015088+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:35.015088+0000","last_peered":"2024-09-10T15:58:35.015088+0000","last_clean":"2024-09-10T15:58:35.015088+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:35.015088+0000","last_undegraded":"2024-09-10T15:58:35.015088+0000","last_fullsized":"2024-09-10T15:58:35.015088+0000","mapping_epoch":47,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10294,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13251","reported_seq":13892,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:35.023639+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:35.023639+0000","last_peered":"2024-09-10T15:58:35.023639+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:35.023639+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28815360,"num_objects":7035,"num_object_clones":0,"num_object_copies":14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9901,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3449},{"shards":"0,2","objects":729}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13642","reported_seq":13777,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:35.022214+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:35.022214+0000","last_peered":"2024-09-10T15:58:35.022214+0000","last_clean":"2024-09-10T15:58:35.022214+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:35.022214+0000","last_undegraded":"2024-09-10T15:58:35.022214+0000","last_fullsized":"2024-09-10T15:58:35.022214+0000","mapping_epoch":35,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29786112,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10401,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13619","reported_seq":21751,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:35.111064+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:35.111064+0000","last_peered":"2024-09-10T15:58:35.111064+0000","last_clean":"2024-09-10T15:58:35.111064+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:35.111064+0000","last_undegraded":"2024-09-10T15:58:35.111064+0000","last_fullsized":"2024-09-10T15:58:35.111064+0000","mapping_epoch":45,"log_start":"37'3600","ondisk_log_start":"37'3600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29081600,"num_objects":7100,"num_object_clones":0,"num_object_copies":14200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11596,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13458","reported_seq":14182,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:34.993734+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:34.993734+0000","last_peered":"2024-09-10T15:58:34.993734+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:34.993734+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":7120,"num_object_clones":0,"num_object_copies":14240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10322,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3553},{"shards":"0,2","objects":698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13526","reported_seq":15186,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:35.275190+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:35.275190+0000","last_peered":"2024-09-10T15:58:35.275190+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:35.275190+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29732864,"num_objects":7259,"num_object_clones":0,"num_object_copies":14518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2384,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":7259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11500,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":765,"num_bytes_recovered":2617344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":685},{"shards":"2","objects":4190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13362","reported_seq":13708,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:35.232832+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:35.232832+0000","last_peered":"2024-09-10T15:58:35.232832+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:35.232832+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29057024,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3723,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11231,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":697},{"shards":"2","objects":3373}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13526","reported_seq":14254,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:34.985424+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:34.985424+0000","last_peered":"2024-09-10T15:58:34.985424+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:34.985424+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30015488,"num_objects":7328,"num_object_clones":0,"num_object_copies":14656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3536,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10718,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3793},{"shards":"0,2","objects":735}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13564","reported_seq":14227,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:34.965615+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:34.965615+0000","last_peered":"2024-09-10T15:58:34.965615+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:34.965615+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'3500","ondisk_log_start":"37'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29302784,"num_objects":7154,"num_object_clones":0,"num_object_copies":14308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9982,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3485},{"shards":"0,2","objects":716}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13515","reported_seq":14211,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:34.898422+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:34.898422+0000","last_peered":"2024-09-10T15:58:34.898422+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:34.898422+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29700096,"num_objects":7251,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10121,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3650},{"shards":"0,2","objects":747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13835","reported_seq":14480,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:35.028313+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:35.028313+0000","last_peered":"2024-09-10T15:58:35.028313+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:35.028313+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"40'3800","ondisk_log_start":"40'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29749731,"num_objects":7264,"num_object_clones":0,"num_object_copies":14528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10654,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3513},{"shards":"0,2","objects":723}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471237115,"num_objects":115050,"num_object_clones":0,"num_object_copies":230100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35152,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":115050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178000,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10811,"num_bytes_recovered":34820096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1104064512,"data_stored":1103987721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626498,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637236,"kb_used_data":316836,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93734604,"statfs":{"total":96636764160,"available":95984234496,"internally_reserved":0,"allocated":324440064,"data_stored":324101686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328068782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149781,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":797496,"kb_used_data":409064,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93574344,"statfs":{"total":96636764160,"available":95820128256,"internally_reserved":0,"allocated":418881536,"data_stored":418605038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":397733549},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,16,0,2,13,1,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653132,"kb_used_data":354936,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93718708,"statfs":{"total":96636764160,"available":95967956992,"internally_reserved":0,"allocated":363454464,"data_stored":363173894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":417153024,"data_stored":417149411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323997696,"data_stored":323932203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":362913792,"data_stored":362906107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:39.682 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:40.219 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:40.220 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:40.239 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-09-10T15:58:38.542596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471237134,"num_objects":115052,"num_object_clones":0,"num_object_copies":230104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35152,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":115052,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178002,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10813,"num_bytes_recovered":34820115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160682,"ondisk_log_size":160682,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087864,"kb_used_data":1080836,"kb_used_omap":12,"kb_used_meta":1006963,"kb_avail":281027656,"statfs":{"total":289910292480,"available":287772319744,"internally_reserved":0,"allocated":1106776064,"data_stored":1105880618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1031130117},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,16,2,8,16,1,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18079744,"num_objects":-4414,"num_object_clones":0,"num_object_copies":-8828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-153,"num_objects_misplaced":-305,"num_objects_unfound":0,"num_objects_dirty":-4414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4414,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005026"},"pg_stats":[{"pgid":"2.f","version":"48'13616","reported_seq":14327,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:34.955407+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:34.955407+0000","last_peered":"2024-09-10T15:58:34.955407+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:34.955407+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29302784,"num_objects":7154,"num_object_clones":0,"num_object_copies":14308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13616,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3496},{"shards":"0,2","objects":742}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13770","reported_seq":13882,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:35.258211+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:35.258211+0000","last_peered":"2024-09-10T15:58:35.258211+0000","last_clean":"2024-09-10T15:58:35.258211+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:35.258211+0000","last_undegraded":"2024-09-10T15:58:35.258211+0000","last_fullsized":"2024-09-10T15:58:35.258211+0000","mapping_epoch":36,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29917184,"num_objects":7304,"num_object_clones":0,"num_object_copies":14608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12106,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13414","reported_seq":13816,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:35.287735+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:35.287735+0000","last_peered":"2024-09-10T15:58:35.287735+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:35.287735+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29552664,"num_objects":7216,"num_object_clones":0,"num_object_copies":14432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11682,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":703},{"shards":"2","objects":3552}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13522","reported_seq":13606,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:35.189086+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:35.189086+0000","last_peered":"2024-09-10T15:58:35.189086+0000","last_clean":"2024-09-10T15:58:35.189086+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:35.189086+0000","last_undegraded":"2024-09-10T15:58:35.189086+0000","last_fullsized":"2024-09-10T15:58:35.189086+0000","mapping_epoch":36,"log_start":"37'3500","ondisk_log_start":"37'3500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29327360,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13522,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13283","reported_seq":13414,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:35.016670+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:35.016670+0000","last_peered":"2024-09-10T15:58:35.016670+0000","last_clean":"2024-09-10T15:58:35.016670+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:35.016670+0000","last_undegraded":"2024-09-10T15:58:35.016670+0000","last_fullsized":"2024-09-10T15:58:35.016670+0000","mapping_epoch":35,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28868608,"num_objects":7048,"num_object_clones":0,"num_object_copies":14096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10354,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13576","reported_seq":21469,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:35.015088+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:35.015088+0000","last_peered":"2024-09-10T15:58:35.015088+0000","last_clean":"2024-09-10T15:58:35.015088+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:35.015088+0000","last_undegraded":"2024-09-10T15:58:35.015088+0000","last_fullsized":"2024-09-10T15:58:35.015088+0000","mapping_epoch":47,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10294,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13251","reported_seq":13892,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:35.023639+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:35.023639+0000","last_peered":"2024-09-10T15:58:35.023639+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:35.023639+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'3200","ondisk_log_start":"37'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28815360,"num_objects":7035,"num_object_clones":0,"num_object_copies":14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9901,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3449},{"shards":"0,2","objects":729}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13642","reported_seq":13777,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:35.022214+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:35.022214+0000","last_peered":"2024-09-10T15:58:35.022214+0000","last_clean":"2024-09-10T15:58:35.022214+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:35.022214+0000","last_undegraded":"2024-09-10T15:58:35.022214+0000","last_fullsized":"2024-09-10T15:58:35.022214+0000","mapping_epoch":35,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29786112,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10401,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13619","reported_seq":21751,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:35.111064+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:35.111064+0000","last_peered":"2024-09-10T15:58:35.111064+0000","last_clean":"2024-09-10T15:58:35.111064+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:35.111064+0000","last_undegraded":"2024-09-10T15:58:35.111064+0000","last_fullsized":"2024-09-10T15:58:35.111064+0000","mapping_epoch":45,"log_start":"37'3600","ondisk_log_start":"37'3600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29081600,"num_objects":7100,"num_object_clones":0,"num_object_copies":14200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11596,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747985+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.747985+0000","last_peered":"2024-09-10T15:58:12.747985+0000","last_clean":"2024-09-10T15:58:12.747985+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.747985+0000","last_undegraded":"2024-09-10T15:58:12.747985+0000","last_fullsized":"2024-09-10T15:58:12.747985+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13458","reported_seq":14182,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:34.993734+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:34.993734+0000","last_peered":"2024-09-10T15:58:34.993734+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:34.993734+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":7120,"num_object_clones":0,"num_object_copies":14240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10322,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3553},{"shards":"0,2","objects":698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13526","reported_seq":15186,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:35.275190+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:35.275190+0000","last_peered":"2024-09-10T15:58:35.275190+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:35.275190+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29732864,"num_objects":7259,"num_object_clones":0,"num_object_copies":14518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2384,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":7259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11500,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":765,"num_bytes_recovered":2617344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":685},{"shards":"2","objects":4190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13362","reported_seq":13708,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:35.232832+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:35.232832+0000","last_peered":"2024-09-10T15:58:35.232832+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:35.232832+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'3300","ondisk_log_start":"37'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29057024,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3723,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11231,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":697},{"shards":"2","objects":3373}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13526","reported_seq":14254,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:34.985424+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:34.985424+0000","last_peered":"2024-09-10T15:58:34.985424+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:34.985424+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30015488,"num_objects":7328,"num_object_clones":0,"num_object_copies":14656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3536,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10718,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3793},{"shards":"0,2","objects":735}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13564","reported_seq":14227,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:34.965615+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:34.965615+0000","last_peered":"2024-09-10T15:58:34.965615+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:34.965615+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"37'3500","ondisk_log_start":"37'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29302784,"num_objects":7154,"num_object_clones":0,"num_object_copies":14308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9982,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3485},{"shards":"0,2","objects":716}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747668+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747668+0000","last_peered":"2024-09-10T15:58:12.747668+0000","last_clean":"2024-09-10T15:58:12.747668+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747668+0000","last_undegraded":"2024-09-10T15:58:12.747668+0000","last_fullsized":"2024-09-10T15:58:12.747668+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13515","reported_seq":14211,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:34.898422+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:34.898422+0000","last_peered":"2024-09-10T15:58:34.898422+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:34.898422+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29700096,"num_objects":7251,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10121,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3650},{"shards":"0,2","objects":747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13835","reported_seq":14480,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:35.028313+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:35.028313+0000","last_peered":"2024-09-10T15:58:35.028313+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:35.028313+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"40'3800","ondisk_log_start":"40'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29749731,"num_objects":7264,"num_object_clones":0,"num_object_copies":14528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10654,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3513},{"shards":"0,2","objects":723}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471237115,"num_objects":115050,"num_object_clones":0,"num_object_copies":230100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35152,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":115050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178000,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10811,"num_bytes_recovered":34820096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1104064512,"data_stored":1103987721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160679,"ondisk_log_size":160679,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626498,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637236,"kb_used_data":316836,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93734604,"statfs":{"total":96636764160,"available":95984234496,"internally_reserved":0,"allocated":324440064,"data_stored":324101686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328068782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149781,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":797496,"kb_used_data":409064,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93574344,"statfs":{"total":96636764160,"available":95820128256,"internally_reserved":0,"allocated":418881536,"data_stored":418605038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":397733549},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,16,0,2,13,1,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653132,"kb_used_data":354936,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93718708,"statfs":{"total":96636764160,"available":95967956992,"internally_reserved":0,"allocated":363454464,"data_stored":363173894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":417153024,"data_stored":417149411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323997696,"data_stored":323932203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":362913792,"data_stored":362906107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:40.240 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:58:40.240 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'13616", 'reported_seq': 14327, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:34.955407+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:58:34.955407+0000', 'last_peered': '2024-09-10T15:58:34.955407+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:58:34.955407+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "40'3600", 'ondisk_log_start': "40'3600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29302784, 'num_objects': 7154, 'num_object_clones': 0, 'num_object_copies': 14308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3658, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13616, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 315, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3496}, {'shards': '0,2', 'objects': 742}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:40.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:58:40.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'13414", 'reported_seq': 13816, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:35.287735+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:58:35.287735+0000', 'last_peered': '2024-09-10T15:58:35.287735+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:58:35.287735+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "37'3400", 'ondisk_log_start': "37'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29552664, 'num_objects': 7216, 'num_object_clones': 0, 'num_object_copies': 14432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3665, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7216, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11682, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 138, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 703}, {'shards': '2', 'objects': 3552}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:40.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:58:40.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'13251", 'reported_seq': 13892, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:35.023639+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:58:35.023639+0000', 'last_peered': '2024-09-10T15:58:35.023639+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:58:35.023639+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'3200", 'ondisk_log_start': "37'3200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28815360, 'num_objects': 7035, 'num_object_clones': 0, 'num_object_copies': 14070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3588, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7035, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9901, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 249, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3449}, {'shards': '0,2', 'objects': 729}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:40.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:58:40.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'13458", 'reported_seq': 14182, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:34.993734+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:58:34.993734+0000', 'last_peered': '2024-09-10T15:58:34.993734+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:58:34.993734+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "37'3400", 'ondisk_log_start': "37'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29163520, 'num_objects': 7120, 'num_object_clones': 0, 'num_object_copies': 14240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3569, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7120, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10322, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 296, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3553}, {'shards': '0,2', 'objects': 698}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:40.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:58:40.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'13526", 'reported_seq': 15186, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:35.275190+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:58:35.275190+0000', 'last_peered': '2024-09-10T15:58:35.275190+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:58:35.275190+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "40'3500", 'ondisk_log_start': "40'3500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29732864, 'num_objects': 7259, 'num_object_clones': 0, 'num_object_copies': 14518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2384, 'num_objects_misplaced': 685, 'num_objects_unfound': 0, 'num_objects_dirty': 7259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11500, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 765, 'num_bytes_recovered': 2617344, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 685}, {'shards': '2', 'objects': 4190}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:40.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:58:40.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'13362", 'reported_seq': 13708, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:35.232832+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:58:35.232832+0000', 'last_peered': '2024-09-10T15:58:35.232832+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:58:35.232832+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'3300", 'ondisk_log_start': "37'3300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29057024, 'num_objects': 7094, 'num_object_clones': 0, 'num_object_copies': 14188, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3723, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7094, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11231, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 108, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 697}, {'shards': '2', 'objects': 3373}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:40.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:58:40.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'13526", 'reported_seq': 14254, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:34.985424+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:58:34.985424+0000', 'last_peered': '2024-09-10T15:58:34.985424+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:58:34.985424+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "40'3500", 'ondisk_log_start': "40'3500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30015488, 'num_objects': 7328, 'num_object_clones': 0, 'num_object_copies': 14656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3536, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7328, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10718, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 298, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3793}, {'shards': '0,2', 'objects': 735}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:40.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:58:40.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'13564", 'reported_seq': 14227, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:34.965615+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:58:34.965615+0000', 'last_peered': '2024-09-10T15:58:34.965615+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:58:34.965615+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "37'3500", 'ondisk_log_start': "37'3500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29302784, 'num_objects': 7154, 'num_object_clones': 0, 'num_object_copies': 14308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3672, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9982, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 248, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3485}, {'shards': '0,2', 'objects': 716}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:40.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:58:40.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'13515", 'reported_seq': 14211, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:34.898422+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:58:34.898422+0000', 'last_peered': '2024-09-10T15:58:34.898422+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:58:34.898422+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "40'3500", 'ondisk_log_start': "40'3500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29700096, 'num_objects': 7251, 'num_object_clones': 0, 'num_object_copies': 14502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3603, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7251, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10121, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 277, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3650}, {'shards': '0,2', 'objects': 747}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:40.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:58:40.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'13835", 'reported_seq': 14480, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:35.028313+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:58:35.028313+0000', 'last_peered': '2024-09-10T15:58:35.028313+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:58:35.028313+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "40'3800", 'ondisk_log_start': "40'3800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29749731, 'num_objects': 7264, 'num_object_clones': 0, 'num_object_copies': 14528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3754, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7264, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10654, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 252, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3513}, {'shards': '0,2', 'objects': 723}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:40.243 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:40.913 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:40.913 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35152/230104 objects degraded (15.277%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":446,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":115052,"data_bytes":471237134,"bytes_used":2137972736,"bytes_avail":287772319744,"bytes_total":289910292480,"degraded_objects":35152,"degraded_total":230104,"degraded_ratio":0.15276570594166117,"misplaced_objects":685,"misplaced_total":230104,"misplaced_ratio":0.0029769147863574737,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":92803,"recovering_keys_per_sec":0,"num_objects_recovered":458,"num_bytes_recovered":1114112,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":367},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:58:40.933 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:58:40.934 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:41.492 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:41.493 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:41.519 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-10T15:58:40.543276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":461783566,"num_objects":112744,"num_object_clones":0,"num_object_copies":225488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35110,"num_objects_misplaced":545,"num_objects_unfound":0,"num_objects_dirty":112744,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180310,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10995,"num_bytes_recovered":35336211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2091540,"kb_used_data":1067868,"kb_used_omap":12,"kb_used_meta":1023603,"kb_avail":281023980,"statfs":{"total":289910292480,"available":287768555520,"internally_reserved":0,"allocated":1093496832,"data_stored":1092601386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1048169477},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,4,3,3,5,0,0,0,6,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27533312,"num_objects":-6722,"num_object_clones":0,"num_object_copies":-13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-195,"num_objects_misplaced":-445,"num_objects_unfound":0,"num_objects_dirty":-6722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6722,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":1630208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004843"},"pg_stats":[{"pgid":"2.f","version":"48'13758","reported_seq":14481,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.036173+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:40.036173+0000","last_peered":"2024-09-10T15:58:40.036173+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:40.036173+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28721152,"num_objects":7012,"num_object_clones":0,"num_object_copies":14024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3652,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13758,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3360},{"shards":"0,2","objects":600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13926","reported_seq":14038,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.289146+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:40.289146+0000","last_peered":"2024-09-10T15:58:40.289146+0000","last_clean":"2024-09-10T15:58:40.289146+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:40.289146+0000","last_undegraded":"2024-09-10T15:58:40.289146+0000","last_fullsized":"2024-09-10T15:58:40.289146+0000","mapping_epoch":36,"log_start":"41'3900","ondisk_log_start":"41'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29278208,"num_objects":7148,"num_object_clones":0,"num_object_copies":14296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12262,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13553","reported_seq":13967,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.318262+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:40.318262+0000","last_peered":"2024-09-10T15:58:40.318262+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:40.318262+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28983320,"num_objects":7077,"num_object_clones":0,"num_object_copies":14154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11821,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":564},{"shards":"2","objects":3419}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13671","reported_seq":13755,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.218644+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:40.218644+0000","last_peered":"2024-09-10T15:58:40.218644+0000","last_clean":"2024-09-10T15:58:40.218644+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:40.218644+0000","last_undegraded":"2024-09-10T15:58:40.218644+0000","last_fullsized":"2024-09-10T15:58:40.218644+0000","mapping_epoch":36,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28717056,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13671,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13426","reported_seq":13557,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:39.953861+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:39.953861+0000","last_peered":"2024-09-10T15:58:39.953861+0000","last_clean":"2024-09-10T15:58:39.953861+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:39.953861+0000","last_undegraded":"2024-09-10T15:58:39.953861+0000","last_fullsized":"2024-09-10T15:58:39.953861+0000","mapping_epoch":35,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28282880,"num_objects":6905,"num_object_clones":0,"num_object_copies":13810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10497,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13710","reported_seq":21603,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.173317+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:40.173317+0000","last_peered":"2024-09-10T15:58:40.173317+0000","last_clean":"2024-09-10T15:58:40.173317+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:40.173317+0000","last_undegraded":"2024-09-10T15:58:40.173317+0000","last_fullsized":"2024-09-10T15:58:40.173317+0000","mapping_epoch":47,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10428,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13406","reported_seq":14057,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.015320+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:40.015320+0000","last_peered":"2024-09-10T15:58:40.015320+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:40.015320+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28180480,"num_objects":6880,"num_object_clones":0,"num_object_copies":13760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10056,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3302},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13781","reported_seq":13917,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.008021+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:40.008021+0000","last_peered":"2024-09-10T15:58:40.008021+0000","last_clean":"2024-09-10T15:58:40.008021+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:40.008021+0000","last_undegraded":"2024-09-10T15:58:40.008021+0000","last_fullsized":"2024-09-10T15:58:40.008021+0000","mapping_epoch":35,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10540,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13774","reported_seq":21906,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.159679+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:40.159679+0000","last_peered":"2024-09-10T15:58:40.159679+0000","last_clean":"2024-09-10T15:58:40.159679+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:40.159679+0000","last_undegraded":"2024-09-10T15:58:40.159679+0000","last_fullsized":"2024-09-10T15:58:40.159679+0000","mapping_epoch":45,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28446720,"num_objects":6945,"num_object_clones":0,"num_object_copies":13890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11751,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13614","reported_seq":14352,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.000606+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:40.000606+0000","last_peered":"2024-09-10T15:58:40.000606+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:40.000606+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28524544,"num_objects":6964,"num_object_clones":0,"num_object_copies":13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3562,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10478,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3404},{"shards":"0,2","objects":542}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13682","reported_seq":15596,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.337067+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:40.337067+0000","last_peered":"2024-09-10T15:58:40.337067+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:40.337067+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29093888,"num_objects":7103,"num_object_clones":0,"num_object_copies":14206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2397,"num_objects_misplaced":545,"num_objects_unfound":0,"num_objects_dirty":7103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11656,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":892,"num_bytes_recovered":3133440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":545},{"shards":"2","objects":4161}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13483","reported_seq":13836,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.141924+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:40.141924+0000","last_peered":"2024-09-10T15:58:40.141924+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:40.141924+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28561408,"num_objects":6973,"num_object_clones":0,"num_object_copies":13946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11352,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":576},{"shards":"2","objects":3255}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13659","reported_seq":14395,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:39.960841+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:39.960841+0000","last_peered":"2024-09-10T15:58:39.960841+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:39.960841+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29470720,"num_objects":7195,"num_object_clones":0,"num_object_copies":14390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10851,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3664},{"shards":"0,2","objects":602}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13687","reported_seq":14369,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:39.963855+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:39.963855+0000","last_peered":"2024-09-10T15:58:39.963855+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:39.963855+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28798976,"num_objects":7031,"num_object_clones":0,"num_object_copies":14062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3663,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10105,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3371},{"shards":"0,2","objects":593}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13665","reported_seq":14370,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.032038+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:40.032038+0000","last_peered":"2024-09-10T15:58:40.032038+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:40.032038+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3599,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10271,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3504},{"shards":"0,2","objects":597}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13992","reported_seq":14656,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.009027+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:40.009027+0000","last_peered":"2024-09-10T15:58:40.009027+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:40.009027+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"40'3900","ondisk_log_start":"40'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29106659,"num_objects":7107,"num_object_clones":0,"num_object_copies":14214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3746,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3364},{"shards":"0,2","objects":566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":461783547,"num_objects":112742,"num_object_clones":0,"num_object_copies":225484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35110,"num_objects_misplaced":545,"num_objects_unfound":0,"num_objects_dirty":112742,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180308,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10993,"num_bytes_recovered":35336192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1090973696,"data_stored":1090896905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160887,"ondisk_log_size":160887,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626499,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":644360,"kb_used_data":315636,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93727480,"statfs":{"total":96636764160,"available":95976939520,"internally_reserved":0,"allocated":323211264,"data_stored":322872886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336588462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149782,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794468,"kb_used_data":401872,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93577372,"statfs":{"total":96636764160,"available":95823228928,"internally_reserved":0,"allocated":411516928,"data_stored":411240430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":401993389},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,1,3,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652712,"kb_used_data":350360,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93719128,"statfs":{"total":96636764160,"available":95968387072,"internally_reserved":0,"allocated":358768640,"data_stored":358488070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,0,0,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":410062848,"data_stored":410059235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322760704,"data_stored":322695211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":358150144,"data_stored":358142459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:44.521 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:45.078 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:45.080 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:45.103 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-10T15:58:44.544990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":461783566,"num_objects":112744,"num_object_clones":0,"num_object_copies":225488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35110,"num_objects_misplaced":545,"num_objects_unfound":0,"num_objects_dirty":112744,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180310,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10995,"num_bytes_recovered":35336211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2091540,"kb_used_data":1067868,"kb_used_omap":12,"kb_used_meta":1023603,"kb_avail":281023980,"statfs":{"total":289910292480,"available":287768555520,"internally_reserved":0,"allocated":1093496832,"data_stored":1092601386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1048169477},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,4,3,3,5,0,0,0,6,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18599936,"num_objects":-4541,"num_object_clones":0,"num_object_copies":-9082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-153,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":-4541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4541,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":436,"num_bytes_recovered":1101824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004836"},"pg_stats":[{"pgid":"2.f","version":"48'13758","reported_seq":14481,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.036173+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:40.036173+0000","last_peered":"2024-09-10T15:58:40.036173+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:40.036173+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28721152,"num_objects":7012,"num_object_clones":0,"num_object_copies":14024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3652,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13758,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3360},{"shards":"0,2","objects":600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13926","reported_seq":14038,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.289146+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:40.289146+0000","last_peered":"2024-09-10T15:58:40.289146+0000","last_clean":"2024-09-10T15:58:40.289146+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:40.289146+0000","last_undegraded":"2024-09-10T15:58:40.289146+0000","last_fullsized":"2024-09-10T15:58:40.289146+0000","mapping_epoch":36,"log_start":"41'3900","ondisk_log_start":"41'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29278208,"num_objects":7148,"num_object_clones":0,"num_object_copies":14296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12262,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13553","reported_seq":13967,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.318262+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:40.318262+0000","last_peered":"2024-09-10T15:58:40.318262+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:40.318262+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28983320,"num_objects":7077,"num_object_clones":0,"num_object_copies":14154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11821,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":564},{"shards":"2","objects":3419}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13671","reported_seq":13755,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.218644+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:40.218644+0000","last_peered":"2024-09-10T15:58:40.218644+0000","last_clean":"2024-09-10T15:58:40.218644+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:40.218644+0000","last_undegraded":"2024-09-10T15:58:40.218644+0000","last_fullsized":"2024-09-10T15:58:40.218644+0000","mapping_epoch":36,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28717056,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13671,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13426","reported_seq":13557,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:39.953861+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:39.953861+0000","last_peered":"2024-09-10T15:58:39.953861+0000","last_clean":"2024-09-10T15:58:39.953861+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:39.953861+0000","last_undegraded":"2024-09-10T15:58:39.953861+0000","last_fullsized":"2024-09-10T15:58:39.953861+0000","mapping_epoch":35,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28282880,"num_objects":6905,"num_object_clones":0,"num_object_copies":13810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10497,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13710","reported_seq":21603,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.173317+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:40.173317+0000","last_peered":"2024-09-10T15:58:40.173317+0000","last_clean":"2024-09-10T15:58:40.173317+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:40.173317+0000","last_undegraded":"2024-09-10T15:58:40.173317+0000","last_fullsized":"2024-09-10T15:58:40.173317+0000","mapping_epoch":47,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10428,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13406","reported_seq":14057,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.015320+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:40.015320+0000","last_peered":"2024-09-10T15:58:40.015320+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:40.015320+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28180480,"num_objects":6880,"num_object_clones":0,"num_object_copies":13760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10056,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3302},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13781","reported_seq":13917,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.008021+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:40.008021+0000","last_peered":"2024-09-10T15:58:40.008021+0000","last_clean":"2024-09-10T15:58:40.008021+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:40.008021+0000","last_undegraded":"2024-09-10T15:58:40.008021+0000","last_fullsized":"2024-09-10T15:58:40.008021+0000","mapping_epoch":35,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10540,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13774","reported_seq":21906,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.159679+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:40.159679+0000","last_peered":"2024-09-10T15:58:40.159679+0000","last_clean":"2024-09-10T15:58:40.159679+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:40.159679+0000","last_undegraded":"2024-09-10T15:58:40.159679+0000","last_fullsized":"2024-09-10T15:58:40.159679+0000","mapping_epoch":45,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28446720,"num_objects":6945,"num_object_clones":0,"num_object_copies":13890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11751,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13614","reported_seq":14352,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.000606+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:40.000606+0000","last_peered":"2024-09-10T15:58:40.000606+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:40.000606+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28524544,"num_objects":6964,"num_object_clones":0,"num_object_copies":13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3562,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10478,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3404},{"shards":"0,2","objects":542}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13682","reported_seq":15596,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.337067+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:40.337067+0000","last_peered":"2024-09-10T15:58:40.337067+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:40.337067+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29093888,"num_objects":7103,"num_object_clones":0,"num_object_copies":14206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2397,"num_objects_misplaced":545,"num_objects_unfound":0,"num_objects_dirty":7103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11656,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":892,"num_bytes_recovered":3133440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":545},{"shards":"2","objects":4161}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13483","reported_seq":13836,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.141924+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:40.141924+0000","last_peered":"2024-09-10T15:58:40.141924+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:40.141924+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28561408,"num_objects":6973,"num_object_clones":0,"num_object_copies":13946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11352,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":576},{"shards":"2","objects":3255}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13659","reported_seq":14395,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:39.960841+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:39.960841+0000","last_peered":"2024-09-10T15:58:39.960841+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:39.960841+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29470720,"num_objects":7195,"num_object_clones":0,"num_object_copies":14390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10851,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3664},{"shards":"0,2","objects":602}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13687","reported_seq":14369,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:39.963855+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:39.963855+0000","last_peered":"2024-09-10T15:58:39.963855+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:39.963855+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28798976,"num_objects":7031,"num_object_clones":0,"num_object_copies":14062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3663,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10105,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3371},{"shards":"0,2","objects":593}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13665","reported_seq":14370,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.032038+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:40.032038+0000","last_peered":"2024-09-10T15:58:40.032038+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:40.032038+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3599,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10271,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3504},{"shards":"0,2","objects":597}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13992","reported_seq":14656,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.009027+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:40.009027+0000","last_peered":"2024-09-10T15:58:40.009027+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:40.009027+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"40'3900","ondisk_log_start":"40'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29106659,"num_objects":7107,"num_object_clones":0,"num_object_copies":14214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3746,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3364},{"shards":"0,2","objects":566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":461783547,"num_objects":112742,"num_object_clones":0,"num_object_copies":225484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35110,"num_objects_misplaced":545,"num_objects_unfound":0,"num_objects_dirty":112742,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180308,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10993,"num_bytes_recovered":35336192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1090973696,"data_stored":1090896905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160887,"ondisk_log_size":160887,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626499,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":644360,"kb_used_data":315636,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93727480,"statfs":{"total":96636764160,"available":95976939520,"internally_reserved":0,"allocated":323211264,"data_stored":322872886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336588462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149782,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794468,"kb_used_data":401872,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93577372,"statfs":{"total":96636764160,"available":95823228928,"internally_reserved":0,"allocated":411516928,"data_stored":411240430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":401993389},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,1,3,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652712,"kb_used_data":350360,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93719128,"statfs":{"total":96636764160,"available":95968387072,"internally_reserved":0,"allocated":358768640,"data_stored":358488070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,0,0,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":410062848,"data_stored":410059235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322760704,"data_stored":322695211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":358150144,"data_stored":358142459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:45.104 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:45.626 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:45.626 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:45.648 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-10T15:58:44.544990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":461783566,"num_objects":112744,"num_object_clones":0,"num_object_copies":225488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35110,"num_objects_misplaced":545,"num_objects_unfound":0,"num_objects_dirty":112744,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180310,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10995,"num_bytes_recovered":35336211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2091540,"kb_used_data":1067868,"kb_used_omap":12,"kb_used_meta":1023603,"kb_avail":281023980,"statfs":{"total":289910292480,"available":287768555520,"internally_reserved":0,"allocated":1093496832,"data_stored":1092601386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1048169477},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,4,3,3,5,0,0,0,6,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18599936,"num_objects":-4541,"num_object_clones":0,"num_object_copies":-9082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-153,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":-4541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4541,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":436,"num_bytes_recovered":1101824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004836"},"pg_stats":[{"pgid":"2.f","version":"48'13758","reported_seq":14481,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.036173+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:40.036173+0000","last_peered":"2024-09-10T15:58:40.036173+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:40.036173+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28721152,"num_objects":7012,"num_object_clones":0,"num_object_copies":14024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3652,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13758,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3360},{"shards":"0,2","objects":600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'13926","reported_seq":14038,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.289146+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:40.289146+0000","last_peered":"2024-09-10T15:58:40.289146+0000","last_clean":"2024-09-10T15:58:40.289146+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:40.289146+0000","last_undegraded":"2024-09-10T15:58:40.289146+0000","last_fullsized":"2024-09-10T15:58:40.289146+0000","mapping_epoch":36,"log_start":"41'3900","ondisk_log_start":"41'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29278208,"num_objects":7148,"num_object_clones":0,"num_object_copies":14296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12262,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13553","reported_seq":13967,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.318262+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:40.318262+0000","last_peered":"2024-09-10T15:58:40.318262+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:40.318262+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28983320,"num_objects":7077,"num_object_clones":0,"num_object_copies":14154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11821,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":564},{"shards":"2","objects":3419}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13671","reported_seq":13755,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.218644+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:40.218644+0000","last_peered":"2024-09-10T15:58:40.218644+0000","last_clean":"2024-09-10T15:58:40.218644+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:40.218644+0000","last_undegraded":"2024-09-10T15:58:40.218644+0000","last_fullsized":"2024-09-10T15:58:40.218644+0000","mapping_epoch":36,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28717056,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13671,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13426","reported_seq":13557,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:39.953861+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:39.953861+0000","last_peered":"2024-09-10T15:58:39.953861+0000","last_clean":"2024-09-10T15:58:39.953861+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:39.953861+0000","last_undegraded":"2024-09-10T15:58:39.953861+0000","last_fullsized":"2024-09-10T15:58:39.953861+0000","mapping_epoch":35,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28282880,"num_objects":6905,"num_object_clones":0,"num_object_copies":13810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10497,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13710","reported_seq":21603,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.173317+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:40.173317+0000","last_peered":"2024-09-10T15:58:40.173317+0000","last_clean":"2024-09-10T15:58:40.173317+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:40.173317+0000","last_undegraded":"2024-09-10T15:58:40.173317+0000","last_fullsized":"2024-09-10T15:58:40.173317+0000","mapping_epoch":47,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10428,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13406","reported_seq":14057,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.015320+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:40.015320+0000","last_peered":"2024-09-10T15:58:40.015320+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:40.015320+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28180480,"num_objects":6880,"num_object_clones":0,"num_object_copies":13760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10056,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3302},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13781","reported_seq":13917,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.008021+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:40.008021+0000","last_peered":"2024-09-10T15:58:40.008021+0000","last_clean":"2024-09-10T15:58:40.008021+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:40.008021+0000","last_undegraded":"2024-09-10T15:58:40.008021+0000","last_fullsized":"2024-09-10T15:58:40.008021+0000","mapping_epoch":35,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10540,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13774","reported_seq":21906,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:40.159679+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:40.159679+0000","last_peered":"2024-09-10T15:58:40.159679+0000","last_clean":"2024-09-10T15:58:40.159679+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:40.159679+0000","last_undegraded":"2024-09-10T15:58:40.159679+0000","last_fullsized":"2024-09-10T15:58:40.159679+0000","mapping_epoch":45,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28446720,"num_objects":6945,"num_object_clones":0,"num_object_copies":13890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11751,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13614","reported_seq":14352,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.000606+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:40.000606+0000","last_peered":"2024-09-10T15:58:40.000606+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:40.000606+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28524544,"num_objects":6964,"num_object_clones":0,"num_object_copies":13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3562,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10478,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3404},{"shards":"0,2","objects":542}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13682","reported_seq":15596,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.337067+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:40.337067+0000","last_peered":"2024-09-10T15:58:40.337067+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:40.337067+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29093888,"num_objects":7103,"num_object_clones":0,"num_object_copies":14206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2397,"num_objects_misplaced":545,"num_objects_unfound":0,"num_objects_dirty":7103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11656,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":892,"num_bytes_recovered":3133440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":545},{"shards":"2","objects":4161}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13483","reported_seq":13836,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.141924+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:40.141924+0000","last_peered":"2024-09-10T15:58:40.141924+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:40.141924+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"37'3400","ondisk_log_start":"37'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28561408,"num_objects":6973,"num_object_clones":0,"num_object_copies":13946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11352,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":576},{"shards":"2","objects":3255}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13659","reported_seq":14395,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:39.960841+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:39.960841+0000","last_peered":"2024-09-10T15:58:39.960841+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:39.960841+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29470720,"num_objects":7195,"num_object_clones":0,"num_object_copies":14390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10851,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3664},{"shards":"0,2","objects":602}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13687","reported_seq":14369,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:39.963855+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:39.963855+0000","last_peered":"2024-09-10T15:58:39.963855+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:39.963855+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28798976,"num_objects":7031,"num_object_clones":0,"num_object_copies":14062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3663,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10105,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3371},{"shards":"0,2","objects":593}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13665","reported_seq":14370,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.032038+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:40.032038+0000","last_peered":"2024-09-10T15:58:40.032038+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:40.032038+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3599,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10271,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3504},{"shards":"0,2","objects":597}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'13992","reported_seq":14656,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:40.009027+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:40.009027+0000","last_peered":"2024-09-10T15:58:40.009027+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:40.009027+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"40'3900","ondisk_log_start":"40'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29106659,"num_objects":7107,"num_object_clones":0,"num_object_copies":14214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3746,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10811,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3364},{"shards":"0,2","objects":566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":461783547,"num_objects":112742,"num_object_clones":0,"num_object_copies":225484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35110,"num_objects_misplaced":545,"num_objects_unfound":0,"num_objects_dirty":112742,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180308,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10993,"num_bytes_recovered":35336192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1090973696,"data_stored":1090896905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160887,"ondisk_log_size":160887,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626499,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":644360,"kb_used_data":315636,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93727480,"statfs":{"total":96636764160,"available":95976939520,"internally_reserved":0,"allocated":323211264,"data_stored":322872886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336588462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149782,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794468,"kb_used_data":401872,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93577372,"statfs":{"total":96636764160,"available":95823228928,"internally_reserved":0,"allocated":411516928,"data_stored":411240430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":401993389},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,1,3,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652712,"kb_used_data":350360,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93719128,"statfs":{"total":96636764160,"available":95968387072,"internally_reserved":0,"allocated":358768640,"data_stored":358488070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,0,0,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":410062848,"data_stored":410059235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322760704,"data_stored":322695211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":358150144,"data_stored":358142459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:45.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:58:45.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'13758", 'reported_seq': 14481, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:40.036173+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:58:40.036173+0000', 'last_peered': '2024-09-10T15:58:40.036173+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:58:40.036173+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "40'3700", 'ondisk_log_start': "40'3700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28721152, 'num_objects': 7012, 'num_object_clones': 0, 'num_object_copies': 14024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3652, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7012, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13758, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 321, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3360}, {'shards': '0,2', 'objects': 600}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:45.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:58:45.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'13553", 'reported_seq': 13967, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:40.318262+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:58:40.318262+0000', 'last_peered': '2024-09-10T15:58:40.318262+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:58:40.318262+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "40'3500", 'ondisk_log_start': "40'3500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28983320, 'num_objects': 7077, 'num_object_clones': 0, 'num_object_copies': 14154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3659, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7077, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11821, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 564}, {'shards': '2', 'objects': 3419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:45.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:58:45.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'13406", 'reported_seq': 14057, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:40.015320+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:58:40.015320+0000', 'last_peered': '2024-09-10T15:58:40.015320+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:58:40.015320+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "37'3400", 'ondisk_log_start': "37'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28180480, 'num_objects': 6880, 'num_object_clones': 0, 'num_object_copies': 13760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3580, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6880, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10056, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 257, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3302}, {'shards': '0,2', 'objects': 574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:45.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:58:45.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'13614", 'reported_seq': 14352, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:40.000606+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:58:40.000606+0000', 'last_peered': '2024-09-10T15:58:40.000606+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:58:40.000606+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "40'3600", 'ondisk_log_start': "40'3600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28524544, 'num_objects': 6964, 'num_object_clones': 0, 'num_object_copies': 13928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3562, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6964, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10478, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3404}, {'shards': '0,2', 'objects': 542}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:45.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:58:45.650 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'13682", 'reported_seq': 15596, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:40.337067+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:58:40.337067+0000', 'last_peered': '2024-09-10T15:58:40.337067+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:58:40.337067+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "40'3600", 'ondisk_log_start': "40'3600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29093888, 'num_objects': 7103, 'num_object_clones': 0, 'num_object_copies': 14206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2397, 'num_objects_misplaced': 545, 'num_objects_unfound': 0, 'num_objects_dirty': 7103, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11656, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 892, 'num_bytes_recovered': 3133440, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 545}, {'shards': '2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:45.651 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:58:45.651 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'13483", 'reported_seq': 13836, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:40.141924+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:58:40.141924+0000', 'last_peered': '2024-09-10T15:58:40.141924+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:58:40.141924+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "37'3400", 'ondisk_log_start': "37'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28561408, 'num_objects': 6973, 'num_object_clones': 0, 'num_object_copies': 13946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3720, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6973, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11352, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 111, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 576}, {'shards': '2', 'objects': 3255}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:45.651 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:58:45.651 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'13659", 'reported_seq': 14395, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:39.960841+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:58:39.960841+0000', 'last_peered': '2024-09-10T15:58:39.960841+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:58:39.960841+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "40'3600", 'ondisk_log_start': "40'3600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29470720, 'num_objects': 7195, 'num_object_clones': 0, 'num_object_copies': 14390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3532, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7195, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10851, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 302, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3664}, {'shards': '0,2', 'objects': 602}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:45.651 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:58:45.651 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'13687", 'reported_seq': 14369, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:39.963855+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:58:39.963855+0000', 'last_peered': '2024-09-10T15:58:39.963855+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:58:39.963855+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "40'3600", 'ondisk_log_start': "40'3600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28798976, 'num_objects': 7031, 'num_object_clones': 0, 'num_object_copies': 14062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3663, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7031, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10105, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 257, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3371}, {'shards': '0,2', 'objects': 593}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:45.651 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:58:45.651 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'13665", 'reported_seq': 14370, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:40.032038+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:58:40.032038+0000', 'last_peered': '2024-09-10T15:58:40.032038+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:58:40.032038+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "40'3600", 'ondisk_log_start': "40'3600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29085696, 'num_objects': 7101, 'num_object_clones': 0, 'num_object_copies': 14202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3599, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10271, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 281, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3504}, {'shards': '0,2', 'objects': 597}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:45.651 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:58:45.651 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'13992", 'reported_seq': 14656, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:40.009027+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:58:40.009027+0000', 'last_peered': '2024-09-10T15:58:40.009027+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:58:40.009027+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "40'3900", 'ondisk_log_start': "40'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29106659, 'num_objects': 7107, 'num_object_clones': 0, 'num_object_copies': 14214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3746, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7107, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10811, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 260, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3364}, {'shards': '0,2', 'objects': 566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:45.652 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:46.333 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:46.334 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35110/225488 objects degraded (15.571%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":452,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":112744,"data_bytes":461783566,"bytes_used":2141736960,"bytes_avail":287768555520,"bytes_total":289910292480,"degraded_objects":35110,"degraded_total":225488,"degraded_ratio":0.15570673383949479,"misplaced_objects":545,"misplaced_total":225488,"misplaced_ratio":0.002416980061023203,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":91781,"recovering_keys_per_sec":0,"num_objects_recovered":436,"num_bytes_recovered":1101824,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":378},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:58:46.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:58:46.366 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:46.945 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:46.973 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-10T15:58:46.545790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":452325902,"num_objects":110435,"num_object_clones":0,"num_object_copies":220870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35091,"num_objects_misplaced":402,"num_objects_unfound":0,"num_objects_dirty":110435,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182619,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11157,"num_bytes_recovered":35803155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2079160,"kb_used_data":1055480,"kb_used_omap":12,"kb_used_meta":1023603,"kb_avail":281036360,"statfs":{"total":289910292480,"available":287781232640,"internally_reserved":0,"allocated":1080811520,"data_stored":1079916074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1048169477},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,6,4,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28057600,"num_objects":-6850,"num_object_clones":0,"num_object_copies":-13700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-172,"num_objects_misplaced":-426,"num_objects_unfound":0,"num_objects_dirty":-6850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6850,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":598,"num_bytes_recovered":1568768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004953"},"pg_stats":[{"pgid":"2.f","version":"48'13908","reported_seq":14639,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:45.030803+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:45.030803+0000","last_peered":"2024-09-10T15:58:45.030803+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:45.030803+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"41'3900","ondisk_log_start":"41'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28106752,"num_objects":6862,"num_object_clones":0,"num_object_copies":13724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13908,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":325,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3214},{"shards":"0,2","objects":450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14068","reported_seq":14180,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:45.323288+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:45.323288+0000","last_peered":"2024-09-10T15:58:45.323288+0000","last_clean":"2024-09-10T15:58:45.323288+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:45.323288+0000","last_undegraded":"2024-09-10T15:58:45.323288+0000","last_fullsized":"2024-09-10T15:58:45.323288+0000","mapping_epoch":36,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28696576,"num_objects":7006,"num_object_clones":0,"num_object_copies":14012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12404,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13688","reported_seq":14119,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:45.274114+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:45.274114+0000","last_peered":"2024-09-10T15:58:45.274114+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:45.274114+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28430360,"num_objects":6942,"num_object_clones":0,"num_object_copies":13884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3652,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11956,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":429},{"shards":"2","objects":3291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13798","reported_seq":13882,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:45.323271+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:45.323271+0000","last_peered":"2024-09-10T15:58:45.323271+0000","last_clean":"2024-09-10T15:58:45.323271+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:45.323271+0000","last_undegraded":"2024-09-10T15:58:45.323271+0000","last_fullsized":"2024-09-10T15:58:45.323271+0000","mapping_epoch":36,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28196864,"num_objects":6884,"num_object_clones":0,"num_object_copies":13768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13798,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13577","reported_seq":13709,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:45.008372+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:45.008372+0000","last_peered":"2024-09-10T15:58:45.008372+0000","last_clean":"2024-09-10T15:58:45.008372+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:45.008372+0000","last_undegraded":"2024-09-10T15:58:45.008372+0000","last_fullsized":"2024-09-10T15:58:45.008372+0000","mapping_epoch":35,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27664384,"num_objects":6754,"num_object_clones":0,"num_object_copies":13508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10648,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13853","reported_seq":21746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:45.108385+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:45.108385+0000","last_peered":"2024-09-10T15:58:45.108385+0000","last_clean":"2024-09-10T15:58:45.108385+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:45.108385+0000","last_undegraded":"2024-09-10T15:58:45.108385+0000","last_fullsized":"2024-09-10T15:58:45.108385+0000","mapping_epoch":47,"log_start":"40'3800","ondisk_log_start":"40'3800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10571,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13562","reported_seq":14237,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:44.948098+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:44.948098+0000","last_peered":"2024-09-10T15:58:44.948098+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:44.948098+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541504,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10212,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3155},{"shards":"0,2","objects":418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13913","reported_seq":14049,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:44.936009+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:44.936009+0000","last_peered":"2024-09-10T15:58:44.936009+0000","last_clean":"2024-09-10T15:58:44.936009+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:44.936009+0000","last_undegraded":"2024-09-10T15:58:44.936009+0000","last_fullsized":"2024-09-10T15:58:44.936009+0000","mapping_epoch":35,"log_start":"41'3900","ondisk_log_start":"41'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28676096,"num_objects":7001,"num_object_clones":0,"num_object_copies":14002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10672,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13916","reported_seq":22048,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:45.132852+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:45.132852+0000","last_peered":"2024-09-10T15:58:45.132852+0000","last_clean":"2024-09-10T15:58:45.132852+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:45.132852+0000","last_undegraded":"2024-09-10T15:58:45.132852+0000","last_fullsized":"2024-09-10T15:58:45.132852+0000","mapping_epoch":45,"log_start":"40'3900","ondisk_log_start":"40'3900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27865088,"num_objects":6803,"num_object_clones":0,"num_object_copies":13606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11893,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13750","reported_seq":14494,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:44.957899+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:44.957899+0000","last_peered":"2024-09-10T15:58:44.957899+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:44.957899+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967488,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3559,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10614,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3271},{"shards":"0,2","objects":406}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13824","reported_seq":15966,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:45.344172+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:45.344172+0000","last_peered":"2024-09-10T15:58:45.344172+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:45.344172+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"41'3800","ondisk_log_start":"41'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28512256,"num_objects":6961,"num_object_clones":0,"num_object_copies":13922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2426,"num_objects_misplaced":402,"num_objects_unfound":0,"num_objects_dirty":6961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11798,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1006,"num_bytes_recovered":3600384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":402},{"shards":"2","objects":4133}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13643","reported_seq":14012,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:45.357069+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:45.357069+0000","last_peered":"2024-09-10T15:58:45.357069+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:45.357069+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27906048,"num_objects":6813,"num_object_clones":0,"num_object_copies":13626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3712,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11512,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":416},{"shards":"2","objects":3103}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13824","reported_seq":14566,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:45.023202+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:45.023202+0000","last_peered":"2024-09-10T15:58:45.023202+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:45.023202+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"41'3800","ondisk_log_start":"41'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28794880,"num_objects":7030,"num_object_clones":0,"num_object_copies":14060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3529,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11016,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3502},{"shards":"0,2","objects":437}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13830","reported_seq":14519,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:45.048905+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:45.048905+0000","last_peered":"2024-09-10T15:58:45.048905+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:45.048905+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"40'3800","ondisk_log_start":"40'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28213248,"num_objects":6888,"num_object_clones":0,"num_object_copies":13776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3660,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10248,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3231},{"shards":"0,2","objects":450}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13800","reported_seq":14527,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:44.993626+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:44.993626+0000","last_peered":"2024-09-10T15:58:44.993626+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:44.993626+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28532736,"num_objects":6966,"num_object_clones":0,"num_object_copies":13932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10406,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3380},{"shards":"0,2","objects":462}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14142","reported_seq":14806,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:44.969703+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:44.969703+0000","last_peered":"2024-09-10T15:58:44.969703+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:44.969703+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28492259,"num_objects":6957,"num_object_clones":0,"num_object_copies":13914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3746,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10961,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3214},{"shards":"0,2","objects":416}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":452325883,"num_objects":110433,"num_object_clones":0,"num_object_copies":220866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35091,"num_objects_misplaced":402,"num_objects_unfound":0,"num_objects_dirty":110433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182617,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11155,"num_bytes_recovered":35803136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1078104064,"data_stored":1078027273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626500,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630768,"kb_used_data":314516,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93741072,"statfs":{"total":96636764160,"available":95990857728,"internally_reserved":0,"allocated":322064384,"data_stored":321726006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323808942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149783,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":796028,"kb_used_data":395112,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93575812,"statfs":{"total":96636764160,"available":95821631488,"internally_reserved":0,"allocated":404594688,"data_stored":404318190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":410513069},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652364,"kb_used_data":345852,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93719476,"statfs":{"total":96636764160,"available":95968743424,"internally_reserved":0,"allocated":354152448,"data_stored":353871878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,2,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":402948096,"data_stored":402944483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321626112,"data_stored":321560619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353529856,"data_stored":353522171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:49.975 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:50.548 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:50.550 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:50.571 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-10T15:58:48.547997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":452325902,"num_objects":110435,"num_object_clones":0,"num_object_copies":220870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35091,"num_objects_misplaced":402,"num_objects_unfound":0,"num_objects_dirty":110435,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182619,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11157,"num_bytes_recovered":35803155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2079160,"kb_used_data":1055480,"kb_used_omap":12,"kb_used_meta":1023603,"kb_avail":281036360,"statfs":{"total":289910292480,"available":287781232640,"internally_reserved":0,"allocated":1080811520,"data_stored":1079916074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1048169477},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,6,4,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18911232,"num_objects":-4617,"num_object_clones":0,"num_object_copies":-9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-61,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":-4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4617,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006577"},"pg_stats":[{"pgid":"2.f","version":"48'13908","reported_seq":14639,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:45.030803+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:45.030803+0000","last_peered":"2024-09-10T15:58:45.030803+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:45.030803+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"41'3900","ondisk_log_start":"41'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28106752,"num_objects":6862,"num_object_clones":0,"num_object_copies":13724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13908,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":325,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3214},{"shards":"0,2","objects":450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14068","reported_seq":14180,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:45.323288+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:45.323288+0000","last_peered":"2024-09-10T15:58:45.323288+0000","last_clean":"2024-09-10T15:58:45.323288+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:45.323288+0000","last_undegraded":"2024-09-10T15:58:45.323288+0000","last_fullsized":"2024-09-10T15:58:45.323288+0000","mapping_epoch":36,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28696576,"num_objects":7006,"num_object_clones":0,"num_object_copies":14012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12404,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13688","reported_seq":14119,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:45.274114+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:45.274114+0000","last_peered":"2024-09-10T15:58:45.274114+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:45.274114+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28430360,"num_objects":6942,"num_object_clones":0,"num_object_copies":13884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3652,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11956,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":429},{"shards":"2","objects":3291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13798","reported_seq":13882,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:45.323271+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:45.323271+0000","last_peered":"2024-09-10T15:58:45.323271+0000","last_clean":"2024-09-10T15:58:45.323271+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:45.323271+0000","last_undegraded":"2024-09-10T15:58:45.323271+0000","last_fullsized":"2024-09-10T15:58:45.323271+0000","mapping_epoch":36,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28196864,"num_objects":6884,"num_object_clones":0,"num_object_copies":13768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13798,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13577","reported_seq":13709,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:45.008372+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:45.008372+0000","last_peered":"2024-09-10T15:58:45.008372+0000","last_clean":"2024-09-10T15:58:45.008372+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:45.008372+0000","last_undegraded":"2024-09-10T15:58:45.008372+0000","last_fullsized":"2024-09-10T15:58:45.008372+0000","mapping_epoch":35,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27664384,"num_objects":6754,"num_object_clones":0,"num_object_copies":13508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10648,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13853","reported_seq":21746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:45.108385+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:45.108385+0000","last_peered":"2024-09-10T15:58:45.108385+0000","last_clean":"2024-09-10T15:58:45.108385+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:45.108385+0000","last_undegraded":"2024-09-10T15:58:45.108385+0000","last_fullsized":"2024-09-10T15:58:45.108385+0000","mapping_epoch":47,"log_start":"40'3800","ondisk_log_start":"40'3800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10571,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13562","reported_seq":14237,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:44.948098+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:44.948098+0000","last_peered":"2024-09-10T15:58:44.948098+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:44.948098+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"40'3500","ondisk_log_start":"40'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541504,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10212,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3155},{"shards":"0,2","objects":418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'13913","reported_seq":14049,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:44.936009+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:44.936009+0000","last_peered":"2024-09-10T15:58:44.936009+0000","last_clean":"2024-09-10T15:58:44.936009+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:44.936009+0000","last_undegraded":"2024-09-10T15:58:44.936009+0000","last_fullsized":"2024-09-10T15:58:44.936009+0000","mapping_epoch":35,"log_start":"41'3900","ondisk_log_start":"41'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28676096,"num_objects":7001,"num_object_clones":0,"num_object_copies":14002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10672,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'13916","reported_seq":22048,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:45.132852+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:45.132852+0000","last_peered":"2024-09-10T15:58:45.132852+0000","last_clean":"2024-09-10T15:58:45.132852+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:45.132852+0000","last_undegraded":"2024-09-10T15:58:45.132852+0000","last_fullsized":"2024-09-10T15:58:45.132852+0000","mapping_epoch":45,"log_start":"40'3900","ondisk_log_start":"40'3900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27865088,"num_objects":6803,"num_object_clones":0,"num_object_copies":13606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11893,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13750","reported_seq":14494,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:44.957899+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:44.957899+0000","last_peered":"2024-09-10T15:58:44.957899+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:44.957899+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967488,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3559,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10614,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3271},{"shards":"0,2","objects":406}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13824","reported_seq":15966,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:45.344172+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:45.344172+0000","last_peered":"2024-09-10T15:58:45.344172+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:45.344172+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"41'3800","ondisk_log_start":"41'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28512256,"num_objects":6961,"num_object_clones":0,"num_object_copies":13922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2426,"num_objects_misplaced":402,"num_objects_unfound":0,"num_objects_dirty":6961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11798,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1006,"num_bytes_recovered":3600384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":402},{"shards":"2","objects":4133}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13643","reported_seq":14012,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:45.357069+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:45.357069+0000","last_peered":"2024-09-10T15:58:45.357069+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:45.357069+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"40'3600","ondisk_log_start":"40'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27906048,"num_objects":6813,"num_object_clones":0,"num_object_copies":13626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3712,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11512,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":416},{"shards":"2","objects":3103}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13824","reported_seq":14566,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:45.023202+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:45.023202+0000","last_peered":"2024-09-10T15:58:45.023202+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:45.023202+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"41'3800","ondisk_log_start":"41'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28794880,"num_objects":7030,"num_object_clones":0,"num_object_copies":14060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3529,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11016,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3502},{"shards":"0,2","objects":437}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13830","reported_seq":14519,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:45.048905+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:45.048905+0000","last_peered":"2024-09-10T15:58:45.048905+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:45.048905+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"40'3800","ondisk_log_start":"40'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28213248,"num_objects":6888,"num_object_clones":0,"num_object_copies":13776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3660,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10248,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3231},{"shards":"0,2","objects":450}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13800","reported_seq":14527,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:44.993626+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:44.993626+0000","last_peered":"2024-09-10T15:58:44.993626+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:44.993626+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28532736,"num_objects":6966,"num_object_clones":0,"num_object_copies":13932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10406,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3380},{"shards":"0,2","objects":462}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14142","reported_seq":14806,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:44.969703+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:44.969703+0000","last_peered":"2024-09-10T15:58:44.969703+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:44.969703+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28492259,"num_objects":6957,"num_object_clones":0,"num_object_copies":13914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3746,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10961,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3214},{"shards":"0,2","objects":416}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":452325883,"num_objects":110433,"num_object_clones":0,"num_object_copies":220866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35091,"num_objects_misplaced":402,"num_objects_unfound":0,"num_objects_dirty":110433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182617,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11155,"num_bytes_recovered":35803136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1078104064,"data_stored":1078027273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626500,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630768,"kb_used_data":314516,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93741072,"statfs":{"total":96636764160,"available":95990857728,"internally_reserved":0,"allocated":322064384,"data_stored":321726006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323808942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149783,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":796028,"kb_used_data":395112,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93575812,"statfs":{"total":96636764160,"available":95821631488,"internally_reserved":0,"allocated":404594688,"data_stored":404318190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":410513069},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652364,"kb_used_data":345852,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93719476,"statfs":{"total":96636764160,"available":95968743424,"internally_reserved":0,"allocated":354152448,"data_stored":353871878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,2,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":402948096,"data_stored":402944483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321626112,"data_stored":321560619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353529856,"data_stored":353522171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:50.573 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:51.102 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:51.103 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:51.136 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-10T15:58:50.548546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":443105806,"num_objects":108184,"num_object_clones":0,"num_object_copies":216368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34955,"num_objects_misplaced":244,"num_objects_unfound":0,"num_objects_dirty":108184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":184870,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11451,"num_bytes_recovered":36413459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2066852,"kb_used_data":1043180,"kb_used_omap":12,"kb_used_meta":1023603,"kb_avail":281048668,"statfs":{"total":289910292480,"available":287793836032,"internally_reserved":0,"allocated":1068216320,"data_stored":1067320874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1048169477},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,4,7,11,6,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28131328,"num_objects":-6868,"num_object_clones":0,"num_object_copies":-13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":-441,"num_objects_unfound":0,"num_objects_dirty":-6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6868,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":1593344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005950"},"pg_stats":[{"pgid":"2.f","version":"48'14055","reported_seq":14810,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.037781+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:50.037781+0000","last_peered":"2024-09-10T15:58:50.037781+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:50.037781+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27504640,"num_objects":6715,"num_object_clones":0,"num_object_copies":13430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14055,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3079},{"shards":"0,2","objects":303}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14205","reported_seq":14317,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.292314+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:50.292314+0000","last_peered":"2024-09-10T15:58:50.292314+0000","last_clean":"2024-09-10T15:58:50.292314+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:50.292314+0000","last_undegraded":"2024-09-10T15:58:50.292314+0000","last_fullsized":"2024-09-10T15:58:50.292314+0000","mapping_epoch":36,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28135424,"num_objects":6869,"num_object_clones":0,"num_object_copies":13738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12541,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13848","reported_seq":14305,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.232543+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:50.232543+0000","last_peered":"2024-09-10T15:58:50.232543+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:50.232543+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"41'3800","ondisk_log_start":"41'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27775000,"num_objects":6782,"num_object_clones":0,"num_object_copies":13564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6782,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12116,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":269},{"shards":"2","objects":3144}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13934","reported_seq":14018,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.376537+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:50.376537+0000","last_peered":"2024-09-10T15:58:50.376537+0000","last_clean":"2024-09-10T15:58:50.376537+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:50.376537+0000","last_undegraded":"2024-09-10T15:58:50.376537+0000","last_fullsized":"2024-09-10T15:58:50.376537+0000","mapping_epoch":36,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27639808,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13934,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13722","reported_seq":13854,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.060555+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:50.060555+0000","last_peered":"2024-09-10T15:58:50.060555+0000","last_clean":"2024-09-10T15:58:50.060555+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:50.060555+0000","last_undegraded":"2024-09-10T15:58:50.060555+0000","last_fullsized":"2024-09-10T15:58:50.060555+0000","mapping_epoch":35,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27070464,"num_objects":6609,"num_object_clones":0,"num_object_copies":13218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10793,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13994","reported_seq":21887,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.185422+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:50.185422+0000","last_peered":"2024-09-10T15:58:50.185422+0000","last_clean":"2024-09-10T15:58:50.185422+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:50.185422+0000","last_undegraded":"2024-09-10T15:58:50.185422+0000","last_fullsized":"2024-09-10T15:58:50.185422+0000","mapping_epoch":47,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28151808,"num_objects":6873,"num_object_clones":0,"num_object_copies":13746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10712,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13710","reported_seq":14431,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:49.959711+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:49.959711+0000","last_peered":"2024-09-10T15:58:49.959711+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:49.959711+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10360,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3030},{"shards":"0,2","objects":270}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14042","reported_seq":14178,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:49.995071+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:49.995071+0000","last_peered":"2024-09-10T15:58:49.995071+0000","last_clean":"2024-09-10T15:58:49.995071+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:49.995071+0000","last_undegraded":"2024-09-10T15:58:49.995071+0000","last_fullsized":"2024-09-10T15:58:49.995071+0000","mapping_epoch":35,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28147712,"num_objects":6872,"num_object_clones":0,"num_object_copies":13744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10801,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14076","reported_seq":22208,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.217902+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:50.217902+0000","last_peered":"2024-09-10T15:58:50.217902+0000","last_clean":"2024-09-10T15:58:50.217902+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:50.217902+0000","last_undegraded":"2024-09-10T15:58:50.217902+0000","last_fullsized":"2024-09-10T15:58:50.217902+0000","mapping_epoch":45,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27209728,"num_objects":6643,"num_object_clones":0,"num_object_copies":13286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12053,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13876","reported_seq":14660,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:49.934698+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:49.934698+0000","last_peered":"2024-09-10T15:58:49.934698+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:49.934698+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"40'3800","ondisk_log_start":"40'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27451392,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10740,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3165},{"shards":"0,2","objects":280}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13953","reported_seq":16398,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.376075+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:50.376075+0000","last_peered":"2024-09-10T15:58:50.376075+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:50.376075+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27983872,"num_objects":6832,"num_object_clones":0,"num_object_copies":13664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2433,"num_objects_misplaced":244,"num_objects_unfound":0,"num_objects_dirty":6832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11927,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1157,"num_bytes_recovered":4210688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":244},{"shards":"2","objects":4155}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13769","reported_seq":14158,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.293634+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:50.293634+0000","last_peered":"2024-09-10T15:58:50.293634+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:50.293634+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27389952,"num_objects":6687,"num_object_clones":0,"num_object_copies":13374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11638,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":290},{"shards":"2","objects":2987}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13976","reported_seq":14760,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.038258+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:50.038258+0000","last_peered":"2024-09-10T15:58:50.038258+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:50.038258+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28172288,"num_objects":6878,"num_object_clones":0,"num_object_copies":13756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11168,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3371},{"shards":"0,2","objects":285}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13974","reported_seq":14689,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:49.951035+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:49.951035+0000","last_peered":"2024-09-10T15:58:49.951035+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:49.951035+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"41'3900","ondisk_log_start":"41'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10392,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3100},{"shards":"0,2","objects":306}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13953","reported_seq":14722,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.048444+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:50.048444+0000","last_peered":"2024-09-10T15:58:50.048444+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:50.048444+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27906048,"num_objects":6813,"num_object_clones":0,"num_object_copies":13626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10559,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3248},{"shards":"0,2","objects":309}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14260","reported_seq":14944,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.050248+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:50.050248+0000","last_peered":"2024-09-10T15:58:50.050248+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:50.050248+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28008931,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3736,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11079,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3106},{"shards":"0,2","objects":298}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":443105787,"num_objects":108182,"num_object_clones":0,"num_object_copies":216364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34955,"num_objects_misplaced":244,"num_objects_unfound":0,"num_objects_dirty":108182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":184868,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11449,"num_bytes_recovered":36413440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1065492480,"data_stored":1065415689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626501,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637976,"kb_used_data":313416,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93733864,"statfs":{"total":96636764160,"available":95983476736,"internally_reserved":0,"allocated":320937984,"data_stored":320599606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332328622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,4,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149784,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":776916,"kb_used_data":388480,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93594924,"statfs":{"total":96636764160,"available":95841202176,"internally_reserved":0,"allocated":397803520,"data_stored":397527022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":397733549},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,1,6,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651960,"kb_used_data":341284,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93719880,"statfs":{"total":96636764160,"available":95969157120,"internally_reserved":0,"allocated":349474816,"data_stored":349194246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,0,0,0,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":396070912,"data_stored":396067299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320487424,"data_stored":320421931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348934144,"data_stored":348926459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:51.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:58:51.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'14055", 'reported_seq': 14810, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:50.037781+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:58:50.037781+0000', 'last_peered': '2024-09-10T15:58:50.037781+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:58:50.037781+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'4000", 'ondisk_log_start': "44'4000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27504640, 'num_objects': 6715, 'num_object_clones': 0, 'num_object_copies': 13430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3636, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6715, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14055, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 337, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3079}, {'shards': '0,2', 'objects': 303}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:51.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:58:51.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'13848", 'reported_seq': 14305, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:50.232543+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:58:50.232543+0000', 'last_peered': '2024-09-10T15:58:50.232543+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:58:50.232543+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "41'3800", 'ondisk_log_start': "41'3800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27775000, 'num_objects': 6782, 'num_object_clones': 0, 'num_object_copies': 13564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3639, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6782, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12116, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 164, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 269}, {'shards': '2', 'objects': 3144}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:51.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:58:51.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'13710", 'reported_seq': 14431, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:49.959711+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:58:49.959711+0000', 'last_peered': '2024-09-10T15:58:49.959711+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:58:49.959711+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "40'3700", 'ondisk_log_start': "40'3700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26935296, 'num_objects': 6576, 'num_object_clones': 0, 'num_object_copies': 13152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3548, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6576, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10360, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 289, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3030}, {'shards': '0,2', 'objects': 270}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:51.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:58:51.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'13876", 'reported_seq': 14660, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:49.934698+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:58:49.934698+0000', 'last_peered': '2024-09-10T15:58:49.934698+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:58:49.934698+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "40'3800", 'ondisk_log_start': "40'3800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27451392, 'num_objects': 6702, 'num_object_clones': 0, 'num_object_copies': 13404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3539, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6702, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10740, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3165}, {'shards': '0,2', 'objects': 280}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:51.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:58:51.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'13953", 'reported_seq': 16398, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:50.376075+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:58:50.376075+0000', 'last_peered': '2024-09-10T15:58:50.376075+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:58:50.376075+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'3900", 'ondisk_log_start': "44'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27983872, 'num_objects': 6832, 'num_object_clones': 0, 'num_object_copies': 13664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2433, 'num_objects_misplaced': 244, 'num_objects_unfound': 0, 'num_objects_dirty': 6832, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11927, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1157, 'num_bytes_recovered': 4210688, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 244}, {'shards': '2', 'objects': 4155}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:51.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:58:51.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'13769", 'reported_seq': 14158, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:50.293634+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:58:50.293634+0000', 'last_peered': '2024-09-10T15:58:50.293634+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:58:50.293634+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "40'3700", 'ondisk_log_start': "40'3700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27389952, 'num_objects': 6687, 'num_object_clones': 0, 'num_object_copies': 13374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3702, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6687, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11638, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 129, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 290}, {'shards': '2', 'objects': 2987}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:51.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:58:51.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'13976", 'reported_seq': 14760, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:50.038258+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:58:50.038258+0000', 'last_peered': '2024-09-10T15:58:50.038258+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:58:50.038258+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'3900", 'ondisk_log_start': "44'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28172288, 'num_objects': 6878, 'num_object_clones': 0, 'num_object_copies': 13756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3508, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6878, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11168, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3371}, {'shards': '0,2', 'objects': 285}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:51.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:58:51.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'13974", 'reported_seq': 14689, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:49.951035+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:58:49.951035+0000', 'last_peered': '2024-09-10T15:58:49.951035+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:58:49.951035+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "41'3900", 'ondisk_log_start': "41'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27623424, 'num_objects': 6744, 'num_object_clones': 0, 'num_object_copies': 13488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3647, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6744, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10392, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 273, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3100}, {'shards': '0,2', 'objects': 306}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:51.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:58:51.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'13953", 'reported_seq': 14722, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:50.048444+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:58:50.048444+0000', 'last_peered': '2024-09-10T15:58:50.048444+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:58:50.048444+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'3900", 'ondisk_log_start': "44'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27906048, 'num_objects': 6813, 'num_object_clones': 0, 'num_object_copies': 13626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3567, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6813, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10559, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 313, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3248}, {'shards': '0,2', 'objects': 309}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:51.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:58:51.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'14260", 'reported_seq': 14944, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:50.050248+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:58:50.050248+0000', 'last_peered': '2024-09-10T15:58:50.050248+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:58:50.050248+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28008931, 'num_objects': 6839, 'num_object_clones': 0, 'num_object_copies': 13678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3736, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6839, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11079, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 270, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3106}, {'shards': '0,2', 'objects': 298}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:51.140 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:51.823 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:51.823 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34955/216368 objects degraded (16.155%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":457,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":108184,"data_bytes":443105806,"bytes_used":2116456448,"bytes_avail":287793836032,"bytes_total":289910292480,"degraded_objects":34955,"degraded_total":216368,"degraded_ratio":0.1615534644679435,"misplaced_objects":244,"misplaced_total":216368,"misplaced_ratio":0.0011277083487391852,"recovering_objects_per_sec":53,"recovering_bytes_per_sec":132712,"recovering_keys_per_sec":0,"num_objects_recovered":638,"num_bytes_recovered":1593344,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":572},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:58:51.858 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:58:51.858 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:52.408 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:52.409 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:52.440 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-10T15:58:50.548546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":443105806,"num_objects":108184,"num_object_clones":0,"num_object_copies":216368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34955,"num_objects_misplaced":244,"num_objects_unfound":0,"num_objects_dirty":108184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":184870,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11451,"num_bytes_recovered":36413459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2066852,"kb_used_data":1043180,"kb_used_omap":12,"kb_used_meta":1023603,"kb_avail":281048668,"statfs":{"total":289910292480,"available":287793836032,"internally_reserved":0,"allocated":1068216320,"data_stored":1067320874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1048169477},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,4,7,11,6,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28131328,"num_objects":-6868,"num_object_clones":0,"num_object_copies":-13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":-441,"num_objects_unfound":0,"num_objects_dirty":-6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6868,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":1593344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005950"},"pg_stats":[{"pgid":"2.f","version":"48'14055","reported_seq":14810,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.037781+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:50.037781+0000","last_peered":"2024-09-10T15:58:50.037781+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:50.037781+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27504640,"num_objects":6715,"num_object_clones":0,"num_object_copies":13430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14055,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3079},{"shards":"0,2","objects":303}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14205","reported_seq":14317,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.292314+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:50.292314+0000","last_peered":"2024-09-10T15:58:50.292314+0000","last_clean":"2024-09-10T15:58:50.292314+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:50.292314+0000","last_undegraded":"2024-09-10T15:58:50.292314+0000","last_fullsized":"2024-09-10T15:58:50.292314+0000","mapping_epoch":36,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28135424,"num_objects":6869,"num_object_clones":0,"num_object_copies":13738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12541,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13848","reported_seq":14305,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.232543+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:50.232543+0000","last_peered":"2024-09-10T15:58:50.232543+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:50.232543+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"41'3800","ondisk_log_start":"41'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27775000,"num_objects":6782,"num_object_clones":0,"num_object_copies":13564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6782,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12116,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":269},{"shards":"2","objects":3144}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13934","reported_seq":14018,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.376537+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:50.376537+0000","last_peered":"2024-09-10T15:58:50.376537+0000","last_clean":"2024-09-10T15:58:50.376537+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:50.376537+0000","last_undegraded":"2024-09-10T15:58:50.376537+0000","last_fullsized":"2024-09-10T15:58:50.376537+0000","mapping_epoch":36,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27639808,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13934,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13722","reported_seq":13854,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.060555+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:50.060555+0000","last_peered":"2024-09-10T15:58:50.060555+0000","last_clean":"2024-09-10T15:58:50.060555+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:50.060555+0000","last_undegraded":"2024-09-10T15:58:50.060555+0000","last_fullsized":"2024-09-10T15:58:50.060555+0000","mapping_epoch":35,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27070464,"num_objects":6609,"num_object_clones":0,"num_object_copies":13218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10793,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13994","reported_seq":21887,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.185422+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:50.185422+0000","last_peered":"2024-09-10T15:58:50.185422+0000","last_clean":"2024-09-10T15:58:50.185422+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:50.185422+0000","last_undegraded":"2024-09-10T15:58:50.185422+0000","last_fullsized":"2024-09-10T15:58:50.185422+0000","mapping_epoch":47,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28151808,"num_objects":6873,"num_object_clones":0,"num_object_copies":13746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10712,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13710","reported_seq":14431,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:49.959711+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:49.959711+0000","last_peered":"2024-09-10T15:58:49.959711+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:49.959711+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10360,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3030},{"shards":"0,2","objects":270}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14042","reported_seq":14178,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:49.995071+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:49.995071+0000","last_peered":"2024-09-10T15:58:49.995071+0000","last_clean":"2024-09-10T15:58:49.995071+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:49.995071+0000","last_undegraded":"2024-09-10T15:58:49.995071+0000","last_fullsized":"2024-09-10T15:58:49.995071+0000","mapping_epoch":35,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28147712,"num_objects":6872,"num_object_clones":0,"num_object_copies":13744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10801,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14076","reported_seq":22208,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.217902+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:50.217902+0000","last_peered":"2024-09-10T15:58:50.217902+0000","last_clean":"2024-09-10T15:58:50.217902+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:50.217902+0000","last_undegraded":"2024-09-10T15:58:50.217902+0000","last_fullsized":"2024-09-10T15:58:50.217902+0000","mapping_epoch":45,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27209728,"num_objects":6643,"num_object_clones":0,"num_object_copies":13286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12053,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13876","reported_seq":14660,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:49.934698+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:49.934698+0000","last_peered":"2024-09-10T15:58:49.934698+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:49.934698+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"40'3800","ondisk_log_start":"40'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27451392,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10740,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3165},{"shards":"0,2","objects":280}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13953","reported_seq":16398,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.376075+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:50.376075+0000","last_peered":"2024-09-10T15:58:50.376075+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:50.376075+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27983872,"num_objects":6832,"num_object_clones":0,"num_object_copies":13664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2433,"num_objects_misplaced":244,"num_objects_unfound":0,"num_objects_dirty":6832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11927,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1157,"num_bytes_recovered":4210688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":244},{"shards":"2","objects":4155}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13769","reported_seq":14158,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.293634+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:50.293634+0000","last_peered":"2024-09-10T15:58:50.293634+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:50.293634+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27389952,"num_objects":6687,"num_object_clones":0,"num_object_copies":13374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11638,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":290},{"shards":"2","objects":2987}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13976","reported_seq":14760,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.038258+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:50.038258+0000","last_peered":"2024-09-10T15:58:50.038258+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:50.038258+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28172288,"num_objects":6878,"num_object_clones":0,"num_object_copies":13756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11168,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3371},{"shards":"0,2","objects":285}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13974","reported_seq":14689,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:49.951035+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:49.951035+0000","last_peered":"2024-09-10T15:58:49.951035+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:49.951035+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"41'3900","ondisk_log_start":"41'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10392,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3100},{"shards":"0,2","objects":306}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13953","reported_seq":14722,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.048444+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:50.048444+0000","last_peered":"2024-09-10T15:58:50.048444+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:50.048444+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27906048,"num_objects":6813,"num_object_clones":0,"num_object_copies":13626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10559,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3248},{"shards":"0,2","objects":309}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14260","reported_seq":14944,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.050248+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:50.050248+0000","last_peered":"2024-09-10T15:58:50.050248+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:50.050248+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28008931,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3736,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11079,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3106},{"shards":"0,2","objects":298}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":443105787,"num_objects":108182,"num_object_clones":0,"num_object_copies":216364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34955,"num_objects_misplaced":244,"num_objects_unfound":0,"num_objects_dirty":108182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":184868,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11449,"num_bytes_recovered":36413440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1065492480,"data_stored":1065415689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626501,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637976,"kb_used_data":313416,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93733864,"statfs":{"total":96636764160,"available":95983476736,"internally_reserved":0,"allocated":320937984,"data_stored":320599606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332328622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,4,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149784,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":776916,"kb_used_data":388480,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93594924,"statfs":{"total":96636764160,"available":95841202176,"internally_reserved":0,"allocated":397803520,"data_stored":397527022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":397733549},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,1,6,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651960,"kb_used_data":341284,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93719880,"statfs":{"total":96636764160,"available":95969157120,"internally_reserved":0,"allocated":349474816,"data_stored":349194246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,0,0,0,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":396070912,"data_stored":396067299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320487424,"data_stored":320421931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348934144,"data_stored":348926459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:55.443 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:55.992 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:55.993 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:56.018 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-10T15:58:54.550591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":443105806,"num_objects":108184,"num_object_clones":0,"num_object_copies":216368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34955,"num_objects_misplaced":244,"num_objects_unfound":0,"num_objects_dirty":108184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":184870,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11451,"num_bytes_recovered":36413459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2066852,"kb_used_data":1043180,"kb_used_omap":12,"kb_used_meta":1023603,"kb_avail":281048668,"statfs":{"total":289910292480,"available":287793836032,"internally_reserved":0,"allocated":1068216320,"data_stored":1067320874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1048169477},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,4,7,11,6,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18677760,"num_objects":-4560,"num_object_clones":0,"num_object_copies":-9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-155,"num_objects_misplaced":-301,"num_objects_unfound":0,"num_objects_dirty":-4560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4560,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":456,"num_bytes_recovered":1077248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006450"},"pg_stats":[{"pgid":"2.f","version":"48'14055","reported_seq":14810,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.037781+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:50.037781+0000","last_peered":"2024-09-10T15:58:50.037781+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:50.037781+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27504640,"num_objects":6715,"num_object_clones":0,"num_object_copies":13430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14055,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3079},{"shards":"0,2","objects":303}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14205","reported_seq":14317,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.292314+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:50.292314+0000","last_peered":"2024-09-10T15:58:50.292314+0000","last_clean":"2024-09-10T15:58:50.292314+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:50.292314+0000","last_undegraded":"2024-09-10T15:58:50.292314+0000","last_fullsized":"2024-09-10T15:58:50.292314+0000","mapping_epoch":36,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28135424,"num_objects":6869,"num_object_clones":0,"num_object_copies":13738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12541,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13848","reported_seq":14305,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.232543+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:50.232543+0000","last_peered":"2024-09-10T15:58:50.232543+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:50.232543+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"41'3800","ondisk_log_start":"41'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27775000,"num_objects":6782,"num_object_clones":0,"num_object_copies":13564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6782,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12116,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":269},{"shards":"2","objects":3144}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'13934","reported_seq":14018,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.376537+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:50.376537+0000","last_peered":"2024-09-10T15:58:50.376537+0000","last_clean":"2024-09-10T15:58:50.376537+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:50.376537+0000","last_undegraded":"2024-09-10T15:58:50.376537+0000","last_fullsized":"2024-09-10T15:58:50.376537+0000","mapping_epoch":36,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27639808,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13934,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13722","reported_seq":13854,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.060555+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:50.060555+0000","last_peered":"2024-09-10T15:58:50.060555+0000","last_clean":"2024-09-10T15:58:50.060555+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:50.060555+0000","last_undegraded":"2024-09-10T15:58:50.060555+0000","last_fullsized":"2024-09-10T15:58:50.060555+0000","mapping_epoch":35,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27070464,"num_objects":6609,"num_object_clones":0,"num_object_copies":13218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10793,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'13994","reported_seq":21887,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.185422+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:50.185422+0000","last_peered":"2024-09-10T15:58:50.185422+0000","last_clean":"2024-09-10T15:58:50.185422+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:50.185422+0000","last_undegraded":"2024-09-10T15:58:50.185422+0000","last_fullsized":"2024-09-10T15:58:50.185422+0000","mapping_epoch":47,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28151808,"num_objects":6873,"num_object_clones":0,"num_object_copies":13746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10712,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13710","reported_seq":14431,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:49.959711+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:49.959711+0000","last_peered":"2024-09-10T15:58:49.959711+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:49.959711+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10360,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3030},{"shards":"0,2","objects":270}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14042","reported_seq":14178,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:49.995071+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:49.995071+0000","last_peered":"2024-09-10T15:58:49.995071+0000","last_clean":"2024-09-10T15:58:49.995071+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:49.995071+0000","last_undegraded":"2024-09-10T15:58:49.995071+0000","last_fullsized":"2024-09-10T15:58:49.995071+0000","mapping_epoch":35,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28147712,"num_objects":6872,"num_object_clones":0,"num_object_copies":13744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10801,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14076","reported_seq":22208,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:50.217902+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:50.217902+0000","last_peered":"2024-09-10T15:58:50.217902+0000","last_clean":"2024-09-10T15:58:50.217902+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:50.217902+0000","last_undegraded":"2024-09-10T15:58:50.217902+0000","last_fullsized":"2024-09-10T15:58:50.217902+0000","mapping_epoch":45,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27209728,"num_objects":6643,"num_object_clones":0,"num_object_copies":13286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12053,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'13876","reported_seq":14660,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:49.934698+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:49.934698+0000","last_peered":"2024-09-10T15:58:49.934698+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:49.934698+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"40'3800","ondisk_log_start":"40'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27451392,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10740,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3165},{"shards":"0,2","objects":280}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'13953","reported_seq":16398,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.376075+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:50.376075+0000","last_peered":"2024-09-10T15:58:50.376075+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:50.376075+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27983872,"num_objects":6832,"num_object_clones":0,"num_object_copies":13664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2433,"num_objects_misplaced":244,"num_objects_unfound":0,"num_objects_dirty":6832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11927,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1157,"num_bytes_recovered":4210688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":244},{"shards":"2","objects":4155}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13769","reported_seq":14158,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.293634+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:50.293634+0000","last_peered":"2024-09-10T15:58:50.293634+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:50.293634+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"40'3700","ondisk_log_start":"40'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27389952,"num_objects":6687,"num_object_clones":0,"num_object_copies":13374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11638,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":290},{"shards":"2","objects":2987}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'13976","reported_seq":14760,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.038258+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:50.038258+0000","last_peered":"2024-09-10T15:58:50.038258+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:50.038258+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28172288,"num_objects":6878,"num_object_clones":0,"num_object_copies":13756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11168,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3371},{"shards":"0,2","objects":285}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'13974","reported_seq":14689,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:49.951035+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:49.951035+0000","last_peered":"2024-09-10T15:58:49.951035+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:49.951035+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"41'3900","ondisk_log_start":"41'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10392,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3100},{"shards":"0,2","objects":306}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'13953","reported_seq":14722,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.048444+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:50.048444+0000","last_peered":"2024-09-10T15:58:50.048444+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:50.048444+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27906048,"num_objects":6813,"num_object_clones":0,"num_object_copies":13626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10559,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3248},{"shards":"0,2","objects":309}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14260","reported_seq":14944,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:50.050248+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:50.050248+0000","last_peered":"2024-09-10T15:58:50.050248+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:50.050248+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28008931,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3736,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11079,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3106},{"shards":"0,2","objects":298}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":443105787,"num_objects":108182,"num_object_clones":0,"num_object_copies":216364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34955,"num_objects_misplaced":244,"num_objects_unfound":0,"num_objects_dirty":108182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":184868,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11449,"num_bytes_recovered":36413440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1065492480,"data_stored":1065415689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626501,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637976,"kb_used_data":313416,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93733864,"statfs":{"total":96636764160,"available":95983476736,"internally_reserved":0,"allocated":320937984,"data_stored":320599606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332328622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,4,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149784,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":776916,"kb_used_data":388480,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93594924,"statfs":{"total":96636764160,"available":95841202176,"internally_reserved":0,"allocated":397803520,"data_stored":397527022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":397733549},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,1,6,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651960,"kb_used_data":341284,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93719880,"statfs":{"total":96636764160,"available":95969157120,"internally_reserved":0,"allocated":349474816,"data_stored":349194246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,0,0,0,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":396070912,"data_stored":396067299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320487424,"data_stored":320421931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348934144,"data_stored":348926459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:56.019 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:56.587 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:56.590 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:56.616 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-10T15:58:56.551230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":433877518,"num_objects":105931,"num_object_clones":0,"num_object_copies":211862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34866,"num_objects_misplaced":101,"num_objects_unfound":0,"num_objects_dirty":105931,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187123,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11683,"num_bytes_recovered":36995091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2068124,"kb_used_data":1030568,"kb_used_omap":12,"kb_used_meta":1037491,"kb_avail":281047396,"statfs":{"total":289910292480,"available":287792533504,"internally_reserved":0,"allocated":1055301632,"data_stored":1054406186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1062390789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,5,0,3,8,0,13],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27906048,"num_objects":-6813,"num_object_clones":0,"num_object_copies":-13626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-244,"num_objects_misplaced":-444,"num_objects_unfound":0,"num_objects_dirty":-6813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6813,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":688,"num_bytes_recovered":1658880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006239"},"pg_stats":[{"pgid":"2.f","version":"48'14194","reported_seq":14973,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.901742+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:54.901742+0000","last_peered":"2024-09-10T15:58:54.901742+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:54.901742+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14194,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2952},{"shards":"0,2","objects":164}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14350","reported_seq":14462,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:55.320721+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:55.320721+0000","last_peered":"2024-09-10T15:58:55.320721+0000","last_clean":"2024-09-10T15:58:55.320721+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:55.320721+0000","last_undegraded":"2024-09-10T15:58:55.320721+0000","last_fullsized":"2024-09-10T15:58:55.320721+0000","mapping_epoch":36,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541504,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12686,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13976","reported_seq":14445,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:55.337422+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:55.337422+0000","last_peered":"2024-09-10T15:58:55.337422+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:55.337422+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27250712,"num_objects":6654,"num_object_clones":0,"num_object_copies":13308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3633,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6654,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12244,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":141},{"shards":"2","objects":3022}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14068","reported_seq":14152,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:55.364504+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:55.364504+0000","last_peered":"2024-09-10T15:58:55.364504+0000","last_clean":"2024-09-10T15:58:55.364504+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:55.364504+0000","last_undegraded":"2024-09-10T15:58:55.364504+0000","last_fullsized":"2024-09-10T15:58:55.364504+0000","mapping_epoch":36,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27090944,"num_objects":6614,"num_object_clones":0,"num_object_copies":13228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14068,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13874","reported_seq":14006,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.967539+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:54.967539+0000","last_peered":"2024-09-10T15:58:54.967539+0000","last_clean":"2024-09-10T15:58:54.967539+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:54.967539+0000","last_undegraded":"2024-09-10T15:58:54.967539+0000","last_fullsized":"2024-09-10T15:58:54.967539+0000","mapping_epoch":35,"log_start":"41'3800","ondisk_log_start":"41'3800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26447872,"num_objects":6457,"num_object_clones":0,"num_object_copies":12914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10945,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14144","reported_seq":22037,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:55.326507+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:55.326507+0000","last_peered":"2024-09-10T15:58:55.326507+0000","last_clean":"2024-09-10T15:58:55.326507+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:55.326507+0000","last_undegraded":"2024-09-10T15:58:55.326507+0000","last_fullsized":"2024-09-10T15:58:55.326507+0000","mapping_epoch":47,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27537408,"num_objects":6723,"num_object_clones":0,"num_object_copies":13446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10862,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13846","reported_seq":14587,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.899201+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:54.899201+0000","last_peered":"2024-09-10T15:58:54.899201+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:54.899201+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"41'3800","ondisk_log_start":"41'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26378240,"num_objects":6440,"num_object_clones":0,"num_object_copies":12880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10496,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2904},{"shards":"0,2","objects":134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14195","reported_seq":14331,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.927876+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:54.927876+0000","last_peered":"2024-09-10T15:58:54.927876+0000","last_clean":"2024-09-10T15:58:54.927876+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:54.927876+0000","last_undegraded":"2024-09-10T15:58:54.927876+0000","last_fullsized":"2024-09-10T15:58:54.927876+0000","mapping_epoch":35,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521024,"num_objects":6719,"num_object_clones":0,"num_object_copies":13438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10954,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14215","reported_seq":22346,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:55.243934+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:55.243934+0000","last_peered":"2024-09-10T15:58:55.243934+0000","last_clean":"2024-09-10T15:58:55.243934+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:55.243934+0000","last_undegraded":"2024-09-10T15:58:55.243934+0000","last_fullsized":"2024-09-10T15:58:55.243934+0000","mapping_epoch":45,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26640384,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12192,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14012","reported_seq":14808,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.853532+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:54.853532+0000","last_peered":"2024-09-10T15:58:54.853532+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:54.853532+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26894336,"num_objects":6566,"num_object_clones":0,"num_object_copies":13132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10876,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3035},{"shards":"0,2","objects":144}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14095","reported_seq":16824,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:55.356993+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:55.356993+0000","last_peered":"2024-09-10T15:58:55.356993+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:55.356993+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27402240,"num_objects":6690,"num_object_clones":0,"num_object_copies":13380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2434,"num_objects_misplaced":101,"num_objects_unfound":0,"num_objects_dirty":6690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12069,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1299,"num_bytes_recovered":4792320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":101},{"shards":"2","objects":4155}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13901","reported_seq":14301,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:55.364096+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:55.364096+0000","last_peered":"2024-09-10T15:58:55.364096+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:55.364096+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26849280,"num_objects":6555,"num_object_clones":0,"num_object_copies":13110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11770,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":158},{"shards":"2","objects":2860}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14123","reported_seq":14938,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.864735+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:54.864735+0000","last_peered":"2024-09-10T15:58:54.864735+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:54.864735+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27570176,"num_objects":6731,"num_object_clones":0,"num_object_copies":13462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11315,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3240},{"shards":"0,2","objects":138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14114","reported_seq":14843,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.813987+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:54.813987+0000","last_peered":"2024-09-10T15:58:54.813987+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:54.813987+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27049984,"num_objects":6604,"num_object_clones":0,"num_object_copies":13208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3640,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10532,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2967},{"shards":"0,2","objects":166}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14099","reported_seq":14896,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.777932+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:54.777932+0000","last_peered":"2024-09-10T15:58:54.777932+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:54.777932+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27308032,"num_objects":6667,"num_object_clones":0,"num_object_copies":13334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3553,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10705,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3116},{"shards":"0,2","objects":163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14394","reported_seq":15106,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.772159+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:54.772159+0000","last_peered":"2024-09-10T15:58:54.772159+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:54.772159+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27460067,"num_objects":6705,"num_object_clones":0,"num_object_copies":13410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3722,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11213,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2986},{"shards":"0,2","objects":164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":433877499,"num_objects":105929,"num_object_clones":0,"num_object_copies":211858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34866,"num_objects_misplaced":101,"num_objects_unfound":0,"num_objects_dirty":105929,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187121,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11681,"num_bytes_recovered":36995072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1052753920,"data_stored":1052677129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160900,"ondisk_log_size":160900,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626503,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640980,"kb_used_data":312260,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93730860,"statfs":{"total":96636764160,"available":95980400640,"internally_reserved":0,"allocated":319754240,"data_stored":319415862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336588462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149785,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774104,"kb_used_data":381508,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93597736,"statfs":{"total":96636764160,"available":95844081664,"internally_reserved":0,"allocated":390664192,"data_stored":390387694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":401993389},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,3,8,0,13],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653040,"kb_used_data":336800,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93718800,"statfs":{"total":96636764160,"available":95968051200,"internally_reserved":0,"allocated":344883200,"data_stored":344602630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":389099520,"data_stored":389095907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319303680,"data_stored":319238187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344350720,"data_stored":344343035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:58:56.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:58:56.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'14194", 'reported_seq': 14973, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:54.901742+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:58:54.901742+0000', 'last_peered': '2024-09-10T15:58:54.901742+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:58:54.901742+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26935296, 'num_objects': 6576, 'num_object_clones': 0, 'num_object_copies': 13152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3624, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6576, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14194, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 349, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2952}, {'shards': '0,2', 'objects': 164}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:56.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:58:56.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'13976", 'reported_seq': 14445, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:55.337422+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:58:55.337422+0000', 'last_peered': '2024-09-10T15:58:55.337422+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:58:55.337422+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'3900", 'ondisk_log_start': "44'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27250712, 'num_objects': 6654, 'num_object_clones': 0, 'num_object_copies': 13308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3633, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6654, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12244, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 170, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 141}, {'shards': '2', 'objects': 3022}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:56.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:58:56.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'13846", 'reported_seq': 14587, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:54.899201+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:58:54.899201+0000', 'last_peered': '2024-09-10T15:58:54.899201+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:58:54.899201+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "41'3800", 'ondisk_log_start': "41'3800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26378240, 'num_objects': 6440, 'num_object_clones': 0, 'num_object_copies': 12880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6440, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10496, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 299, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2904}, {'shards': '0,2', 'objects': 134}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:58:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'14012", 'reported_seq': 14808, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:54.853532+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:58:54.853532+0000', 'last_peered': '2024-09-10T15:58:54.853532+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:58:54.853532+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'4000", 'ondisk_log_start': "44'4000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26894336, 'num_objects': 6566, 'num_object_clones': 0, 'num_object_copies': 13132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3533, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10876, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 332, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3035}, {'shards': '0,2', 'objects': 144}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:58:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'14095", 'reported_seq': 16824, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:55.356993+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:58:55.356993+0000', 'last_peered': '2024-09-10T15:58:55.356993+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:58:55.356993+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'4000", 'ondisk_log_start': "44'4000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27402240, 'num_objects': 6690, 'num_object_clones': 0, 'num_object_copies': 13380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2434, 'num_objects_misplaced': 101, 'num_objects_unfound': 0, 'num_objects_dirty': 6690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12069, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1299, 'num_bytes_recovered': 4792320, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 101}, {'shards': '2', 'objects': 4155}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:58:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'13901", 'reported_seq': 14301, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:55.364096+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:58:55.364096+0000', 'last_peered': '2024-09-10T15:58:55.364096+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:58:55.364096+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'3900", 'ondisk_log_start': "44'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26849280, 'num_objects': 6555, 'num_object_clones': 0, 'num_object_copies': 13110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6555, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11770, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 134, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 158}, {'shards': '2', 'objects': 2860}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:58:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:58:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'14123", 'reported_seq': 14938, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:54.864735+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:58:54.864735+0000', 'last_peered': '2024-09-10T15:58:54.864735+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:58:54.864735+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27570176, 'num_objects': 6731, 'num_object_clones': 0, 'num_object_copies': 13462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3492, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6731, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11315, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3240}, {'shards': '0,2', 'objects': 138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:58:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'14114", 'reported_seq': 14843, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:54.813987+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:58:54.813987+0000', 'last_peered': '2024-09-10T15:58:54.813987+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:58:54.813987+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27049984, 'num_objects': 6604, 'num_object_clones': 0, 'num_object_copies': 13208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3640, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6604, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10532, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 280, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2967}, {'shards': '0,2', 'objects': 166}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:56.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:58:56.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'14099", 'reported_seq': 14896, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:54.777932+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:58:54.777932+0000', 'last_peered': '2024-09-10T15:58:54.777932+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:58:54.777932+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'4000", 'ondisk_log_start': "44'4000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27308032, 'num_objects': 6667, 'num_object_clones': 0, 'num_object_copies': 13334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3553, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6667, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10705, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3116}, {'shards': '0,2', 'objects': 163}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:56.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:58:56.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'14394", 'reported_seq': 15106, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:58:54.772159+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:58:54.772159+0000', 'last_peered': '2024-09-10T15:58:54.772159+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:58:54.772159+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'4300", 'ondisk_log_start': "44'4300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27460067, 'num_objects': 6705, 'num_object_clones': 0, 'num_object_copies': 13410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3722, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6705, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11213, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 284, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2986}, {'shards': '0,2', 'objects': 164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:58:56.620 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:58:57.234 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:57.234 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34866/211862 objects degraded (16.457%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":463,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":105931,"data_bytes":433877518,"bytes_used":2117758976,"bytes_avail":287792533504,"bytes_total":289910292480,"degraded_objects":34866,"degraded_total":211862,"degraded_ratio":0.1645693895082648,"misplaced_objects":101,"misplaced_total":211862,"misplaced_ratio":0.00047672541560072121,"recovering_objects_per_sec":57,"recovering_bytes_per_sec":138168,"recovering_keys_per_sec":0,"num_objects_recovered":688,"num_bytes_recovered":1658880,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":567},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:58:57.257 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:58:57.258 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:58:57.800 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:58:57.801 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:58:57.822 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-10T15:58:56.551230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":433877518,"num_objects":105931,"num_object_clones":0,"num_object_copies":211862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34866,"num_objects_misplaced":101,"num_objects_unfound":0,"num_objects_dirty":105931,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187123,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11683,"num_bytes_recovered":36995091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2068124,"kb_used_data":1030568,"kb_used_omap":12,"kb_used_meta":1037491,"kb_avail":281047396,"statfs":{"total":289910292480,"available":287792533504,"internally_reserved":0,"allocated":1055301632,"data_stored":1054406186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1062390789},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,5,0,3,8,0,13],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27906048,"num_objects":-6813,"num_object_clones":0,"num_object_copies":-13626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-244,"num_objects_misplaced":-444,"num_objects_unfound":0,"num_objects_dirty":-6813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6813,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":688,"num_bytes_recovered":1658880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006239"},"pg_stats":[{"pgid":"2.f","version":"48'14194","reported_seq":14973,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.901742+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:58:54.901742+0000","last_peered":"2024-09-10T15:58:54.901742+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:58:54.901742+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3624,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14194,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2952},{"shards":"0,2","objects":164}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14350","reported_seq":14462,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:55.320721+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:58:55.320721+0000","last_peered":"2024-09-10T15:58:55.320721+0000","last_clean":"2024-09-10T15:58:55.320721+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:58:55.320721+0000","last_undegraded":"2024-09-10T15:58:55.320721+0000","last_fullsized":"2024-09-10T15:58:55.320721+0000","mapping_epoch":36,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541504,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12686,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'13976","reported_seq":14445,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:55.337422+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:58:55.337422+0000","last_peered":"2024-09-10T15:58:55.337422+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:58:55.337422+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27250712,"num_objects":6654,"num_object_clones":0,"num_object_copies":13308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3633,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6654,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12244,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":141},{"shards":"2","objects":3022}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14068","reported_seq":14152,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:55.364504+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:58:55.364504+0000","last_peered":"2024-09-10T15:58:55.364504+0000","last_clean":"2024-09-10T15:58:55.364504+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:58:55.364504+0000","last_undegraded":"2024-09-10T15:58:55.364504+0000","last_fullsized":"2024-09-10T15:58:55.364504+0000","mapping_epoch":36,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27090944,"num_objects":6614,"num_object_clones":0,"num_object_copies":13228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14068,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'13874","reported_seq":14006,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.967539+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:58:54.967539+0000","last_peered":"2024-09-10T15:58:54.967539+0000","last_clean":"2024-09-10T15:58:54.967539+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:58:54.967539+0000","last_undegraded":"2024-09-10T15:58:54.967539+0000","last_fullsized":"2024-09-10T15:58:54.967539+0000","mapping_epoch":35,"log_start":"41'3800","ondisk_log_start":"41'3800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26447872,"num_objects":6457,"num_object_clones":0,"num_object_copies":12914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10945,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14144","reported_seq":22037,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:55.326507+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:58:55.326507+0000","last_peered":"2024-09-10T15:58:55.326507+0000","last_clean":"2024-09-10T15:58:55.326507+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:58:55.326507+0000","last_undegraded":"2024-09-10T15:58:55.326507+0000","last_fullsized":"2024-09-10T15:58:55.326507+0000","mapping_epoch":47,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27537408,"num_objects":6723,"num_object_clones":0,"num_object_copies":13446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10862,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13846","reported_seq":14587,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.899201+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:58:54.899201+0000","last_peered":"2024-09-10T15:58:54.899201+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:58:54.899201+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"41'3800","ondisk_log_start":"41'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26378240,"num_objects":6440,"num_object_clones":0,"num_object_copies":12880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10496,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2904},{"shards":"0,2","objects":134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14195","reported_seq":14331,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:54.927876+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:58:54.927876+0000","last_peered":"2024-09-10T15:58:54.927876+0000","last_clean":"2024-09-10T15:58:54.927876+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:58:54.927876+0000","last_undegraded":"2024-09-10T15:58:54.927876+0000","last_fullsized":"2024-09-10T15:58:54.927876+0000","mapping_epoch":35,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521024,"num_objects":6719,"num_object_clones":0,"num_object_copies":13438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10954,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14215","reported_seq":22346,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:55.243934+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:58:55.243934+0000","last_peered":"2024-09-10T15:58:55.243934+0000","last_clean":"2024-09-10T15:58:55.243934+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:58:55.243934+0000","last_undegraded":"2024-09-10T15:58:55.243934+0000","last_fullsized":"2024-09-10T15:58:55.243934+0000","mapping_epoch":45,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26640384,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12192,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14012","reported_seq":14808,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.853532+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:58:54.853532+0000","last_peered":"2024-09-10T15:58:54.853532+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:58:54.853532+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26894336,"num_objects":6566,"num_object_clones":0,"num_object_copies":13132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10876,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3035},{"shards":"0,2","objects":144}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14095","reported_seq":16824,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:55.356993+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:58:55.356993+0000","last_peered":"2024-09-10T15:58:55.356993+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:58:55.356993+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27402240,"num_objects":6690,"num_object_clones":0,"num_object_copies":13380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2434,"num_objects_misplaced":101,"num_objects_unfound":0,"num_objects_dirty":6690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12069,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1299,"num_bytes_recovered":4792320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":101},{"shards":"2","objects":4155}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'13901","reported_seq":14301,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:55.364096+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:58:55.364096+0000","last_peered":"2024-09-10T15:58:55.364096+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:58:55.364096+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26849280,"num_objects":6555,"num_object_clones":0,"num_object_copies":13110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11770,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":158},{"shards":"2","objects":2860}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14123","reported_seq":14938,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.864735+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:58:54.864735+0000","last_peered":"2024-09-10T15:58:54.864735+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:58:54.864735+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27570176,"num_objects":6731,"num_object_clones":0,"num_object_copies":13462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11315,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3240},{"shards":"0,2","objects":138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14114","reported_seq":14843,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.813987+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:58:54.813987+0000","last_peered":"2024-09-10T15:58:54.813987+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:58:54.813987+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27049984,"num_objects":6604,"num_object_clones":0,"num_object_copies":13208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3640,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10532,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2967},{"shards":"0,2","objects":166}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14099","reported_seq":14896,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.777932+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:58:54.777932+0000","last_peered":"2024-09-10T15:58:54.777932+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:58:54.777932+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27308032,"num_objects":6667,"num_object_clones":0,"num_object_copies":13334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3553,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10705,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3116},{"shards":"0,2","objects":163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14394","reported_seq":15106,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:58:54.772159+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:58:54.772159+0000","last_peered":"2024-09-10T15:58:54.772159+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:58:54.772159+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27460067,"num_objects":6705,"num_object_clones":0,"num_object_copies":13410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3722,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11213,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2986},{"shards":"0,2","objects":164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":433877499,"num_objects":105929,"num_object_clones":0,"num_object_copies":211858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34866,"num_objects_misplaced":101,"num_objects_unfound":0,"num_objects_dirty":105929,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187121,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11681,"num_bytes_recovered":36995072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1052753920,"data_stored":1052677129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160900,"ondisk_log_size":160900,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626503,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640980,"kb_used_data":312260,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93730860,"statfs":{"total":96636764160,"available":95980400640,"internally_reserved":0,"allocated":319754240,"data_stored":319415862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336588462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149785,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774104,"kb_used_data":381508,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93597736,"statfs":{"total":96636764160,"available":95844081664,"internally_reserved":0,"allocated":390664192,"data_stored":390387694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":401993389},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,3,8,0,13],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653040,"kb_used_data":336800,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93718800,"statfs":{"total":96636764160,"available":95968051200,"internally_reserved":0,"allocated":344883200,"data_stored":344602630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":389099520,"data_stored":389095907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319303680,"data_stored":319238187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344350720,"data_stored":344343035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:00.823 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:01.385 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:01.386 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-10T15:59:00.552432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":424727054,"num_objects":103697,"num_object_clones":0,"num_object_copies":207394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103697,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":189357,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11930,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2060152,"kb_used_data":1018440,"kb_used_omap":12,"kb_used_meta":1041651,"kb_avail":281055368,"statfs":{"total":289910292480,"available":287800696832,"internally_reserved":0,"allocated":1042882560,"data_stored":1041987114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1066650629},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,4,3,4,9,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27598848,"num_objects":-6738,"num_object_clones":0,"num_object_copies":-13476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-371,"num_objects_misplaced":-402,"num_objects_unfound":0,"num_objects_dirty":-6738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6738,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004435"},"pg_stats":[{"pgid":"2.f","version":"48'14344","reported_seq":15140,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.027178+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:00.027178+0000","last_peered":"2024-09-10T15:59:00.027178+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:00.027178+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26320896,"num_objects":6426,"num_object_clones":0,"num_object_copies":12852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14344,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":357,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2810},{"shards":"0,2","objects":14}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14485","reported_seq":14597,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.148204+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:00.148204+0000","last_peered":"2024-09-10T15:59:00.148204+0000","last_clean":"2024-09-10T15:59:00.148204+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:00.148204+0000","last_undegraded":"2024-09-10T15:59:00.148204+0000","last_fullsized":"2024-09-10T15:59:00.148204+0000","mapping_epoch":36,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26988544,"num_objects":6589,"num_object_clones":0,"num_object_copies":13178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12821,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14108","reported_seq":14579,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.127176+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:00.127176+0000","last_peered":"2024-09-10T15:59:00.127176+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:00.127176+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710040,"num_objects":6522,"num_object_clones":0,"num_object_copies":13044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6522,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12376,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":9},{"shards":"2","objects":2891}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14213","reported_seq":14297,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.144992+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:00.144992+0000","last_peered":"2024-09-10T15:59:00.144992+0000","last_clean":"2024-09-10T15:59:00.144992+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:00.144992+0000","last_undegraded":"2024-09-10T15:59:00.144992+0000","last_fullsized":"2024-09-10T15:59:00.144992+0000","mapping_epoch":36,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26497024,"num_objects":6469,"num_object_clones":0,"num_object_copies":12938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14213,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14008","reported_seq":14140,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.080569+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:00.080569+0000","last_peered":"2024-09-10T15:59:00.080569+0000","last_clean":"2024-09-10T15:59:00.080569+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:00.080569+0000","last_undegraded":"2024-09-10T15:59:00.080569+0000","last_fullsized":"2024-09-10T15:59:00.080569+0000","mapping_epoch":35,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25899008,"num_objects":6323,"num_object_clones":0,"num_object_copies":12646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11079,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14292","reported_seq":22184,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.372484+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:00.372484+0000","last_peered":"2024-09-10T15:59:00.372484+0000","last_clean":"2024-09-10T15:59:00.372484+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:00.372484+0000","last_undegraded":"2024-09-10T15:59:00.372484+0000","last_fullsized":"2024-09-10T15:59:00.372484+0000","mapping_epoch":47,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26931200,"num_objects":6575,"num_object_clones":0,"num_object_copies":13150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11010,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13965","reported_seq":14727,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.067416+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:00.067416+0000","last_peered":"2024-09-10T15:59:00.067416+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:00.067416+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25890816,"num_objects":6321,"num_object_clones":0,"num_object_copies":12642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10615,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2795},{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14358","reported_seq":14494,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.099920+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:00.099920+0000","last_peered":"2024-09-10T15:59:00.099920+0000","last_clean":"2024-09-10T15:59:00.099920+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:00.099920+0000","last_undegraded":"2024-09-10T15:59:00.099920+0000","last_fullsized":"2024-09-10T15:59:00.099920+0000","mapping_epoch":35,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11117,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14342","reported_seq":22474,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.139852+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:00.139852+0000","last_peered":"2024-09-10T15:59:00.139852+0000","last_clean":"2024-09-10T15:59:00.139852+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:00.139852+0000","last_undegraded":"2024-09-10T15:59:00.139852+0000","last_fullsized":"2024-09-10T15:59:00.139852+0000","mapping_epoch":45,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26120192,"num_objects":6377,"num_object_clones":0,"num_object_copies":12754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14141","reported_seq":14951,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.046005+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:00.046005+0000","last_peered":"2024-09-10T15:59:00.046005+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:00.046005+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11005,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2913},{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14217","reported_seq":17269,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.385080+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:00.385080+0000","last_peered":"2024-09-10T15:59:00.385080+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:00.385080+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26902528,"num_objects":6568,"num_object_clones":0,"num_object_copies":13136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12191,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4195}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14053","reported_seq":14471,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.379255+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:00.379255+0000","last_peered":"2024-09-10T15:59:00.379255+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:00.379255+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226688,"num_objects":6403,"num_object_clones":0,"num_object_copies":12806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11922,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":6},{"shards":"2","objects":2717}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14252","reported_seq":15084,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.074250+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:00.074250+0000","last_peered":"2024-09-10T15:59:00.074250+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:00.074250+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27041792,"num_objects":6602,"num_object_clones":0,"num_object_copies":13204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11444,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3119},{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14268","reported_seq":15017,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.053472+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:00.053472+0000","last_peered":"2024-09-10T15:59:00.053472+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:00.053472+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26419200,"num_objects":6450,"num_object_clones":0,"num_object_copies":12900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10686,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2823},{"shards":"0,2","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14250","reported_seq":15068,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.082894+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:00.082894+0000","last_peered":"2024-09-10T15:59:00.082894+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:00.082894+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10856,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2977},{"shards":"0,2","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14538","reported_seq":15290,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.073902+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:00.073902+0000","last_peered":"2024-09-10T15:59:00.073902+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:00.073902+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26870243,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11357,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2862},{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":424727035,"num_objects":103695,"num_object_clones":0,"num_object_copies":207390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":189355,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11928,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1039986688,"data_stored":1039909897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626504,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631492,"kb_used_data":311092,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93740348,"statfs":{"total":96636764160,"available":95990116352,"internally_reserved":0,"allocated":318558208,"data_stored":318219830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328068782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149786,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":775940,"kb_used_data":375028,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93595900,"statfs":{"total":96636764160,"available":95842201600,"internally_reserved":0,"allocated":384028672,"data_stored":383752174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":410513069},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,4,9,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652720,"kb_used_data":332320,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93719120,"statfs":{"total":96636764160,"available":95968378880,"internally_reserved":0,"allocated":340295680,"data_stored":340015110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382083072,"data_stored":382079459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318095360,"data_stored":318029867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":339808256,"data_stored":339800571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:01.416 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:01.956 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:01.958 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:01.981 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-10T15:59:00.552432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":424727054,"num_objects":103697,"num_object_clones":0,"num_object_copies":207394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103697,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":189357,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11930,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2060152,"kb_used_data":1018440,"kb_used_omap":12,"kb_used_meta":1041651,"kb_avail":281055368,"statfs":{"total":289910292480,"available":287800696832,"internally_reserved":0,"allocated":1042882560,"data_stored":1041987114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1066650629},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,4,3,4,9,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27598848,"num_objects":-6738,"num_object_clones":0,"num_object_copies":-13476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-371,"num_objects_misplaced":-402,"num_objects_unfound":0,"num_objects_dirty":-6738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6738,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":773,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004435"},"pg_stats":[{"pgid":"2.f","version":"48'14344","reported_seq":15140,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.027178+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:00.027178+0000","last_peered":"2024-09-10T15:59:00.027178+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:00.027178+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26320896,"num_objects":6426,"num_object_clones":0,"num_object_copies":12852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14344,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":357,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2810},{"shards":"0,2","objects":14}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14485","reported_seq":14597,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.148204+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:00.148204+0000","last_peered":"2024-09-10T15:59:00.148204+0000","last_clean":"2024-09-10T15:59:00.148204+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:00.148204+0000","last_undegraded":"2024-09-10T15:59:00.148204+0000","last_fullsized":"2024-09-10T15:59:00.148204+0000","mapping_epoch":36,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26988544,"num_objects":6589,"num_object_clones":0,"num_object_copies":13178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12821,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14108","reported_seq":14579,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.127176+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:00.127176+0000","last_peered":"2024-09-10T15:59:00.127176+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:00.127176+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710040,"num_objects":6522,"num_object_clones":0,"num_object_copies":13044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6522,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12376,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":9},{"shards":"2","objects":2891}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14213","reported_seq":14297,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.144992+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:00.144992+0000","last_peered":"2024-09-10T15:59:00.144992+0000","last_clean":"2024-09-10T15:59:00.144992+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:00.144992+0000","last_undegraded":"2024-09-10T15:59:00.144992+0000","last_fullsized":"2024-09-10T15:59:00.144992+0000","mapping_epoch":36,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26497024,"num_objects":6469,"num_object_clones":0,"num_object_copies":12938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14213,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14008","reported_seq":14140,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.080569+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:00.080569+0000","last_peered":"2024-09-10T15:59:00.080569+0000","last_clean":"2024-09-10T15:59:00.080569+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:00.080569+0000","last_undegraded":"2024-09-10T15:59:00.080569+0000","last_fullsized":"2024-09-10T15:59:00.080569+0000","mapping_epoch":35,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25899008,"num_objects":6323,"num_object_clones":0,"num_object_copies":12646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11079,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14292","reported_seq":22184,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.372484+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:00.372484+0000","last_peered":"2024-09-10T15:59:00.372484+0000","last_clean":"2024-09-10T15:59:00.372484+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:00.372484+0000","last_undegraded":"2024-09-10T15:59:00.372484+0000","last_fullsized":"2024-09-10T15:59:00.372484+0000","mapping_epoch":47,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26931200,"num_objects":6575,"num_object_clones":0,"num_object_copies":13150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11010,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13965","reported_seq":14727,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.067416+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:00.067416+0000","last_peered":"2024-09-10T15:59:00.067416+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:00.067416+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25890816,"num_objects":6321,"num_object_clones":0,"num_object_copies":12642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10615,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2795},{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14358","reported_seq":14494,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.099920+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:00.099920+0000","last_peered":"2024-09-10T15:59:00.099920+0000","last_clean":"2024-09-10T15:59:00.099920+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:00.099920+0000","last_undegraded":"2024-09-10T15:59:00.099920+0000","last_fullsized":"2024-09-10T15:59:00.099920+0000","mapping_epoch":35,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11117,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14342","reported_seq":22474,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.139852+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:00.139852+0000","last_peered":"2024-09-10T15:59:00.139852+0000","last_clean":"2024-09-10T15:59:00.139852+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:00.139852+0000","last_undegraded":"2024-09-10T15:59:00.139852+0000","last_fullsized":"2024-09-10T15:59:00.139852+0000","mapping_epoch":45,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26120192,"num_objects":6377,"num_object_clones":0,"num_object_copies":12754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14141","reported_seq":14951,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.046005+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:00.046005+0000","last_peered":"2024-09-10T15:59:00.046005+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:00.046005+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11005,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2913},{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14217","reported_seq":17269,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.385080+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:00.385080+0000","last_peered":"2024-09-10T15:59:00.385080+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:00.385080+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26902528,"num_objects":6568,"num_object_clones":0,"num_object_copies":13136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12191,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4195}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14053","reported_seq":14471,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.379255+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:00.379255+0000","last_peered":"2024-09-10T15:59:00.379255+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:00.379255+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226688,"num_objects":6403,"num_object_clones":0,"num_object_copies":12806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11922,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":6},{"shards":"2","objects":2717}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14252","reported_seq":15084,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.074250+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:00.074250+0000","last_peered":"2024-09-10T15:59:00.074250+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:00.074250+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27041792,"num_objects":6602,"num_object_clones":0,"num_object_copies":13204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11444,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3119},{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14268","reported_seq":15017,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.053472+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:00.053472+0000","last_peered":"2024-09-10T15:59:00.053472+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:00.053472+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26419200,"num_objects":6450,"num_object_clones":0,"num_object_copies":12900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10686,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2823},{"shards":"0,2","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14250","reported_seq":15068,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.082894+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:00.082894+0000","last_peered":"2024-09-10T15:59:00.082894+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:00.082894+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10856,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2977},{"shards":"0,2","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14538","reported_seq":15290,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.073902+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:00.073902+0000","last_peered":"2024-09-10T15:59:00.073902+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:00.073902+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26870243,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11357,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2862},{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":424727035,"num_objects":103695,"num_object_clones":0,"num_object_copies":207390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":189355,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11928,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1039986688,"data_stored":1039909897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626504,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631492,"kb_used_data":311092,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93740348,"statfs":{"total":96636764160,"available":95990116352,"internally_reserved":0,"allocated":318558208,"data_stored":318219830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328068782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149786,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":775940,"kb_used_data":375028,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93595900,"statfs":{"total":96636764160,"available":95842201600,"internally_reserved":0,"allocated":384028672,"data_stored":383752174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":410513069},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,4,9,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652720,"kb_used_data":332320,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93719120,"statfs":{"total":96636764160,"available":95968378880,"internally_reserved":0,"allocated":340295680,"data_stored":340015110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382083072,"data_stored":382079459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318095360,"data_stored":318029867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":339808256,"data_stored":339800571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:01.982 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:01.982 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'14344", 'reported_seq': 15140, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:00.027178+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:59:00.027178+0000', 'last_peered': '2024-09-10T15:59:00.027178+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:59:00.027178+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'4300", 'ondisk_log_start': "44'4300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26320896, 'num_objects': 6426, 'num_object_clones': 0, 'num_object_copies': 12852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3616, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14344, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 357, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2810}, {'shards': '0,2', 'objects': 14}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:01.982 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:01.983 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'14108", 'reported_seq': 14579, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:00.127176+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:59:00.127176+0000', 'last_peered': '2024-09-10T15:59:00.127176+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:59:00.127176+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26710040, 'num_objects': 6522, 'num_object_clones': 0, 'num_object_copies': 13044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3632, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6522, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12376, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 171, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 9}, {'shards': '2', 'objects': 2891}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:01.983 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:01.983 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'13965", 'reported_seq': 14727, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:00.067416+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:59:00.067416+0000', 'last_peered': '2024-09-10T15:59:00.067416+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:59:00.067416+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'3900", 'ondisk_log_start': "44'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25890816, 'num_objects': 6321, 'num_object_clones': 0, 'num_object_copies': 12642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6321, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10615, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2795}, {'shards': '0,2', 'objects': 15}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:01.983 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:01.983 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'14141", 'reported_seq': 14951, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:00.046005+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:59:00.046005+0000', 'last_peered': '2024-09-10T15:59:00.046005+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:59:00.046005+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26365952, 'num_objects': 6437, 'num_object_clones': 0, 'num_object_copies': 12874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6437, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11005, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 339, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2913}, {'shards': '0,2', 'objects': 15}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:01.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:01.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'14217", 'reported_seq': 17269, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:00.385080+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:59:00.385080+0000', 'last_peered': '2024-09-10T15:59:00.385080+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:59:00.385080+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26902528, 'num_objects': 6568, 'num_object_clones': 0, 'num_object_copies': 13136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6568, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12191, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1461, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4195}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:01.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:01.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'14053", 'reported_seq': 14471, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:00.379255+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:59:00.379255+0000', 'last_peered': '2024-09-10T15:59:00.379255+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:59:00.379255+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'4000", 'ondisk_log_start': "44'4000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26226688, 'num_objects': 6403, 'num_object_clones': 0, 'num_object_copies': 12806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6403, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11922, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 6}, {'shards': '2', 'objects': 2717}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:01.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:01.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'14252", 'reported_seq': 15084, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:00.074250+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:59:00.074250+0000', 'last_peered': '2024-09-10T15:59:00.074250+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:59:00.074250+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27041792, 'num_objects': 6602, 'num_object_clones': 0, 'num_object_copies': 13204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3484, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6602, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11444, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 350, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3119}, {'shards': '0,2', 'objects': 9}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:01.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:01.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'14268", 'reported_seq': 15017, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:00.053472+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:59:00.053472+0000', 'last_peered': '2024-09-10T15:59:00.053472+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:59:00.053472+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26419200, 'num_objects': 6450, 'num_object_clones': 0, 'num_object_copies': 12900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6450, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10686, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 290, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2823}, {'shards': '0,2', 'objects': 12}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:01.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:01.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'14250", 'reported_seq': 15068, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:00.082894+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:59:00.082894+0000', 'last_peered': '2024-09-10T15:59:00.082894+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:59:00.082894+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26689536, 'num_objects': 6516, 'num_object_clones': 0, 'num_object_copies': 13032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6516, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10856, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 339, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2977}, {'shards': '0,2', 'objects': 12}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:01.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:01.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'14538", 'reported_seq': 15290, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:00.073902+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:59:00.073902+0000', 'last_peered': '2024-09-10T15:59:00.073902+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:59:00.073902+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'4500", 'ondisk_log_start': "44'4500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26870243, 'num_objects': 6561, 'num_object_clones': 0, 'num_object_copies': 13122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3702, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6561, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11357, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 304, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2862}, {'shards': '0,2', 'objects': 20}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:01.986 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:02.645 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:02.645 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34720/207394 objects degraded (16.741%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":468,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":103697,"data_bytes":424727054,"bytes_used":2109595648,"bytes_avail":287800696832,"bytes_total":289910292480,"degraded_objects":34720,"degraded_total":207394,"degraded_ratio":0.16741082191384515,"recovering_objects_per_sec":64,"recovering_bytes_per_sec":133753,"recovering_keys_per_sec":0,"num_objects_recovered":773,"num_bytes_recovered":1605632,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":561},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:59:02.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:02.676 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:03.248 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:03.249 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:03.277 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-10T15:59:02.553128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":424727054,"num_objects":103697,"num_object_clones":0,"num_object_copies":207394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103697,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":189357,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11930,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2060152,"kb_used_data":1018440,"kb_used_omap":12,"kb_used_meta":1041651,"kb_avail":281055368,"statfs":{"total":289910292480,"available":287800696832,"internally_reserved":0,"allocated":1042882560,"data_stored":1041987114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1066650629},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,4,3,4,9,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18378752,"num_objects":-4487,"num_object_clones":0,"num_object_copies":-8974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-235,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":-4487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4487,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":479,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004581"},"pg_stats":[{"pgid":"2.f","version":"48'14344","reported_seq":15140,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.027178+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:00.027178+0000","last_peered":"2024-09-10T15:59:00.027178+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:00.027178+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26320896,"num_objects":6426,"num_object_clones":0,"num_object_copies":12852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14344,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":357,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2810},{"shards":"0,2","objects":14}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14485","reported_seq":14597,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.148204+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:00.148204+0000","last_peered":"2024-09-10T15:59:00.148204+0000","last_clean":"2024-09-10T15:59:00.148204+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:00.148204+0000","last_undegraded":"2024-09-10T15:59:00.148204+0000","last_fullsized":"2024-09-10T15:59:00.148204+0000","mapping_epoch":36,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26988544,"num_objects":6589,"num_object_clones":0,"num_object_copies":13178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12821,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14108","reported_seq":14579,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.127176+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:00.127176+0000","last_peered":"2024-09-10T15:59:00.127176+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:00.127176+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710040,"num_objects":6522,"num_object_clones":0,"num_object_copies":13044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6522,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12376,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":9},{"shards":"2","objects":2891}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14213","reported_seq":14297,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.144992+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:00.144992+0000","last_peered":"2024-09-10T15:59:00.144992+0000","last_clean":"2024-09-10T15:59:00.144992+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:00.144992+0000","last_undegraded":"2024-09-10T15:59:00.144992+0000","last_fullsized":"2024-09-10T15:59:00.144992+0000","mapping_epoch":36,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26497024,"num_objects":6469,"num_object_clones":0,"num_object_copies":12938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14213,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14008","reported_seq":14140,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.080569+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:00.080569+0000","last_peered":"2024-09-10T15:59:00.080569+0000","last_clean":"2024-09-10T15:59:00.080569+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:00.080569+0000","last_undegraded":"2024-09-10T15:59:00.080569+0000","last_fullsized":"2024-09-10T15:59:00.080569+0000","mapping_epoch":35,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25899008,"num_objects":6323,"num_object_clones":0,"num_object_copies":12646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11079,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14292","reported_seq":22184,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.372484+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:00.372484+0000","last_peered":"2024-09-10T15:59:00.372484+0000","last_clean":"2024-09-10T15:59:00.372484+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:00.372484+0000","last_undegraded":"2024-09-10T15:59:00.372484+0000","last_fullsized":"2024-09-10T15:59:00.372484+0000","mapping_epoch":47,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26931200,"num_objects":6575,"num_object_clones":0,"num_object_copies":13150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11010,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'13965","reported_seq":14727,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.067416+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:00.067416+0000","last_peered":"2024-09-10T15:59:00.067416+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:00.067416+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25890816,"num_objects":6321,"num_object_clones":0,"num_object_copies":12642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10615,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2795},{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14358","reported_seq":14494,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.099920+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:00.099920+0000","last_peered":"2024-09-10T15:59:00.099920+0000","last_clean":"2024-09-10T15:59:00.099920+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:00.099920+0000","last_undegraded":"2024-09-10T15:59:00.099920+0000","last_fullsized":"2024-09-10T15:59:00.099920+0000","mapping_epoch":35,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11117,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14342","reported_seq":22474,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:00.139852+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:00.139852+0000","last_peered":"2024-09-10T15:59:00.139852+0000","last_clean":"2024-09-10T15:59:00.139852+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:00.139852+0000","last_undegraded":"2024-09-10T15:59:00.139852+0000","last_fullsized":"2024-09-10T15:59:00.139852+0000","mapping_epoch":45,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26120192,"num_objects":6377,"num_object_clones":0,"num_object_copies":12754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14141","reported_seq":14951,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.046005+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:00.046005+0000","last_peered":"2024-09-10T15:59:00.046005+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:00.046005+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11005,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2913},{"shards":"0,2","objects":15}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14217","reported_seq":17269,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.385080+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:00.385080+0000","last_peered":"2024-09-10T15:59:00.385080+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:00.385080+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26902528,"num_objects":6568,"num_object_clones":0,"num_object_copies":13136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12191,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4195}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14053","reported_seq":14471,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.379255+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:00.379255+0000","last_peered":"2024-09-10T15:59:00.379255+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:00.379255+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226688,"num_objects":6403,"num_object_clones":0,"num_object_copies":12806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11922,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":6},{"shards":"2","objects":2717}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14252","reported_seq":15084,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.074250+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:00.074250+0000","last_peered":"2024-09-10T15:59:00.074250+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:00.074250+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27041792,"num_objects":6602,"num_object_clones":0,"num_object_copies":13204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11444,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3119},{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14268","reported_seq":15017,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.053472+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:00.053472+0000","last_peered":"2024-09-10T15:59:00.053472+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:00.053472+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26419200,"num_objects":6450,"num_object_clones":0,"num_object_copies":12900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10686,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2823},{"shards":"0,2","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14250","reported_seq":15068,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.082894+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:00.082894+0000","last_peered":"2024-09-10T15:59:00.082894+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:00.082894+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10856,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2977},{"shards":"0,2","objects":12}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14538","reported_seq":15290,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:00.073902+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:00.073902+0000","last_peered":"2024-09-10T15:59:00.073902+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:00.073902+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26870243,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11357,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2862},{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":424727035,"num_objects":103695,"num_object_clones":0,"num_object_copies":207390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":189355,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11928,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1039986688,"data_stored":1039909897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626504,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631492,"kb_used_data":311092,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93740348,"statfs":{"total":96636764160,"available":95990116352,"internally_reserved":0,"allocated":318558208,"data_stored":318219830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328068782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149786,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":775940,"kb_used_data":375028,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93595900,"statfs":{"total":96636764160,"available":95842201600,"internally_reserved":0,"allocated":384028672,"data_stored":383752174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":410513069},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,4,9,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652720,"kb_used_data":332320,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93719120,"statfs":{"total":96636764160,"available":95968378880,"internally_reserved":0,"allocated":340295680,"data_stored":340015110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382083072,"data_stored":382079459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318095360,"data_stored":318029867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":339808256,"data_stored":339800571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:06.279 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:06.875 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:06.876 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:06.903 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-10T15:59:06.554404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":415146510,"num_objects":101358,"num_object_clones":0,"num_object_copies":202716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191696,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12065,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2054828,"kb_used_data":999212,"kb_used_omap":12,"kb_used_meta":1055539,"kb_avail":281060692,"statfs":{"total":289910292480,"available":287806148608,"internally_reserved":0,"allocated":1023193088,"data_stored":1022297642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1080871941},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,7,2,1,11,7,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27959296,"num_objects":-6826,"num_object_clones":0,"num_object_copies":-13652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-370,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":-6826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6826,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":614,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003812"},"pg_stats":[{"pgid":"2.f","version":"48'14511","reported_seq":15315,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.063136+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:05.063136+0000","last_peered":"2024-09-10T15:59:05.063136+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:05.063136+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25636864,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14511,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2647}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14646","reported_seq":14758,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.424826+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:05.424826+0000","last_peered":"2024-09-10T15:59:05.424826+0000","last_clean":"2024-09-10T15:59:05.424826+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:05.424826+0000","last_undegraded":"2024-09-10T15:59:05.424826+0000","last_fullsized":"2024-09-10T15:59:05.424826+0000","mapping_epoch":36,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26329088,"num_objects":6428,"num_object_clones":0,"num_object_copies":12856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12982,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14262","reported_seq":14737,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.363700+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:05.363700+0000","last_peered":"2024-09-10T15:59:05.363700+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:05.363700+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26079256,"num_objects":6368,"num_object_clones":0,"num_object_copies":12736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6368,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12530,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2739}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14362","reported_seq":14446,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.400300+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:05.400300+0000","last_peered":"2024-09-10T15:59:05.400300+0000","last_clean":"2024-09-10T15:59:05.400300+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:05.400300+0000","last_undegraded":"2024-09-10T15:59:05.400300+0000","last_fullsized":"2024-09-10T15:59:05.400300+0000","mapping_epoch":36,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25886720,"num_objects":6320,"num_object_clones":0,"num_object_copies":12640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14362,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14143","reported_seq":14275,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.062265+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:05.062265+0000","last_peered":"2024-09-10T15:59:05.062265+0000","last_clean":"2024-09-10T15:59:05.062265+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:05.062265+0000","last_undegraded":"2024-09-10T15:59:05.062265+0000","last_fullsized":"2024-09-10T15:59:05.062265+0000","mapping_epoch":35,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11214,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14438","reported_seq":22330,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.385953+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:05.385953+0000","last_peered":"2024-09-10T15:59:05.385953+0000","last_clean":"2024-09-10T15:59:05.385953+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:05.385953+0000","last_undegraded":"2024-09-10T15:59:05.385953+0000","last_fullsized":"2024-09-10T15:59:05.385953+0000","mapping_epoch":47,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11156,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14101","reported_seq":14867,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.112805+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:05.112805+0000","last_peered":"2024-09-10T15:59:05.112805+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:05.112805+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10751,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14501","reported_seq":14637,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.069849+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:05.069849+0000","last_peered":"2024-09-10T15:59:05.069849+0000","last_clean":"2024-09-10T15:59:05.069849+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:05.069849+0000","last_undegraded":"2024-09-10T15:59:05.069849+0000","last_fullsized":"2024-09-10T15:59:05.069849+0000","mapping_epoch":35,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26267648,"num_objects":6413,"num_object_clones":0,"num_object_copies":12826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11260,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14506","reported_seq":22638,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.309747+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:05.309747+0000","last_peered":"2024-09-10T15:59:05.309747+0000","last_clean":"2024-09-10T15:59:05.309747+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:05.309747+0000","last_undegraded":"2024-09-10T15:59:05.309747+0000","last_fullsized":"2024-09-10T15:59:05.309747+0000","mapping_epoch":45,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25448448,"num_objects":6213,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12483,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14269","reported_seq":15089,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.108948+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:05.108948+0000","last_peered":"2024-09-10T15:59:05.108948+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:05.108948+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25841664,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11133,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2790}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14338","reported_seq":17607,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.393582+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:05.393582+0000","last_peered":"2024-09-10T15:59:05.393582+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:05.393582+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26406912,"num_objects":6447,"num_object_clones":0,"num_object_copies":12894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12312,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1569,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4182}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14188","reported_seq":14606,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.344551+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:05.344551+0000","last_peered":"2024-09-10T15:59:05.344551+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:05.344551+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25673728,"num_objects":6268,"num_object_clones":0,"num_object_copies":12536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12057,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2582}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14394","reported_seq":15237,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.004129+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:05.004129+0000","last_peered":"2024-09-10T15:59:05.004129+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:05.004129+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26460160,"num_objects":6460,"num_object_clones":0,"num_object_copies":12920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11586,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2982}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14430","reported_seq":15180,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.144279+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:05.144279+0000","last_peered":"2024-09-10T15:59:05.144279+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:05.144279+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25755648,"num_objects":6288,"num_object_clones":0,"num_object_copies":12576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10848,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2662}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14394","reported_seq":15221,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.127365+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:05.127365+0000","last_peered":"2024-09-10T15:59:05.127365+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:05.127365+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26099712,"num_objects":6372,"num_object_clones":0,"num_object_copies":12744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11000,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2836}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14690","reported_seq":15452,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.084376+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:05.084376+0000","last_peered":"2024-09-10T15:59:05.084376+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:05.084376+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247651,"num_objects":6409,"num_object_clones":0,"num_object_copies":12818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11509,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":415146491,"num_objects":101356,"num_object_clones":0,"num_object_copies":202712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191694,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12063,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1021083648,"data_stored":1021006857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626505,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647460,"kb_used_data":304836,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93724380,"statfs":{"total":96636764160,"available":95973765120,"internally_reserved":0,"allocated":312152064,"data_stored":311813686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350809774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,7,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149787,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":755116,"kb_used_data":366684,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93616724,"statfs":{"total":96636764160,"available":95863525376,"internally_reserved":0,"allocated":375484416,"data_stored":375207918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":397733549},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,0,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652252,"kb_used_data":327692,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93719588,"statfs":{"total":96636764160,"available":95968858112,"internally_reserved":0,"allocated":335556608,"data_stored":335276038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,1,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":374919168,"data_stored":374915555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311152640,"data_stored":311087147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335011840,"data_stored":335004155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:06.904 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:07.471 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:07.472 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:07.497 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-10T15:59:06.554404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":415146510,"num_objects":101358,"num_object_clones":0,"num_object_copies":202716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191696,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12065,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2054828,"kb_used_data":999212,"kb_used_omap":12,"kb_used_meta":1055539,"kb_avail":281060692,"statfs":{"total":289910292480,"available":287806148608,"internally_reserved":0,"allocated":1023193088,"data_stored":1022297642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1080871941},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,7,2,1,11,7,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27959296,"num_objects":-6826,"num_object_clones":0,"num_object_copies":-13652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-370,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":-6826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6826,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":614,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003812"},"pg_stats":[{"pgid":"2.f","version":"48'14511","reported_seq":15315,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.063136+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:05.063136+0000","last_peered":"2024-09-10T15:59:05.063136+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:05.063136+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25636864,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14511,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2647}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14646","reported_seq":14758,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.424826+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:05.424826+0000","last_peered":"2024-09-10T15:59:05.424826+0000","last_clean":"2024-09-10T15:59:05.424826+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:05.424826+0000","last_undegraded":"2024-09-10T15:59:05.424826+0000","last_fullsized":"2024-09-10T15:59:05.424826+0000","mapping_epoch":36,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26329088,"num_objects":6428,"num_object_clones":0,"num_object_copies":12856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12982,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14262","reported_seq":14737,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.363700+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:05.363700+0000","last_peered":"2024-09-10T15:59:05.363700+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:05.363700+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26079256,"num_objects":6368,"num_object_clones":0,"num_object_copies":12736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6368,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12530,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2739}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14362","reported_seq":14446,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.400300+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:05.400300+0000","last_peered":"2024-09-10T15:59:05.400300+0000","last_clean":"2024-09-10T15:59:05.400300+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:05.400300+0000","last_undegraded":"2024-09-10T15:59:05.400300+0000","last_fullsized":"2024-09-10T15:59:05.400300+0000","mapping_epoch":36,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25886720,"num_objects":6320,"num_object_clones":0,"num_object_copies":12640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14362,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14143","reported_seq":14275,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.062265+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:05.062265+0000","last_peered":"2024-09-10T15:59:05.062265+0000","last_clean":"2024-09-10T15:59:05.062265+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:05.062265+0000","last_undegraded":"2024-09-10T15:59:05.062265+0000","last_fullsized":"2024-09-10T15:59:05.062265+0000","mapping_epoch":35,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11214,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14438","reported_seq":22330,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.385953+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:05.385953+0000","last_peered":"2024-09-10T15:59:05.385953+0000","last_clean":"2024-09-10T15:59:05.385953+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:05.385953+0000","last_undegraded":"2024-09-10T15:59:05.385953+0000","last_fullsized":"2024-09-10T15:59:05.385953+0000","mapping_epoch":47,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11156,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14101","reported_seq":14867,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.112805+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:05.112805+0000","last_peered":"2024-09-10T15:59:05.112805+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:05.112805+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10751,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14501","reported_seq":14637,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.069849+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:05.069849+0000","last_peered":"2024-09-10T15:59:05.069849+0000","last_clean":"2024-09-10T15:59:05.069849+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:05.069849+0000","last_undegraded":"2024-09-10T15:59:05.069849+0000","last_fullsized":"2024-09-10T15:59:05.069849+0000","mapping_epoch":35,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26267648,"num_objects":6413,"num_object_clones":0,"num_object_copies":12826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11260,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14506","reported_seq":22638,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.309747+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:05.309747+0000","last_peered":"2024-09-10T15:59:05.309747+0000","last_clean":"2024-09-10T15:59:05.309747+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:05.309747+0000","last_undegraded":"2024-09-10T15:59:05.309747+0000","last_fullsized":"2024-09-10T15:59:05.309747+0000","mapping_epoch":45,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25448448,"num_objects":6213,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12483,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14269","reported_seq":15089,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.108948+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:05.108948+0000","last_peered":"2024-09-10T15:59:05.108948+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:05.108948+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25841664,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11133,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2790}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14338","reported_seq":17607,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.393582+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:05.393582+0000","last_peered":"2024-09-10T15:59:05.393582+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:05.393582+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26406912,"num_objects":6447,"num_object_clones":0,"num_object_copies":12894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12312,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1569,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4182}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14188","reported_seq":14606,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.344551+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:05.344551+0000","last_peered":"2024-09-10T15:59:05.344551+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:05.344551+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25673728,"num_objects":6268,"num_object_clones":0,"num_object_copies":12536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12057,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2582}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14394","reported_seq":15237,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.004129+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:05.004129+0000","last_peered":"2024-09-10T15:59:05.004129+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:05.004129+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26460160,"num_objects":6460,"num_object_clones":0,"num_object_copies":12920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11586,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2982}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14430","reported_seq":15180,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.144279+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:05.144279+0000","last_peered":"2024-09-10T15:59:05.144279+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:05.144279+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25755648,"num_objects":6288,"num_object_clones":0,"num_object_copies":12576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10848,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2662}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14394","reported_seq":15221,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.127365+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:05.127365+0000","last_peered":"2024-09-10T15:59:05.127365+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:05.127365+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26099712,"num_objects":6372,"num_object_clones":0,"num_object_copies":12744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11000,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2836}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14690","reported_seq":15452,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.084376+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:05.084376+0000","last_peered":"2024-09-10T15:59:05.084376+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:05.084376+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247651,"num_objects":6409,"num_object_clones":0,"num_object_copies":12818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11509,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":415146491,"num_objects":101356,"num_object_clones":0,"num_object_copies":202712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191694,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12063,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1021083648,"data_stored":1021006857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626505,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647460,"kb_used_data":304836,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93724380,"statfs":{"total":96636764160,"available":95973765120,"internally_reserved":0,"allocated":312152064,"data_stored":311813686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350809774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,7,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149787,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":755116,"kb_used_data":366684,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93616724,"statfs":{"total":96636764160,"available":95863525376,"internally_reserved":0,"allocated":375484416,"data_stored":375207918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":397733549},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,0,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652252,"kb_used_data":327692,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93719588,"statfs":{"total":96636764160,"available":95968858112,"internally_reserved":0,"allocated":335556608,"data_stored":335276038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,1,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":374919168,"data_stored":374915555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311152640,"data_stored":311087147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335011840,"data_stored":335004155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:07.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:07.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'14511", 'reported_seq': 15315, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:05.063136+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:59:05.063136+0000', 'last_peered': '2024-09-10T15:59:05.063136+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:59:05.063136+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'4500", 'ondisk_log_start': "44'4500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25636864, 'num_objects': 6259, 'num_object_clones': 0, 'num_object_copies': 12518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14511, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2647}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:07.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:07.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'14262", 'reported_seq': 14737, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:05.363700+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:59:05.363700+0000', 'last_peered': '2024-09-10T15:59:05.363700+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:59:05.363700+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26079256, 'num_objects': 6368, 'num_object_clones': 0, 'num_object_copies': 12736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6368, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12530, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2739}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:07.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:07.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'14101", 'reported_seq': 14867, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:05.112805+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:59:05.112805+0000', 'last_peered': '2024-09-10T15:59:05.112805+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:59:05.112805+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25333760, 'num_objects': 6185, 'num_object_clones': 0, 'num_object_copies': 12370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10751, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2661}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:07.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:07.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'14269", 'reported_seq': 15089, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:05.108948+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:59:05.108948+0000', 'last_peered': '2024-09-10T15:59:05.108948+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:59:05.108948+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25841664, 'num_objects': 6309, 'num_object_clones': 0, 'num_object_copies': 12618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6309, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11133, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2790}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:07.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:07.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'14338", 'reported_seq': 17607, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:05.393582+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:59:05.393582+0000', 'last_peered': '2024-09-10T15:59:05.393582+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:59:05.393582+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'4300", 'ondisk_log_start': "44'4300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26406912, 'num_objects': 6447, 'num_object_clones': 0, 'num_object_copies': 12894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2265, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6447, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12312, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1569, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4182}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:07.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:07.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'14188", 'reported_seq': 14606, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:05.344551+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:59:05.344551+0000', 'last_peered': '2024-09-10T15:59:05.344551+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:59:05.344551+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25673728, 'num_objects': 6268, 'num_object_clones': 0, 'num_object_copies': 12536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6268, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12057, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2582}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:07.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:07.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'14394", 'reported_seq': 15237, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:05.004129+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:59:05.004129+0000', 'last_peered': '2024-09-10T15:59:05.004129+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:59:05.004129+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'4300", 'ondisk_log_start': "44'4300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26460160, 'num_objects': 6460, 'num_object_clones': 0, 'num_object_copies': 12920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6460, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11586, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2982}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:07.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:07.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'14430", 'reported_seq': 15180, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:05.144279+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:59:05.144279+0000', 'last_peered': '2024-09-10T15:59:05.144279+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:59:05.144279+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'4400", 'ondisk_log_start': "44'4400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25755648, 'num_objects': 6288, 'num_object_clones': 0, 'num_object_copies': 12576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6288, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10848, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2662}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:07.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:07.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'14394", 'reported_seq': 15221, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:05.127365+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:59:05.127365+0000', 'last_peered': '2024-09-10T15:59:05.127365+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:59:05.127365+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'4300", 'ondisk_log_start': "44'4300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26099712, 'num_objects': 6372, 'num_object_clones': 0, 'num_object_copies': 12744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6372, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11000, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2836}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:07.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:07.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'14690", 'reported_seq': 15452, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:05.084376+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:59:05.084376+0000', 'last_peered': '2024-09-10T15:59:05.084376+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:59:05.084376+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'4600", 'ondisk_log_start': "44'4600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26247651, 'num_objects': 6409, 'num_object_clones': 0, 'num_object_copies': 12818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6409, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11509, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2715}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:07.501 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:08.196 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:08.196 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34585/202716 objects degraded (17.061%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":474,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":101358,"data_bytes":415146510,"bytes_used":2104143872,"bytes_avail":287806148608,"bytes_total":289910292480,"degraded_objects":34585,"degraded_total":202716,"degraded_ratio":0.17060814143925496,"recovering_objects_per_sec":51,"recovering_bytes_per_sec":82917,"recovering_keys_per_sec":0,"num_objects_recovered":614,"num_bytes_recovered":995328,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":568},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:59:08.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:08.223 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:08.777 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:08.779 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:08.806 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-10T15:59:08.555288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":415146510,"num_objects":101358,"num_object_clones":0,"num_object_copies":202716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191696,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12065,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2054828,"kb_used_data":999212,"kb_used_omap":12,"kb_used_meta":1055539,"kb_avail":281060692,"statfs":{"total":289910292480,"available":287806148608,"internally_reserved":0,"allocated":1023193088,"data_stored":1022297642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1080871941},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,7,2,1,11,7,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18731008,"num_objects":-4573,"num_object_clones":0,"num_object_copies":-9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-281,"num_objects_misplaced":-101,"num_objects_unfound":0,"num_objects_dirty":-4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4573,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":413696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004057"},"pg_stats":[{"pgid":"2.f","version":"48'14511","reported_seq":15315,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.063136+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:05.063136+0000","last_peered":"2024-09-10T15:59:05.063136+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:05.063136+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25636864,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14511,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2647}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14646","reported_seq":14758,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.424826+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:05.424826+0000","last_peered":"2024-09-10T15:59:05.424826+0000","last_clean":"2024-09-10T15:59:05.424826+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:05.424826+0000","last_undegraded":"2024-09-10T15:59:05.424826+0000","last_fullsized":"2024-09-10T15:59:05.424826+0000","mapping_epoch":36,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26329088,"num_objects":6428,"num_object_clones":0,"num_object_copies":12856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12982,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14262","reported_seq":14737,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.363700+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:05.363700+0000","last_peered":"2024-09-10T15:59:05.363700+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:05.363700+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26079256,"num_objects":6368,"num_object_clones":0,"num_object_copies":12736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6368,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12530,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2739}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14362","reported_seq":14446,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.400300+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:05.400300+0000","last_peered":"2024-09-10T15:59:05.400300+0000","last_clean":"2024-09-10T15:59:05.400300+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:05.400300+0000","last_undegraded":"2024-09-10T15:59:05.400300+0000","last_fullsized":"2024-09-10T15:59:05.400300+0000","mapping_epoch":36,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25886720,"num_objects":6320,"num_object_clones":0,"num_object_copies":12640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14362,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14143","reported_seq":14275,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.062265+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:05.062265+0000","last_peered":"2024-09-10T15:59:05.062265+0000","last_clean":"2024-09-10T15:59:05.062265+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:05.062265+0000","last_undegraded":"2024-09-10T15:59:05.062265+0000","last_fullsized":"2024-09-10T15:59:05.062265+0000","mapping_epoch":35,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11214,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14438","reported_seq":22330,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.385953+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:05.385953+0000","last_peered":"2024-09-10T15:59:05.385953+0000","last_clean":"2024-09-10T15:59:05.385953+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:05.385953+0000","last_undegraded":"2024-09-10T15:59:05.385953+0000","last_fullsized":"2024-09-10T15:59:05.385953+0000","mapping_epoch":47,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11156,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14101","reported_seq":14867,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.112805+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:05.112805+0000","last_peered":"2024-09-10T15:59:05.112805+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:05.112805+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10751,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14501","reported_seq":14637,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.069849+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:05.069849+0000","last_peered":"2024-09-10T15:59:05.069849+0000","last_clean":"2024-09-10T15:59:05.069849+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:05.069849+0000","last_undegraded":"2024-09-10T15:59:05.069849+0000","last_fullsized":"2024-09-10T15:59:05.069849+0000","mapping_epoch":35,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26267648,"num_objects":6413,"num_object_clones":0,"num_object_copies":12826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11260,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14506","reported_seq":22638,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:05.309747+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:05.309747+0000","last_peered":"2024-09-10T15:59:05.309747+0000","last_clean":"2024-09-10T15:59:05.309747+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:05.309747+0000","last_undegraded":"2024-09-10T15:59:05.309747+0000","last_fullsized":"2024-09-10T15:59:05.309747+0000","mapping_epoch":45,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25448448,"num_objects":6213,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12483,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14269","reported_seq":15089,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.108948+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:05.108948+0000","last_peered":"2024-09-10T15:59:05.108948+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:05.108948+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25841664,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11133,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2790}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14338","reported_seq":17607,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.393582+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:05.393582+0000","last_peered":"2024-09-10T15:59:05.393582+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:05.393582+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26406912,"num_objects":6447,"num_object_clones":0,"num_object_copies":12894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12312,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1569,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4182}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14188","reported_seq":14606,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.344551+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:05.344551+0000","last_peered":"2024-09-10T15:59:05.344551+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:05.344551+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25673728,"num_objects":6268,"num_object_clones":0,"num_object_copies":12536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12057,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2582}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14394","reported_seq":15237,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.004129+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:05.004129+0000","last_peered":"2024-09-10T15:59:05.004129+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:05.004129+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26460160,"num_objects":6460,"num_object_clones":0,"num_object_copies":12920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11586,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2982}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14430","reported_seq":15180,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.144279+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:05.144279+0000","last_peered":"2024-09-10T15:59:05.144279+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:05.144279+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25755648,"num_objects":6288,"num_object_clones":0,"num_object_copies":12576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10848,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2662}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14394","reported_seq":15221,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.127365+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:05.127365+0000","last_peered":"2024-09-10T15:59:05.127365+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:05.127365+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26099712,"num_objects":6372,"num_object_clones":0,"num_object_copies":12744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11000,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2836}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14690","reported_seq":15452,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:05.084376+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:05.084376+0000","last_peered":"2024-09-10T15:59:05.084376+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:05.084376+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247651,"num_objects":6409,"num_object_clones":0,"num_object_copies":12818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11509,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":415146491,"num_objects":101356,"num_object_clones":0,"num_object_copies":202712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191694,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12063,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1021083648,"data_stored":1021006857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626505,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647460,"kb_used_data":304836,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93724380,"statfs":{"total":96636764160,"available":95973765120,"internally_reserved":0,"allocated":312152064,"data_stored":311813686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350809774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,7,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149787,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":755116,"kb_used_data":366684,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93616724,"statfs":{"total":96636764160,"available":95863525376,"internally_reserved":0,"allocated":375484416,"data_stored":375207918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":397733549},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,0,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652252,"kb_used_data":327692,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93719588,"statfs":{"total":96636764160,"available":95968858112,"internally_reserved":0,"allocated":335556608,"data_stored":335276038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,1,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":374919168,"data_stored":374915555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311152640,"data_stored":311087147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335011840,"data_stored":335004155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:11.808 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:12.377 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:12.402 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-10T15:59:10.556048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":405422606,"num_objects":98984,"num_object_clones":0,"num_object_copies":197968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98984,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":194070,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12165,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2058440,"kb_used_data":980612,"kb_used_omap":12,"kb_used_meta":1077747,"kb_avail":281057080,"statfs":{"total":289910292480,"available":287802449920,"internally_reserved":0,"allocated":1004146688,"data_stored":1003251242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1103612933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,6,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28454912,"num_objects":-6947,"num_object_clones":0,"num_object_copies":-13894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-381,"num_objects_misplaced":-101,"num_objects_unfound":0,"num_objects_dirty":-6947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6947,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":413696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004168"},"pg_stats":[{"pgid":"2.f","version":"48'14643","reported_seq":15447,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.117509+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:10.117509+0000","last_peered":"2024-09-10T15:59:10.117509+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:10.117509+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25096192,"num_objects":6127,"num_object_clones":0,"num_object_copies":12254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14643,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2515}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14795","reported_seq":14907,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.433510+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:10.433510+0000","last_peered":"2024-09-10T15:59:10.433510+0000","last_clean":"2024-09-10T15:59:10.433510+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:10.433510+0000","last_undegraded":"2024-09-10T15:59:10.433510+0000","last_fullsized":"2024-09-10T15:59:10.433510+0000","mapping_epoch":36,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25718784,"num_objects":6279,"num_object_clones":0,"num_object_copies":12558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13131,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14402","reported_seq":14877,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.385738+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:10.385738+0000","last_peered":"2024-09-10T15:59:10.385738+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:10.385738+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505816,"num_objects":6228,"num_object_clones":0,"num_object_copies":12456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6228,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12670,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2599}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14519","reported_seq":14604,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.440027+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:10.440027+0000","last_peered":"2024-09-10T15:59:10.440027+0000","last_clean":"2024-09-10T15:59:10.440027+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:10.440027+0000","last_undegraded":"2024-09-10T15:59:10.440027+0000","last_fullsized":"2024-09-10T15:59:10.440027+0000","mapping_epoch":36,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25243648,"num_objects":6163,"num_object_clones":0,"num_object_copies":12326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14519,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14275","reported_seq":14407,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.149524+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:10.149524+0000","last_peered":"2024-09-10T15:59:10.149524+0000","last_clean":"2024-09-10T15:59:10.149524+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:10.149524+0000","last_undegraded":"2024-09-10T15:59:10.149524+0000","last_fullsized":"2024-09-10T15:59:10.149524+0000","mapping_epoch":35,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24805376,"num_objects":6056,"num_object_clones":0,"num_object_copies":12112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11346,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14599","reported_seq":22492,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.330408+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:10.330408+0000","last_peered":"2024-09-10T15:59:10.330408+0000","last_clean":"2024-09-10T15:59:10.330408+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:10.330408+0000","last_undegraded":"2024-09-10T15:59:10.330408+0000","last_fullsized":"2024-09-10T15:59:10.330408+0000","mapping_epoch":47,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25673728,"num_objects":6268,"num_object_clones":0,"num_object_copies":12536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11317,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14257","reported_seq":15023,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.021665+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:10.021665+0000","last_peered":"2024-09-10T15:59:10.021665+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:10.021665+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24694784,"num_objects":6029,"num_object_clones":0,"num_object_copies":12058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10907,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2505}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14665","reported_seq":14801,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.166673+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:10.166673+0000","last_peered":"2024-09-10T15:59:10.166673+0000","last_clean":"2024-09-10T15:59:10.166673+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:10.166673+0000","last_undegraded":"2024-09-10T15:59:10.166673+0000","last_fullsized":"2024-09-10T15:59:10.166673+0000","mapping_epoch":35,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25595904,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11424,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14639","reported_seq":22771,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.384321+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:10.384321+0000","last_peered":"2024-09-10T15:59:10.384321+0000","last_clean":"2024-09-10T15:59:10.384321+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:10.384321+0000","last_undegraded":"2024-09-10T15:59:10.384321+0000","last_fullsized":"2024-09-10T15:59:10.384321+0000","mapping_epoch":45,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24903680,"num_objects":6080,"num_object_clones":0,"num_object_copies":12160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12616,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14442","reported_seq":15259,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.159060+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:10.159060+0000","last_peered":"2024-09-10T15:59:10.159060+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:10.159060+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25133056,"num_objects":6136,"num_object_clones":0,"num_object_copies":12272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11306,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2617}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14476","reported_seq":17943,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.390564+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:10.390564+0000","last_peered":"2024-09-10T15:59:10.390564+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:10.390564+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25841664,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12450,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1669,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4144}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14337","reported_seq":14754,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.402649+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:10.402649+0000","last_peered":"2024-09-10T15:59:10.402649+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:10.402649+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063424,"num_objects":6119,"num_object_clones":0,"num_object_copies":12238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12206,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2433}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14551","reported_seq":15394,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.074008+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:10.074008+0000","last_peered":"2024-09-10T15:59:10.074008+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:10.074008+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25817088,"num_objects":6303,"num_object_clones":0,"num_object_copies":12606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11743,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2825}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14576","reported_seq":15327,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.177401+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:10.177401+0000","last_peered":"2024-09-10T15:59:10.177401+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:10.177401+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25157632,"num_objects":6142,"num_object_clones":0,"num_object_copies":12284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10994,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14539","reported_seq":15366,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.112789+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:10.112789+0000","last_peered":"2024-09-10T15:59:10.112789+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:10.112789+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505792,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11145,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2691}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14832","reported_seq":15594,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.072650+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:10.072650+0000","last_peered":"2024-09-10T15:59:10.072650+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:10.072650+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25666019,"num_objects":6267,"num_object_clones":0,"num_object_copies":12534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11651,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":405422587,"num_objects":98982,"num_object_clones":0,"num_object_copies":197964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98982,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":194068,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12163,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1001771008,"data_stored":1001694217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626506,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632252,"kb_used_data":297948,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93739588,"statfs":{"total":96636764160,"available":95989338112,"internally_reserved":0,"allocated":305098752,"data_stored":304760374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342290094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149788,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774288,"kb_used_data":359484,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93597552,"statfs":{"total":96636764160,"available":95843893248,"internally_reserved":0,"allocated":368111616,"data_stored":367835118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":424734381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651900,"kb_used_data":323180,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93719940,"statfs":{"total":96636764160,"available":95969218560,"internally_reserved":0,"allocated":330936320,"data_stored":330655750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":367509504,"data_stored":367505891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303972352,"data_stored":303906859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330289152,"data_stored":330281467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:12.404 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:12.964 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:12.966 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:12.990 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-10T15:59:12.556881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":405422606,"num_objects":98984,"num_object_clones":0,"num_object_copies":197968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98984,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":194070,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12165,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2058440,"kb_used_data":980612,"kb_used_omap":12,"kb_used_meta":1077747,"kb_avail":281057080,"statfs":{"total":289910292480,"available":287802449920,"internally_reserved":0,"allocated":1004146688,"data_stored":1003251242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1103612933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,6,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19304448,"num_objects":-4713,"num_object_clones":0,"num_object_copies":-9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-235,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004449"},"pg_stats":[{"pgid":"2.f","version":"48'14643","reported_seq":15447,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.117509+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:10.117509+0000","last_peered":"2024-09-10T15:59:10.117509+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:10.117509+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25096192,"num_objects":6127,"num_object_clones":0,"num_object_copies":12254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14643,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2515}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14795","reported_seq":14907,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.433510+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:10.433510+0000","last_peered":"2024-09-10T15:59:10.433510+0000","last_clean":"2024-09-10T15:59:10.433510+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:10.433510+0000","last_undegraded":"2024-09-10T15:59:10.433510+0000","last_fullsized":"2024-09-10T15:59:10.433510+0000","mapping_epoch":36,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25718784,"num_objects":6279,"num_object_clones":0,"num_object_copies":12558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13131,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14402","reported_seq":14877,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.385738+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:10.385738+0000","last_peered":"2024-09-10T15:59:10.385738+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:10.385738+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505816,"num_objects":6228,"num_object_clones":0,"num_object_copies":12456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6228,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12670,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2599}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14519","reported_seq":14604,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.440027+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:10.440027+0000","last_peered":"2024-09-10T15:59:10.440027+0000","last_clean":"2024-09-10T15:59:10.440027+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:10.440027+0000","last_undegraded":"2024-09-10T15:59:10.440027+0000","last_fullsized":"2024-09-10T15:59:10.440027+0000","mapping_epoch":36,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25243648,"num_objects":6163,"num_object_clones":0,"num_object_copies":12326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14519,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14275","reported_seq":14407,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.149524+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:10.149524+0000","last_peered":"2024-09-10T15:59:10.149524+0000","last_clean":"2024-09-10T15:59:10.149524+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:10.149524+0000","last_undegraded":"2024-09-10T15:59:10.149524+0000","last_fullsized":"2024-09-10T15:59:10.149524+0000","mapping_epoch":35,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24805376,"num_objects":6056,"num_object_clones":0,"num_object_copies":12112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11346,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14599","reported_seq":22492,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.330408+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:10.330408+0000","last_peered":"2024-09-10T15:59:10.330408+0000","last_clean":"2024-09-10T15:59:10.330408+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:10.330408+0000","last_undegraded":"2024-09-10T15:59:10.330408+0000","last_fullsized":"2024-09-10T15:59:10.330408+0000","mapping_epoch":47,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25673728,"num_objects":6268,"num_object_clones":0,"num_object_copies":12536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11317,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14257","reported_seq":15023,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.021665+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:10.021665+0000","last_peered":"2024-09-10T15:59:10.021665+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:10.021665+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24694784,"num_objects":6029,"num_object_clones":0,"num_object_copies":12058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10907,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2505}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14665","reported_seq":14801,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.166673+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:10.166673+0000","last_peered":"2024-09-10T15:59:10.166673+0000","last_clean":"2024-09-10T15:59:10.166673+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:10.166673+0000","last_undegraded":"2024-09-10T15:59:10.166673+0000","last_fullsized":"2024-09-10T15:59:10.166673+0000","mapping_epoch":35,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25595904,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11424,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14639","reported_seq":22771,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.384321+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:10.384321+0000","last_peered":"2024-09-10T15:59:10.384321+0000","last_clean":"2024-09-10T15:59:10.384321+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:10.384321+0000","last_undegraded":"2024-09-10T15:59:10.384321+0000","last_fullsized":"2024-09-10T15:59:10.384321+0000","mapping_epoch":45,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24903680,"num_objects":6080,"num_object_clones":0,"num_object_copies":12160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12616,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14442","reported_seq":15259,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.159060+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:10.159060+0000","last_peered":"2024-09-10T15:59:10.159060+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:10.159060+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25133056,"num_objects":6136,"num_object_clones":0,"num_object_copies":12272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11306,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2617}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14476","reported_seq":17943,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.390564+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:10.390564+0000","last_peered":"2024-09-10T15:59:10.390564+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:10.390564+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25841664,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12450,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1669,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4144}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14337","reported_seq":14754,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.402649+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:10.402649+0000","last_peered":"2024-09-10T15:59:10.402649+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:10.402649+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063424,"num_objects":6119,"num_object_clones":0,"num_object_copies":12238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12206,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2433}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14551","reported_seq":15394,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.074008+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:10.074008+0000","last_peered":"2024-09-10T15:59:10.074008+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:10.074008+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25817088,"num_objects":6303,"num_object_clones":0,"num_object_copies":12606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11743,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2825}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14576","reported_seq":15327,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.177401+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:10.177401+0000","last_peered":"2024-09-10T15:59:10.177401+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:10.177401+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25157632,"num_objects":6142,"num_object_clones":0,"num_object_copies":12284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10994,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14539","reported_seq":15366,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.112789+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:10.112789+0000","last_peered":"2024-09-10T15:59:10.112789+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:10.112789+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505792,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11145,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2691}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14832","reported_seq":15594,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.072650+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:10.072650+0000","last_peered":"2024-09-10T15:59:10.072650+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:10.072650+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25666019,"num_objects":6267,"num_object_clones":0,"num_object_copies":12534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11651,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":405422587,"num_objects":98982,"num_object_clones":0,"num_object_copies":197964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98982,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":194068,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12163,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1001771008,"data_stored":1001694217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626506,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632252,"kb_used_data":297948,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93739588,"statfs":{"total":96636764160,"available":95989338112,"internally_reserved":0,"allocated":305098752,"data_stored":304760374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342290094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149788,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774288,"kb_used_data":359484,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93597552,"statfs":{"total":96636764160,"available":95843893248,"internally_reserved":0,"allocated":368111616,"data_stored":367835118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":424734381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651900,"kb_used_data":323180,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93719940,"statfs":{"total":96636764160,"available":95969218560,"internally_reserved":0,"allocated":330936320,"data_stored":330655750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":367509504,"data_stored":367505891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303972352,"data_stored":303906859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330289152,"data_stored":330281467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:12.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:12.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'14643", 'reported_seq': 15447, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:10.117509+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:59:10.117509+0000', 'last_peered': '2024-09-10T15:59:10.117509+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:59:10.117509+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'4600", 'ondisk_log_start': "44'4600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25096192, 'num_objects': 6127, 'num_object_clones': 0, 'num_object_copies': 12254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14643, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2515}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:12.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:12.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'14402", 'reported_seq': 14877, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:10.385738+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:59:10.385738+0000', 'last_peered': '2024-09-10T15:59:10.385738+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:59:10.385738+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'4400", 'ondisk_log_start': "44'4400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25505816, 'num_objects': 6228, 'num_object_clones': 0, 'num_object_copies': 12456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6228, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12670, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2599}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:12.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:12.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'14257", 'reported_seq': 15023, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:10.021665+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:59:10.021665+0000', 'last_peered': '2024-09-10T15:59:10.021665+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:59:10.021665+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24694784, 'num_objects': 6029, 'num_object_clones': 0, 'num_object_copies': 12058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6029, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10907, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2505}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:12.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:12.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'14442", 'reported_seq': 15259, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:10.159060+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:59:10.159060+0000', 'last_peered': '2024-09-10T15:59:10.159060+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:59:10.159060+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'4400", 'ondisk_log_start': "44'4400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25133056, 'num_objects': 6136, 'num_object_clones': 0, 'num_object_copies': 12272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11306, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2617}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:12.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:12.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'14476", 'reported_seq': 17943, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:10.390564+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:59:10.390564+0000', 'last_peered': '2024-09-10T15:59:10.390564+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:59:10.390564+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'4400", 'ondisk_log_start': "44'4400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25841664, 'num_objects': 6309, 'num_object_clones': 0, 'num_object_copies': 12618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6309, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12450, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1669, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4144}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:12.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:12.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'14337", 'reported_seq': 14754, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:10.402649+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:59:10.402649+0000', 'last_peered': '2024-09-10T15:59:10.402649+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:59:10.402649+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'4300", 'ondisk_log_start': "44'4300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25063424, 'num_objects': 6119, 'num_object_clones': 0, 'num_object_copies': 12238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6119, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12206, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2433}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:12.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:12.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'14551", 'reported_seq': 15394, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:10.074008+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:59:10.074008+0000', 'last_peered': '2024-09-10T15:59:10.074008+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:59:10.074008+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'4500", 'ondisk_log_start': "44'4500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25817088, 'num_objects': 6303, 'num_object_clones': 0, 'num_object_copies': 12606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6303, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11743, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2825}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:12.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:12.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'14576", 'reported_seq': 15327, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:10.177401+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:59:10.177401+0000', 'last_peered': '2024-09-10T15:59:10.177401+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:59:10.177401+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'4500", 'ondisk_log_start': "44'4500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25157632, 'num_objects': 6142, 'num_object_clones': 0, 'num_object_copies': 12284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6142, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10994, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2516}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:12.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:12.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'14539", 'reported_seq': 15366, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:10.112789+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:59:10.112789+0000', 'last_peered': '2024-09-10T15:59:10.112789+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:59:10.112789+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'4500", 'ondisk_log_start': "44'4500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25505792, 'num_objects': 6227, 'num_object_clones': 0, 'num_object_copies': 12454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6227, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11145, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2691}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:12.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:12.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'14832", 'reported_seq': 15594, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:10.072650+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:59:10.072650+0000', 'last_peered': '2024-09-10T15:59:10.072650+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:59:10.072650+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'4800", 'ondisk_log_start': "44'4800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25666019, 'num_objects': 6267, 'num_object_clones': 0, 'num_object_copies': 12534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6267, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11651, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2573}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:12.993 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:13.716 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:13.717 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34485/197968 objects degraded (17.419%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":479,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":98984,"data_bytes":405422606,"bytes_used":2107842560,"bytes_avail":287802449920,"bytes_total":289910292480,"degraded_objects":34485,"degraded_total":197968,"degraded_ratio":0.17419481936474582,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":235,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":392},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:59:13.741 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:13.741 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:14.313 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:14.314 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:14.347 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-10T15:59:12.556881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":405422606,"num_objects":98984,"num_object_clones":0,"num_object_copies":197968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98984,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":194070,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12165,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2058440,"kb_used_data":980612,"kb_used_omap":12,"kb_used_meta":1077747,"kb_avail":281057080,"statfs":{"total":289910292480,"available":287802449920,"internally_reserved":0,"allocated":1004146688,"data_stored":1003251242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1103612933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,6,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19304448,"num_objects":-4713,"num_object_clones":0,"num_object_copies":-9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-235,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004449"},"pg_stats":[{"pgid":"2.f","version":"48'14643","reported_seq":15447,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.117509+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:10.117509+0000","last_peered":"2024-09-10T15:59:10.117509+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:10.117509+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25096192,"num_objects":6127,"num_object_clones":0,"num_object_copies":12254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14643,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2515}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14795","reported_seq":14907,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.433510+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:10.433510+0000","last_peered":"2024-09-10T15:59:10.433510+0000","last_clean":"2024-09-10T15:59:10.433510+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:10.433510+0000","last_undegraded":"2024-09-10T15:59:10.433510+0000","last_fullsized":"2024-09-10T15:59:10.433510+0000","mapping_epoch":36,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25718784,"num_objects":6279,"num_object_clones":0,"num_object_copies":12558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13131,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14402","reported_seq":14877,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.385738+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:10.385738+0000","last_peered":"2024-09-10T15:59:10.385738+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:10.385738+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505816,"num_objects":6228,"num_object_clones":0,"num_object_copies":12456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6228,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12670,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2599}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14519","reported_seq":14604,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.440027+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:10.440027+0000","last_peered":"2024-09-10T15:59:10.440027+0000","last_clean":"2024-09-10T15:59:10.440027+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:10.440027+0000","last_undegraded":"2024-09-10T15:59:10.440027+0000","last_fullsized":"2024-09-10T15:59:10.440027+0000","mapping_epoch":36,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25243648,"num_objects":6163,"num_object_clones":0,"num_object_copies":12326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14519,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14275","reported_seq":14407,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.149524+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:10.149524+0000","last_peered":"2024-09-10T15:59:10.149524+0000","last_clean":"2024-09-10T15:59:10.149524+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:10.149524+0000","last_undegraded":"2024-09-10T15:59:10.149524+0000","last_fullsized":"2024-09-10T15:59:10.149524+0000","mapping_epoch":35,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24805376,"num_objects":6056,"num_object_clones":0,"num_object_copies":12112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11346,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14599","reported_seq":22492,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.330408+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:10.330408+0000","last_peered":"2024-09-10T15:59:10.330408+0000","last_clean":"2024-09-10T15:59:10.330408+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:10.330408+0000","last_undegraded":"2024-09-10T15:59:10.330408+0000","last_fullsized":"2024-09-10T15:59:10.330408+0000","mapping_epoch":47,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25673728,"num_objects":6268,"num_object_clones":0,"num_object_copies":12536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11317,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14257","reported_seq":15023,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.021665+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:10.021665+0000","last_peered":"2024-09-10T15:59:10.021665+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:10.021665+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24694784,"num_objects":6029,"num_object_clones":0,"num_object_copies":12058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10907,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2505}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14665","reported_seq":14801,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.166673+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:10.166673+0000","last_peered":"2024-09-10T15:59:10.166673+0000","last_clean":"2024-09-10T15:59:10.166673+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:10.166673+0000","last_undegraded":"2024-09-10T15:59:10.166673+0000","last_fullsized":"2024-09-10T15:59:10.166673+0000","mapping_epoch":35,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25595904,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11424,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14639","reported_seq":22771,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:10.384321+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:10.384321+0000","last_peered":"2024-09-10T15:59:10.384321+0000","last_clean":"2024-09-10T15:59:10.384321+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:10.384321+0000","last_undegraded":"2024-09-10T15:59:10.384321+0000","last_fullsized":"2024-09-10T15:59:10.384321+0000","mapping_epoch":45,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24903680,"num_objects":6080,"num_object_clones":0,"num_object_copies":12160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12616,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14442","reported_seq":15259,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.159060+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:10.159060+0000","last_peered":"2024-09-10T15:59:10.159060+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:10.159060+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25133056,"num_objects":6136,"num_object_clones":0,"num_object_copies":12272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11306,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2617}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14476","reported_seq":17943,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.390564+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:10.390564+0000","last_peered":"2024-09-10T15:59:10.390564+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:10.390564+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25841664,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12450,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1669,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4144}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14337","reported_seq":14754,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.402649+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:10.402649+0000","last_peered":"2024-09-10T15:59:10.402649+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:10.402649+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063424,"num_objects":6119,"num_object_clones":0,"num_object_copies":12238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12206,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2433}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14551","reported_seq":15394,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.074008+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:10.074008+0000","last_peered":"2024-09-10T15:59:10.074008+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:10.074008+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25817088,"num_objects":6303,"num_object_clones":0,"num_object_copies":12606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11743,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2825}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14576","reported_seq":15327,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.177401+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:10.177401+0000","last_peered":"2024-09-10T15:59:10.177401+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:10.177401+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25157632,"num_objects":6142,"num_object_clones":0,"num_object_copies":12284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10994,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14539","reported_seq":15366,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.112789+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:10.112789+0000","last_peered":"2024-09-10T15:59:10.112789+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:10.112789+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505792,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11145,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2691}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14832","reported_seq":15594,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:10.072650+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:10.072650+0000","last_peered":"2024-09-10T15:59:10.072650+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:10.072650+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25666019,"num_objects":6267,"num_object_clones":0,"num_object_copies":12534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11651,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":405422587,"num_objects":98982,"num_object_clones":0,"num_object_copies":197964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98982,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":194068,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12163,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1001771008,"data_stored":1001694217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626506,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632252,"kb_used_data":297948,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93739588,"statfs":{"total":96636764160,"available":95989338112,"internally_reserved":0,"allocated":305098752,"data_stored":304760374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342290094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149788,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774288,"kb_used_data":359484,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93597552,"statfs":{"total":96636764160,"available":95843893248,"internally_reserved":0,"allocated":368111616,"data_stored":367835118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":424734381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651900,"kb_used_data":323180,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93719940,"statfs":{"total":96636764160,"available":95969218560,"internally_reserved":0,"allocated":330936320,"data_stored":330655750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":367509504,"data_stored":367505891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303972352,"data_stored":303906859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330289152,"data_stored":330281467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:17.350 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:17.911 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:17.911 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:17.937 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-10T15:59:16.558365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":395899406,"num_objects":96659,"num_object_clones":0,"num_object_copies":193318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96659,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196395,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12283,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2035536,"kb_used_data":961884,"kb_used_omap":12,"kb_used_meta":1073587,"kb_avail":281079984,"statfs":{"total":289910292480,"available":287825903616,"internally_reserved":0,"allocated":984969216,"data_stored":984073770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1099353093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,7,15,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28827648,"num_objects":-7038,"num_object_clones":0,"num_object_copies":-14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7038,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004524"},"pg_stats":[{"pgid":"2.f","version":"48'14797","reported_seq":15601,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.139846+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:15.139846+0000","last_peered":"2024-09-10T15:59:15.139846+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:15.139846+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24465408,"num_objects":5973,"num_object_clones":0,"num_object_copies":11946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14797,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2361}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14929","reported_seq":15041,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.432051+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:15.432051+0000","last_peered":"2024-09-10T15:59:15.432051+0000","last_clean":"2024-09-10T15:59:15.432051+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:15.432051+0000","last_undegraded":"2024-09-10T15:59:15.432051+0000","last_fullsized":"2024-09-10T15:59:15.432051+0000","mapping_epoch":36,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25169920,"num_objects":6145,"num_object_clones":0,"num_object_copies":12290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13265,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14537","reported_seq":15012,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.203455+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:15.203455+0000","last_peered":"2024-09-10T15:59:15.203455+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:15.203455+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952856,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12805,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2464}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14671","reported_seq":14756,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.419620+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:15.419620+0000","last_peered":"2024-09-10T15:59:15.419620+0000","last_clean":"2024-09-10T15:59:15.419620+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:15.419620+0000","last_undegraded":"2024-09-10T15:59:15.419620+0000","last_fullsized":"2024-09-10T15:59:15.419620+0000","mapping_epoch":36,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24621056,"num_objects":6011,"num_object_clones":0,"num_object_copies":12022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14671,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14420","reported_seq":14550,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.160712+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:15.160712+0000","last_peered":"2024-09-10T15:59:15.160712+0000","last_clean":"2024-09-10T15:59:15.160712+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:15.160712+0000","last_undegraded":"2024-09-10T15:59:15.160712+0000","last_fullsized":"2024-09-10T15:59:15.160712+0000","mapping_epoch":35,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24211456,"num_objects":5911,"num_object_clones":0,"num_object_copies":11822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11491,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14727","reported_seq":22620,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.423170+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:15.423170+0000","last_peered":"2024-09-10T15:59:15.423170+0000","last_clean":"2024-09-10T15:59:15.423170+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:15.423170+0000","last_undegraded":"2024-09-10T15:59:15.423170+0000","last_fullsized":"2024-09-10T15:59:15.423170+0000","mapping_epoch":47,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25149440,"num_objects":6140,"num_object_clones":0,"num_object_copies":12280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11445,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14392","reported_seq":15158,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.053194+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:15.053194+0000","last_peered":"2024-09-10T15:59:15.053194+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:15.053194+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24141824,"num_objects":5894,"num_object_clones":0,"num_object_copies":11788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11042,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2370}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14818","reported_seq":14953,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.081046+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:15.081046+0000","last_peered":"2024-09-10T15:59:15.081046+0000","last_clean":"2024-09-10T15:59:15.081046+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:15.081046+0000","last_undegraded":"2024-09-10T15:59:15.081046+0000","last_fullsized":"2024-09-10T15:59:15.081046+0000","mapping_epoch":35,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11577,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14777","reported_seq":22909,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.190479+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:15.190479+0000","last_peered":"2024-09-10T15:59:15.190479+0000","last_clean":"2024-09-10T15:59:15.190479+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:15.190479+0000","last_undegraded":"2024-09-10T15:59:15.190479+0000","last_fullsized":"2024-09-10T15:59:15.190479+0000","mapping_epoch":45,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24338432,"num_objects":5942,"num_object_clones":0,"num_object_copies":11884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12754,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14603","reported_seq":15421,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.043234+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:15.043234+0000","last_peered":"2024-09-10T15:59:15.043234+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:15.043234+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24473600,"num_objects":5975,"num_object_clones":0,"num_object_copies":11950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14627","reported_seq":18332,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.411719+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:15.411719+0000","last_peered":"2024-09-10T15:59:15.411719+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:15.411719+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25223168,"num_objects":6158,"num_object_clones":0,"num_object_copies":12316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2047,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12601,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1787,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4111}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14479","reported_seq":14896,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.443715+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:15.443715+0000","last_peered":"2024-09-10T15:59:15.443715+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:15.443715+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24481792,"num_objects":5977,"num_object_clones":0,"num_object_copies":11954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12348,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2291}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14694","reported_seq":15537,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.163084+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:15.163084+0000","last_peered":"2024-09-10T15:59:15.163084+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:15.163084+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25231360,"num_objects":6160,"num_object_clones":0,"num_object_copies":12320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11886,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14736","reported_seq":15487,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.102983+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:15.102983+0000","last_peered":"2024-09-10T15:59:15.102983+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:15.102983+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24502272,"num_objects":5982,"num_object_clones":0,"num_object_copies":11964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11154,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14693","reported_seq":15520,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.159026+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:15.159026+0000","last_peered":"2024-09-10T15:59:15.159026+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:15.159026+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24875008,"num_objects":6073,"num_object_clones":0,"num_object_copies":12146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11299,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14972","reported_seq":15733,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.147333+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:15.147333+0000","last_peered":"2024-09-10T15:59:15.147333+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:15.147333+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25092579,"num_objects":6127,"num_object_clones":0,"num_object_copies":12254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11791,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":395899387,"num_objects":96657,"num_object_clones":0,"num_object_copies":193314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96657,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196393,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12281,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":982605824,"data_stored":982529033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626507,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629420,"kb_used_data":290972,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93742420,"statfs":{"total":96636764160,"available":95992238080,"internally_reserved":0,"allocated":297955328,"data_stored":297616950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346549934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149789,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":771392,"kb_used_data":352428,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93600448,"statfs":{"total":96636764160,"available":95846858752,"internally_reserved":0,"allocated":360886272,"data_stored":360609774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":428994221},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634724,"kb_used_data":318484,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93737116,"statfs":{"total":96636764160,"available":95986806784,"internally_reserved":0,"allocated":326127616,"data_stored":325847046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":360230912,"data_stored":360227299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296812544,"data_stored":296747051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":325562368,"data_stored":325554683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:17.940 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:18.492 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:18.493 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:18.519 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-10T15:59:16.558365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":395899406,"num_objects":96659,"num_object_clones":0,"num_object_copies":193318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96659,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196395,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12283,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2035536,"kb_used_data":961884,"kb_used_omap":12,"kb_used_meta":1073587,"kb_avail":281079984,"statfs":{"total":289910292480,"available":287825903616,"internally_reserved":0,"allocated":984969216,"data_stored":984073770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1099353093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,7,15,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28827648,"num_objects":-7038,"num_object_clones":0,"num_object_copies":-14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7038,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004524"},"pg_stats":[{"pgid":"2.f","version":"48'14797","reported_seq":15601,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.139846+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:15.139846+0000","last_peered":"2024-09-10T15:59:15.139846+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:15.139846+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24465408,"num_objects":5973,"num_object_clones":0,"num_object_copies":11946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14797,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2361}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14929","reported_seq":15041,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.432051+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:15.432051+0000","last_peered":"2024-09-10T15:59:15.432051+0000","last_clean":"2024-09-10T15:59:15.432051+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:15.432051+0000","last_undegraded":"2024-09-10T15:59:15.432051+0000","last_fullsized":"2024-09-10T15:59:15.432051+0000","mapping_epoch":36,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25169920,"num_objects":6145,"num_object_clones":0,"num_object_copies":12290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13265,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14537","reported_seq":15012,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.203455+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:15.203455+0000","last_peered":"2024-09-10T15:59:15.203455+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:15.203455+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952856,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12805,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2464}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14671","reported_seq":14756,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.419620+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:15.419620+0000","last_peered":"2024-09-10T15:59:15.419620+0000","last_clean":"2024-09-10T15:59:15.419620+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:15.419620+0000","last_undegraded":"2024-09-10T15:59:15.419620+0000","last_fullsized":"2024-09-10T15:59:15.419620+0000","mapping_epoch":36,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24621056,"num_objects":6011,"num_object_clones":0,"num_object_copies":12022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14671,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14420","reported_seq":14550,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.160712+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:15.160712+0000","last_peered":"2024-09-10T15:59:15.160712+0000","last_clean":"2024-09-10T15:59:15.160712+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:15.160712+0000","last_undegraded":"2024-09-10T15:59:15.160712+0000","last_fullsized":"2024-09-10T15:59:15.160712+0000","mapping_epoch":35,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24211456,"num_objects":5911,"num_object_clones":0,"num_object_copies":11822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11491,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14727","reported_seq":22620,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.423170+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:15.423170+0000","last_peered":"2024-09-10T15:59:15.423170+0000","last_clean":"2024-09-10T15:59:15.423170+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:15.423170+0000","last_undegraded":"2024-09-10T15:59:15.423170+0000","last_fullsized":"2024-09-10T15:59:15.423170+0000","mapping_epoch":47,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25149440,"num_objects":6140,"num_object_clones":0,"num_object_copies":12280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11445,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14392","reported_seq":15158,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.053194+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:15.053194+0000","last_peered":"2024-09-10T15:59:15.053194+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:15.053194+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24141824,"num_objects":5894,"num_object_clones":0,"num_object_copies":11788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11042,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2370}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14818","reported_seq":14953,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.081046+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:15.081046+0000","last_peered":"2024-09-10T15:59:15.081046+0000","last_clean":"2024-09-10T15:59:15.081046+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:15.081046+0000","last_undegraded":"2024-09-10T15:59:15.081046+0000","last_fullsized":"2024-09-10T15:59:15.081046+0000","mapping_epoch":35,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11577,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14777","reported_seq":22909,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.190479+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:15.190479+0000","last_peered":"2024-09-10T15:59:15.190479+0000","last_clean":"2024-09-10T15:59:15.190479+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:15.190479+0000","last_undegraded":"2024-09-10T15:59:15.190479+0000","last_fullsized":"2024-09-10T15:59:15.190479+0000","mapping_epoch":45,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24338432,"num_objects":5942,"num_object_clones":0,"num_object_copies":11884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12754,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14603","reported_seq":15421,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.043234+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:15.043234+0000","last_peered":"2024-09-10T15:59:15.043234+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:15.043234+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24473600,"num_objects":5975,"num_object_clones":0,"num_object_copies":11950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14627","reported_seq":18332,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.411719+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:15.411719+0000","last_peered":"2024-09-10T15:59:15.411719+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:15.411719+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25223168,"num_objects":6158,"num_object_clones":0,"num_object_copies":12316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2047,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12601,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1787,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4111}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14479","reported_seq":14896,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.443715+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:15.443715+0000","last_peered":"2024-09-10T15:59:15.443715+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:15.443715+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24481792,"num_objects":5977,"num_object_clones":0,"num_object_copies":11954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12348,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2291}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14694","reported_seq":15537,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.163084+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:15.163084+0000","last_peered":"2024-09-10T15:59:15.163084+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:15.163084+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25231360,"num_objects":6160,"num_object_clones":0,"num_object_copies":12320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11886,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14736","reported_seq":15487,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.102983+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:15.102983+0000","last_peered":"2024-09-10T15:59:15.102983+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:15.102983+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24502272,"num_objects":5982,"num_object_clones":0,"num_object_copies":11964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11154,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14693","reported_seq":15520,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.159026+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:15.159026+0000","last_peered":"2024-09-10T15:59:15.159026+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:15.159026+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24875008,"num_objects":6073,"num_object_clones":0,"num_object_copies":12146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11299,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14972","reported_seq":15733,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.147333+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:15.147333+0000","last_peered":"2024-09-10T15:59:15.147333+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:15.147333+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25092579,"num_objects":6127,"num_object_clones":0,"num_object_copies":12254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11791,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":395899387,"num_objects":96657,"num_object_clones":0,"num_object_copies":193314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96657,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196393,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12281,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":982605824,"data_stored":982529033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626507,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629420,"kb_used_data":290972,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93742420,"statfs":{"total":96636764160,"available":95992238080,"internally_reserved":0,"allocated":297955328,"data_stored":297616950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346549934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149789,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":771392,"kb_used_data":352428,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93600448,"statfs":{"total":96636764160,"available":95846858752,"internally_reserved":0,"allocated":360886272,"data_stored":360609774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":428994221},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634724,"kb_used_data":318484,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93737116,"statfs":{"total":96636764160,"available":95986806784,"internally_reserved":0,"allocated":326127616,"data_stored":325847046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":360230912,"data_stored":360227299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296812544,"data_stored":296747051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":325562368,"data_stored":325554683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:18.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:18.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'14797", 'reported_seq': 15601, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:15.139846+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:59:15.139846+0000', 'last_peered': '2024-09-10T15:59:15.139846+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:59:15.139846+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'4700", 'ondisk_log_start': "44'4700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24465408, 'num_objects': 5973, 'num_object_clones': 0, 'num_object_copies': 11946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5973, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14797, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2361}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:18.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:18.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'14537", 'reported_seq': 15012, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:15.203455+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:59:15.203455+0000', 'last_peered': '2024-09-10T15:59:15.203455+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:59:15.203455+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'4500", 'ondisk_log_start': "44'4500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24952856, 'num_objects': 6093, 'num_object_clones': 0, 'num_object_copies': 12186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6093, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12805, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2464}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:18.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:18.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'14392", 'reported_seq': 15158, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:15.053194+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:59:15.053194+0000', 'last_peered': '2024-09-10T15:59:15.053194+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:59:15.053194+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'4300", 'ondisk_log_start': "44'4300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24141824, 'num_objects': 5894, 'num_object_clones': 0, 'num_object_copies': 11788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5894, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11042, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2370}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:18.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:18.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'14603", 'reported_seq': 15421, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:15.043234+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:59:15.043234+0000', 'last_peered': '2024-09-10T15:59:15.043234+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:59:15.043234+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'4600", 'ondisk_log_start': "44'4600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24473600, 'num_objects': 5975, 'num_object_clones': 0, 'num_object_copies': 11950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5975, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11467, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2456}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:18.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:18.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'14627", 'reported_seq': 18332, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:15.411719+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:59:15.411719+0000', 'last_peered': '2024-09-10T15:59:15.411719+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:59:15.411719+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'4600", 'ondisk_log_start': "44'4600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25223168, 'num_objects': 6158, 'num_object_clones': 0, 'num_object_copies': 12316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2047, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12601, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1787, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4111}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:18.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:18.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'14479", 'reported_seq': 14896, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:15.443715+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:59:15.443715+0000', 'last_peered': '2024-09-10T15:59:15.443715+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:59:15.443715+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'4400", 'ondisk_log_start': "44'4400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24481792, 'num_objects': 5977, 'num_object_clones': 0, 'num_object_copies': 11954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5977, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12348, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2291}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:18.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:18.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'14694", 'reported_seq': 15537, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:15.163084+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:59:15.163084+0000', 'last_peered': '2024-09-10T15:59:15.163084+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:59:15.163084+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'4600", 'ondisk_log_start': "44'4600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25231360, 'num_objects': 6160, 'num_object_clones': 0, 'num_object_copies': 12320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11886, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2682}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:18.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:18.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'14736", 'reported_seq': 15487, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:15.102983+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:59:15.102983+0000', 'last_peered': '2024-09-10T15:59:15.102983+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:59:15.102983+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'4700", 'ondisk_log_start': "44'4700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24502272, 'num_objects': 5982, 'num_object_clones': 0, 'num_object_copies': 11964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11154, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2356}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:18.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:18.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'14693", 'reported_seq': 15520, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:15.159026+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:59:15.159026+0000', 'last_peered': '2024-09-10T15:59:15.159026+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:59:15.159026+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'4600", 'ondisk_log_start': "44'4600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24875008, 'num_objects': 6073, 'num_object_clones': 0, 'num_object_copies': 12146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6073, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11299, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2537}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:18.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:18.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'14972", 'reported_seq': 15733, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:15.147333+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:59:15.147333+0000', 'last_peered': '2024-09-10T15:59:15.147333+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:59:15.147333+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'4900", 'ondisk_log_start': "44'4900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25092579, 'num_objects': 6127, 'num_object_clones': 0, 'num_object_copies': 12254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11791, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:18.523 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:19.226 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:19.226 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34367/193318 objects degraded (17.777%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":485,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":96659,"data_bytes":395899406,"bytes_used":2084388864,"bytes_avail":287825903616,"bytes_total":289910292480,"degraded_objects":34367,"degraded_total":193318,"degraded_ratio":0.17777444418005564,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":391},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:59:19.256 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:19.257 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:19.813 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:19.813 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:19.835 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-09-10T15:59:18.559243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":395899406,"num_objects":96659,"num_object_clones":0,"num_object_copies":193318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96659,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196395,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12283,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2035536,"kb_used_data":961884,"kb_used_omap":12,"kb_used_meta":1073587,"kb_avail":281079984,"statfs":{"total":289910292480,"available":287825903616,"internally_reserved":0,"allocated":984969216,"data_stored":984073770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1099353093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,7,15,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19247104,"num_objects":-4699,"num_object_clones":0,"num_object_copies":-9398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4699,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004839"},"pg_stats":[{"pgid":"2.f","version":"48'14797","reported_seq":15601,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.139846+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:15.139846+0000","last_peered":"2024-09-10T15:59:15.139846+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:15.139846+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24465408,"num_objects":5973,"num_object_clones":0,"num_object_copies":11946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14797,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2361}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'14929","reported_seq":15041,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.432051+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:15.432051+0000","last_peered":"2024-09-10T15:59:15.432051+0000","last_clean":"2024-09-10T15:59:15.432051+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:15.432051+0000","last_undegraded":"2024-09-10T15:59:15.432051+0000","last_fullsized":"2024-09-10T15:59:15.432051+0000","mapping_epoch":36,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25169920,"num_objects":6145,"num_object_clones":0,"num_object_copies":12290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13265,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14537","reported_seq":15012,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.203455+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:15.203455+0000","last_peered":"2024-09-10T15:59:15.203455+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:15.203455+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952856,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12805,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2464}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14671","reported_seq":14756,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.419620+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:15.419620+0000","last_peered":"2024-09-10T15:59:15.419620+0000","last_clean":"2024-09-10T15:59:15.419620+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:15.419620+0000","last_undegraded":"2024-09-10T15:59:15.419620+0000","last_fullsized":"2024-09-10T15:59:15.419620+0000","mapping_epoch":36,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24621056,"num_objects":6011,"num_object_clones":0,"num_object_copies":12022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14671,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14420","reported_seq":14550,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.160712+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:15.160712+0000","last_peered":"2024-09-10T15:59:15.160712+0000","last_clean":"2024-09-10T15:59:15.160712+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:15.160712+0000","last_undegraded":"2024-09-10T15:59:15.160712+0000","last_fullsized":"2024-09-10T15:59:15.160712+0000","mapping_epoch":35,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24211456,"num_objects":5911,"num_object_clones":0,"num_object_copies":11822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11491,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14727","reported_seq":22620,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.423170+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:15.423170+0000","last_peered":"2024-09-10T15:59:15.423170+0000","last_clean":"2024-09-10T15:59:15.423170+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:15.423170+0000","last_undegraded":"2024-09-10T15:59:15.423170+0000","last_fullsized":"2024-09-10T15:59:15.423170+0000","mapping_epoch":47,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25149440,"num_objects":6140,"num_object_clones":0,"num_object_copies":12280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11445,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14392","reported_seq":15158,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.053194+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:15.053194+0000","last_peered":"2024-09-10T15:59:15.053194+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:15.053194+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24141824,"num_objects":5894,"num_object_clones":0,"num_object_copies":11788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11042,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2370}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14818","reported_seq":14953,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.081046+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:15.081046+0000","last_peered":"2024-09-10T15:59:15.081046+0000","last_clean":"2024-09-10T15:59:15.081046+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:15.081046+0000","last_undegraded":"2024-09-10T15:59:15.081046+0000","last_fullsized":"2024-09-10T15:59:15.081046+0000","mapping_epoch":35,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11577,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746302+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746302+0000","last_peered":"2024-09-10T15:58:12.746302+0000","last_clean":"2024-09-10T15:58:12.746302+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746302+0000","last_undegraded":"2024-09-10T15:58:12.746302+0000","last_fullsized":"2024-09-10T15:58:12.746302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14777","reported_seq":22909,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:15.190479+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:15.190479+0000","last_peered":"2024-09-10T15:59:15.190479+0000","last_clean":"2024-09-10T15:59:15.190479+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:15.190479+0000","last_undegraded":"2024-09-10T15:59:15.190479+0000","last_fullsized":"2024-09-10T15:59:15.190479+0000","mapping_epoch":45,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24338432,"num_objects":5942,"num_object_clones":0,"num_object_copies":11884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12754,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14603","reported_seq":15421,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.043234+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:15.043234+0000","last_peered":"2024-09-10T15:59:15.043234+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:15.043234+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24473600,"num_objects":5975,"num_object_clones":0,"num_object_copies":11950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14627","reported_seq":18332,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.411719+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:15.411719+0000","last_peered":"2024-09-10T15:59:15.411719+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:15.411719+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25223168,"num_objects":6158,"num_object_clones":0,"num_object_copies":12316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2047,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12601,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1787,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4111}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14479","reported_seq":14896,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.443715+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:15.443715+0000","last_peered":"2024-09-10T15:59:15.443715+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:15.443715+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24481792,"num_objects":5977,"num_object_clones":0,"num_object_copies":11954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12348,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2291}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14694","reported_seq":15537,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.163084+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:15.163084+0000","last_peered":"2024-09-10T15:59:15.163084+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:15.163084+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25231360,"num_objects":6160,"num_object_clones":0,"num_object_copies":12320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11886,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14736","reported_seq":15487,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.102983+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:15.102983+0000","last_peered":"2024-09-10T15:59:15.102983+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:15.102983+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24502272,"num_objects":5982,"num_object_clones":0,"num_object_copies":11964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11154,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14693","reported_seq":15520,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.159026+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:15.159026+0000","last_peered":"2024-09-10T15:59:15.159026+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:15.159026+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24875008,"num_objects":6073,"num_object_clones":0,"num_object_copies":12146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11299,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'14972","reported_seq":15733,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:15.147333+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:15.147333+0000","last_peered":"2024-09-10T15:59:15.147333+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:15.147333+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25092579,"num_objects":6127,"num_object_clones":0,"num_object_copies":12254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11791,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":395899387,"num_objects":96657,"num_object_clones":0,"num_object_copies":193314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96657,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196393,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12281,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":982605824,"data_stored":982529033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626507,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629420,"kb_used_data":290972,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93742420,"statfs":{"total":96636764160,"available":95992238080,"internally_reserved":0,"allocated":297955328,"data_stored":297616950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346549934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149789,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":771392,"kb_used_data":352428,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93600448,"statfs":{"total":96636764160,"available":95846858752,"internally_reserved":0,"allocated":360886272,"data_stored":360609774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":428994221},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634724,"kb_used_data":318484,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93737116,"statfs":{"total":96636764160,"available":95986806784,"internally_reserved":0,"allocated":326127616,"data_stored":325847046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":360230912,"data_stored":360227299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296812544,"data_stored":296747051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":325562368,"data_stored":325554683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:22.838 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:23.399 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:23.401 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:23.430 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-10T15:59:22.560742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":385937934,"num_objects":94227,"num_object_clones":0,"num_object_copies":188454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94227,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198827,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12376,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2016468,"kb_used_data":942796,"kb_used_omap":12,"kb_used_meta":1073587,"kb_avail":281099052,"statfs":{"total":289910292480,"available":287845429248,"internally_reserved":0,"allocated":965423104,"data_stored":964527658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1099353093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,1,6,2,9,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19484672,"num_objects":-4757,"num_object_clones":0,"num_object_copies":-9514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4757,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004693"},"pg_stats":[{"pgid":"2.f","version":"48'14936","reported_seq":15740,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.121238+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:20.121238+0000","last_peered":"2024-09-10T15:59:20.121238+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:20.121238+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23896064,"num_objects":5834,"num_object_clones":0,"num_object_copies":11668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14936,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15089","reported_seq":15201,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.490083+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:20.490083+0000","last_peered":"2024-09-10T15:59:20.490083+0000","last_clean":"2024-09-10T15:59:20.490083+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:20.490083+0000","last_undegraded":"2024-09-10T15:59:20.490083+0000","last_fullsized":"2024-09-10T15:59:20.490083+0000","mapping_epoch":36,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24514560,"num_objects":5985,"num_object_clones":0,"num_object_copies":11970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13425,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14705","reported_seq":15176,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.454046+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:20.454046+0000","last_peered":"2024-09-10T15:59:20.454046+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:20.454046+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264728,"num_objects":5925,"num_object_clones":0,"num_object_copies":11850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12973,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2296}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14834","reported_seq":14919,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.436374+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:20.436374+0000","last_peered":"2024-09-10T15:59:20.436374+0000","last_clean":"2024-09-10T15:59:20.436374+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:20.436374+0000","last_undegraded":"2024-09-10T15:59:20.436374+0000","last_fullsized":"2024-09-10T15:59:20.436374+0000","mapping_epoch":36,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23953408,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14834,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14572","reported_seq":14704,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.147501+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:20.147501+0000","last_peered":"2024-09-10T15:59:20.147501+0000","last_clean":"2024-09-10T15:59:20.147501+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:20.147501+0000","last_undegraded":"2024-09-10T15:59:20.147501+0000","last_fullsized":"2024-09-10T15:59:20.147501+0000","mapping_epoch":35,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23588864,"num_objects":5759,"num_object_clones":0,"num_object_copies":11518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11643,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14872","reported_seq":22765,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.476680+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:20.476680+0000","last_peered":"2024-09-10T15:59:20.476680+0000","last_clean":"2024-09-10T15:59:20.476680+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:20.476680+0000","last_undegraded":"2024-09-10T15:59:20.476680+0000","last_fullsized":"2024-09-10T15:59:20.476680+0000","mapping_epoch":47,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24555520,"num_objects":5995,"num_object_clones":0,"num_object_copies":11990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11590,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14546","reported_seq":15311,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.106385+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:20.106385+0000","last_peered":"2024-09-10T15:59:20.106385+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:20.106385+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23511040,"num_objects":5740,"num_object_clones":0,"num_object_copies":11480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11196,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2216}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14964","reported_seq":15100,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.150459+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:20.150459+0000","last_peered":"2024-09-10T15:59:20.150459+0000","last_clean":"2024-09-10T15:59:20.150459+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:20.150459+0000","last_undegraded":"2024-09-10T15:59:20.150459+0000","last_fullsized":"2024-09-10T15:59:20.150459+0000","mapping_epoch":35,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24371200,"num_objects":5950,"num_object_clones":0,"num_object_copies":11900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11723,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14932","reported_seq":23064,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.425310+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:20.425310+0000","last_peered":"2024-09-10T15:59:20.425310+0000","last_clean":"2024-09-10T15:59:20.425310+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:20.425310+0000","last_undegraded":"2024-09-10T15:59:20.425310+0000","last_fullsized":"2024-09-10T15:59:20.425310+0000","mapping_epoch":45,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23703552,"num_objects":5787,"num_object_clones":0,"num_object_copies":11574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12909,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14727","reported_seq":15547,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.042988+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:20.042988+0000","last_peered":"2024-09-10T15:59:20.042988+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:20.042988+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23965696,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11591,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2332}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14797","reported_seq":18688,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.482775+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:20.482775+0000","last_peered":"2024-09-10T15:59:20.482775+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:20.482775+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24526848,"num_objects":5988,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12771,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1880,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14643","reported_seq":15061,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.450162+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:20.450162+0000","last_peered":"2024-09-10T15:59:20.450162+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:20.450162+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23810048,"num_objects":5813,"num_object_clones":0,"num_object_copies":11626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12512,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14852","reported_seq":15695,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.103062+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:20.103062+0000","last_peered":"2024-09-10T15:59:20.103062+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:20.103062+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24584192,"num_objects":6002,"num_object_clones":0,"num_object_copies":12004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12044,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14877","reported_seq":15628,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.013642+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:20.013642+0000","last_peered":"2024-09-10T15:59:20.013642+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:20.013642+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23924736,"num_objects":5841,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11295,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2215}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14848","reported_seq":15673,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.153024+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:20.153024+0000","last_peered":"2024-09-10T15:59:20.153024+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:20.153024+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24240128,"num_objects":5918,"num_object_clones":0,"num_object_copies":11836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11454,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15110","reported_seq":15872,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.178441+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:20.178441+0000","last_peered":"2024-09-10T15:59:20.178441+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:20.178441+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24527331,"num_objects":5989,"num_object_clones":0,"num_object_copies":11978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11929,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2295}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":385937915,"num_objects":94225,"num_object_clones":0,"num_object_copies":188450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94225,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198825,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12374,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":962707456,"data_stored":962630665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626508,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630752,"kb_used_data":283972,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93741088,"statfs":{"total":96636764160,"available":95990874112,"internally_reserved":0,"allocated":290787328,"data_stored":290448950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355069614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149790,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751796,"kb_used_data":345304,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93620044,"statfs":{"total":96636764160,"available":95866925056,"internally_reserved":0,"allocated":353591296,"data_stored":353314798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":416214701},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633920,"kb_used_data":313520,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93737920,"statfs":{"total":96636764160,"available":95987630080,"internally_reserved":0,"allocated":321044480,"data_stored":320763910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,1,0,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":353030144,"data_stored":353026531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289349632,"data_stored":289284139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":320327680,"data_stored":320319995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:23.432 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:23.990 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:23.990 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:24.011 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-10T15:59:22.560742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":385937934,"num_objects":94227,"num_object_clones":0,"num_object_copies":188454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94227,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198827,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12376,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2016468,"kb_used_data":942796,"kb_used_omap":12,"kb_used_meta":1073587,"kb_avail":281099052,"statfs":{"total":289910292480,"available":287845429248,"internally_reserved":0,"allocated":965423104,"data_stored":964527658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1099353093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,1,6,2,9,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19484672,"num_objects":-4757,"num_object_clones":0,"num_object_copies":-9514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4757,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004693"},"pg_stats":[{"pgid":"2.f","version":"48'14936","reported_seq":15740,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.121238+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:20.121238+0000","last_peered":"2024-09-10T15:59:20.121238+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:20.121238+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23896064,"num_objects":5834,"num_object_clones":0,"num_object_copies":11668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14936,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15089","reported_seq":15201,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.490083+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:20.490083+0000","last_peered":"2024-09-10T15:59:20.490083+0000","last_clean":"2024-09-10T15:59:20.490083+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:20.490083+0000","last_undegraded":"2024-09-10T15:59:20.490083+0000","last_fullsized":"2024-09-10T15:59:20.490083+0000","mapping_epoch":36,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24514560,"num_objects":5985,"num_object_clones":0,"num_object_copies":11970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13425,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14705","reported_seq":15176,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.454046+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:20.454046+0000","last_peered":"2024-09-10T15:59:20.454046+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:20.454046+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264728,"num_objects":5925,"num_object_clones":0,"num_object_copies":11850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12973,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2296}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14834","reported_seq":14919,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.436374+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:20.436374+0000","last_peered":"2024-09-10T15:59:20.436374+0000","last_clean":"2024-09-10T15:59:20.436374+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:20.436374+0000","last_undegraded":"2024-09-10T15:59:20.436374+0000","last_fullsized":"2024-09-10T15:59:20.436374+0000","mapping_epoch":36,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23953408,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14834,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14572","reported_seq":14704,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.147501+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:20.147501+0000","last_peered":"2024-09-10T15:59:20.147501+0000","last_clean":"2024-09-10T15:59:20.147501+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:20.147501+0000","last_undegraded":"2024-09-10T15:59:20.147501+0000","last_fullsized":"2024-09-10T15:59:20.147501+0000","mapping_epoch":35,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23588864,"num_objects":5759,"num_object_clones":0,"num_object_copies":11518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11643,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14872","reported_seq":22765,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.476680+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:20.476680+0000","last_peered":"2024-09-10T15:59:20.476680+0000","last_clean":"2024-09-10T15:59:20.476680+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:20.476680+0000","last_undegraded":"2024-09-10T15:59:20.476680+0000","last_fullsized":"2024-09-10T15:59:20.476680+0000","mapping_epoch":47,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24555520,"num_objects":5995,"num_object_clones":0,"num_object_copies":11990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11590,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14546","reported_seq":15311,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.106385+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:20.106385+0000","last_peered":"2024-09-10T15:59:20.106385+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:20.106385+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23511040,"num_objects":5740,"num_object_clones":0,"num_object_copies":11480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11196,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2216}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14964","reported_seq":15100,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.150459+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:20.150459+0000","last_peered":"2024-09-10T15:59:20.150459+0000","last_clean":"2024-09-10T15:59:20.150459+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:20.150459+0000","last_undegraded":"2024-09-10T15:59:20.150459+0000","last_fullsized":"2024-09-10T15:59:20.150459+0000","mapping_epoch":35,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24371200,"num_objects":5950,"num_object_clones":0,"num_object_copies":11900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11723,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14932","reported_seq":23064,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.425310+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:20.425310+0000","last_peered":"2024-09-10T15:59:20.425310+0000","last_clean":"2024-09-10T15:59:20.425310+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:20.425310+0000","last_undegraded":"2024-09-10T15:59:20.425310+0000","last_fullsized":"2024-09-10T15:59:20.425310+0000","mapping_epoch":45,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23703552,"num_objects":5787,"num_object_clones":0,"num_object_copies":11574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12909,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14727","reported_seq":15547,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.042988+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:20.042988+0000","last_peered":"2024-09-10T15:59:20.042988+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:20.042988+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23965696,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11591,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2332}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14797","reported_seq":18688,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.482775+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:20.482775+0000","last_peered":"2024-09-10T15:59:20.482775+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:20.482775+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24526848,"num_objects":5988,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12771,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1880,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14643","reported_seq":15061,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.450162+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:20.450162+0000","last_peered":"2024-09-10T15:59:20.450162+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:20.450162+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23810048,"num_objects":5813,"num_object_clones":0,"num_object_copies":11626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12512,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14852","reported_seq":15695,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.103062+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:20.103062+0000","last_peered":"2024-09-10T15:59:20.103062+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:20.103062+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24584192,"num_objects":6002,"num_object_clones":0,"num_object_copies":12004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12044,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14877","reported_seq":15628,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.013642+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:20.013642+0000","last_peered":"2024-09-10T15:59:20.013642+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:20.013642+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23924736,"num_objects":5841,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11295,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2215}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14848","reported_seq":15673,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.153024+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:20.153024+0000","last_peered":"2024-09-10T15:59:20.153024+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:20.153024+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24240128,"num_objects":5918,"num_object_clones":0,"num_object_copies":11836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11454,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15110","reported_seq":15872,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.178441+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:20.178441+0000","last_peered":"2024-09-10T15:59:20.178441+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:20.178441+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24527331,"num_objects":5989,"num_object_clones":0,"num_object_copies":11978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11929,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2295}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":385937915,"num_objects":94225,"num_object_clones":0,"num_object_copies":188450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94225,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198825,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12374,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":962707456,"data_stored":962630665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626508,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630752,"kb_used_data":283972,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93741088,"statfs":{"total":96636764160,"available":95990874112,"internally_reserved":0,"allocated":290787328,"data_stored":290448950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355069614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149790,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751796,"kb_used_data":345304,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93620044,"statfs":{"total":96636764160,"available":95866925056,"internally_reserved":0,"allocated":353591296,"data_stored":353314798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":416214701},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633920,"kb_used_data":313520,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93737920,"statfs":{"total":96636764160,"available":95987630080,"internally_reserved":0,"allocated":321044480,"data_stored":320763910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,1,0,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":353030144,"data_stored":353026531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289349632,"data_stored":289284139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":320327680,"data_stored":320319995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:24.012 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:24.012 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'14936", 'reported_seq': 15740, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:20.121238+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:59:20.121238+0000', 'last_peered': '2024-09-10T15:59:20.121238+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:59:20.121238+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'4900", 'ondisk_log_start': "44'4900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23896064, 'num_objects': 5834, 'num_object_clones': 0, 'num_object_copies': 11668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5834, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14936, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2222}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:24.012 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:24.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'14705", 'reported_seq': 15176, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:20.454046+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:59:20.454046+0000', 'last_peered': '2024-09-10T15:59:20.454046+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:59:20.454046+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'4700", 'ondisk_log_start': "44'4700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24264728, 'num_objects': 5925, 'num_object_clones': 0, 'num_object_copies': 11850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5925, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12973, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2296}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:24.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:24.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'14546", 'reported_seq': 15311, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:20.106385+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:59:20.106385+0000', 'last_peered': '2024-09-10T15:59:20.106385+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:59:20.106385+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'4500", 'ondisk_log_start': "44'4500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23511040, 'num_objects': 5740, 'num_object_clones': 0, 'num_object_copies': 11480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5740, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11196, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2216}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:24.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:24.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'14727", 'reported_seq': 15547, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:20.042988+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:59:20.042988+0000', 'last_peered': '2024-09-10T15:59:20.042988+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:59:20.042988+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'4700", 'ondisk_log_start': "44'4700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23965696, 'num_objects': 5851, 'num_object_clones': 0, 'num_object_copies': 11702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11591, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2332}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:24.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:24.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'14797", 'reported_seq': 18688, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:20.482775+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:59:20.482775+0000', 'last_peered': '2024-09-10T15:59:20.482775+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:59:20.482775+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'4700", 'ondisk_log_start': "44'4700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24526848, 'num_objects': 5988, 'num_object_clones': 0, 'num_object_copies': 11976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1954, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5988, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12771, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1880, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4034}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:24.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:24.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'14643", 'reported_seq': 15061, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:20.450162+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:59:20.450162+0000', 'last_peered': '2024-09-10T15:59:20.450162+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:59:20.450162+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'4600", 'ondisk_log_start': "44'4600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23810048, 'num_objects': 5813, 'num_object_clones': 0, 'num_object_copies': 11626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5813, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12512, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2127}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:24.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:24.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'14852", 'reported_seq': 15695, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:20.103062+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:59:20.103062+0000', 'last_peered': '2024-09-10T15:59:20.103062+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:59:20.103062+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'4800", 'ondisk_log_start': "44'4800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24584192, 'num_objects': 6002, 'num_object_clones': 0, 'num_object_copies': 12004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12044, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:24.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:24.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'14877", 'reported_seq': 15628, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:20.013642+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:59:20.013642+0000', 'last_peered': '2024-09-10T15:59:20.013642+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:59:20.013642+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'4800", 'ondisk_log_start': "44'4800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23924736, 'num_objects': 5841, 'num_object_clones': 0, 'num_object_copies': 11682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5841, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11295, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2215}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:24.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:24.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'14848", 'reported_seq': 15673, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:20.153024+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:59:20.153024+0000', 'last_peered': '2024-09-10T15:59:20.153024+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:59:20.153024+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'4800", 'ondisk_log_start': "44'4800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24240128, 'num_objects': 5918, 'num_object_clones': 0, 'num_object_copies': 11836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5918, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11454, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2382}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:24.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:24.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'15110", 'reported_seq': 15872, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:20.178441+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:59:20.178441+0000', 'last_peered': '2024-09-10T15:59:20.178441+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:59:20.178441+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'5100", 'ondisk_log_start': "44'5100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24527331, 'num_objects': 5989, 'num_object_clones': 0, 'num_object_copies': 11978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5989, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11929, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2295}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:24.014 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:24.732 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:24.732 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34274/188454 objects degraded (18.187%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":490,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":94227,"data_bytes":385937934,"bytes_used":2064863232,"bytes_avail":287845429248,"bytes_total":289910292480,"degraded_objects":34274,"degraded_total":188454,"degraded_ratio":0.18186931558895009,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":396},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:59:24.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:24.765 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:25.325 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:25.326 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:25.353 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-10T15:59:24.561447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":385937934,"num_objects":94227,"num_object_clones":0,"num_object_copies":188454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94227,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198827,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12376,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2016468,"kb_used_data":942796,"kb_used_omap":12,"kb_used_meta":1073587,"kb_avail":281099052,"statfs":{"total":289910292480,"available":287845429248,"internally_reserved":0,"allocated":965423104,"data_stored":964527658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1099353093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,1,6,2,9,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19484672,"num_objects":-4757,"num_object_clones":0,"num_object_copies":-9514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4757,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004565"},"pg_stats":[{"pgid":"2.f","version":"48'14936","reported_seq":15740,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.121238+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:20.121238+0000","last_peered":"2024-09-10T15:59:20.121238+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:20.121238+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23896064,"num_objects":5834,"num_object_clones":0,"num_object_copies":11668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14936,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15089","reported_seq":15201,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.490083+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:20.490083+0000","last_peered":"2024-09-10T15:59:20.490083+0000","last_clean":"2024-09-10T15:59:20.490083+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:20.490083+0000","last_undegraded":"2024-09-10T15:59:20.490083+0000","last_fullsized":"2024-09-10T15:59:20.490083+0000","mapping_epoch":36,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24514560,"num_objects":5985,"num_object_clones":0,"num_object_copies":11970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13425,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14705","reported_seq":15176,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.454046+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:20.454046+0000","last_peered":"2024-09-10T15:59:20.454046+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:20.454046+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264728,"num_objects":5925,"num_object_clones":0,"num_object_copies":11850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12973,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2296}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14834","reported_seq":14919,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.436374+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:20.436374+0000","last_peered":"2024-09-10T15:59:20.436374+0000","last_clean":"2024-09-10T15:59:20.436374+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:20.436374+0000","last_undegraded":"2024-09-10T15:59:20.436374+0000","last_fullsized":"2024-09-10T15:59:20.436374+0000","mapping_epoch":36,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23953408,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14834,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14572","reported_seq":14704,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.147501+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:20.147501+0000","last_peered":"2024-09-10T15:59:20.147501+0000","last_clean":"2024-09-10T15:59:20.147501+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:20.147501+0000","last_undegraded":"2024-09-10T15:59:20.147501+0000","last_fullsized":"2024-09-10T15:59:20.147501+0000","mapping_epoch":35,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23588864,"num_objects":5759,"num_object_clones":0,"num_object_copies":11518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11643,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'14872","reported_seq":22765,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.476680+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:20.476680+0000","last_peered":"2024-09-10T15:59:20.476680+0000","last_clean":"2024-09-10T15:59:20.476680+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:20.476680+0000","last_undegraded":"2024-09-10T15:59:20.476680+0000","last_fullsized":"2024-09-10T15:59:20.476680+0000","mapping_epoch":47,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24555520,"num_objects":5995,"num_object_clones":0,"num_object_copies":11990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11590,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14546","reported_seq":15311,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.106385+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:20.106385+0000","last_peered":"2024-09-10T15:59:20.106385+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:20.106385+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23511040,"num_objects":5740,"num_object_clones":0,"num_object_copies":11480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11196,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2216}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'14964","reported_seq":15100,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.150459+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:20.150459+0000","last_peered":"2024-09-10T15:59:20.150459+0000","last_clean":"2024-09-10T15:59:20.150459+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:20.150459+0000","last_undegraded":"2024-09-10T15:59:20.150459+0000","last_fullsized":"2024-09-10T15:59:20.150459+0000","mapping_epoch":35,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24371200,"num_objects":5950,"num_object_clones":0,"num_object_copies":11900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11723,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'14932","reported_seq":23064,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:20.425310+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:20.425310+0000","last_peered":"2024-09-10T15:59:20.425310+0000","last_clean":"2024-09-10T15:59:20.425310+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:20.425310+0000","last_undegraded":"2024-09-10T15:59:20.425310+0000","last_fullsized":"2024-09-10T15:59:20.425310+0000","mapping_epoch":45,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23703552,"num_objects":5787,"num_object_clones":0,"num_object_copies":11574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12909,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14727","reported_seq":15547,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.042988+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:20.042988+0000","last_peered":"2024-09-10T15:59:20.042988+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:20.042988+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23965696,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11591,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2332}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14797","reported_seq":18688,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.482775+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:20.482775+0000","last_peered":"2024-09-10T15:59:20.482775+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:20.482775+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24526848,"num_objects":5988,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12771,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1880,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14643","reported_seq":15061,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.450162+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:20.450162+0000","last_peered":"2024-09-10T15:59:20.450162+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:20.450162+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23810048,"num_objects":5813,"num_object_clones":0,"num_object_copies":11626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12512,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14852","reported_seq":15695,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.103062+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:20.103062+0000","last_peered":"2024-09-10T15:59:20.103062+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:20.103062+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24584192,"num_objects":6002,"num_object_clones":0,"num_object_copies":12004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12044,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'14877","reported_seq":15628,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.013642+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:20.013642+0000","last_peered":"2024-09-10T15:59:20.013642+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:20.013642+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23924736,"num_objects":5841,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11295,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2215}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14848","reported_seq":15673,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.153024+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:20.153024+0000","last_peered":"2024-09-10T15:59:20.153024+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:20.153024+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24240128,"num_objects":5918,"num_object_clones":0,"num_object_copies":11836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11454,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15110","reported_seq":15872,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:20.178441+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:20.178441+0000","last_peered":"2024-09-10T15:59:20.178441+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:20.178441+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24527331,"num_objects":5989,"num_object_clones":0,"num_object_copies":11978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11929,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2295}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":385937915,"num_objects":94225,"num_object_clones":0,"num_object_copies":188450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94225,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198825,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12374,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":962707456,"data_stored":962630665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626508,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630752,"kb_used_data":283972,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93741088,"statfs":{"total":96636764160,"available":95990874112,"internally_reserved":0,"allocated":290787328,"data_stored":290448950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355069614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149790,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751796,"kb_used_data":345304,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93620044,"statfs":{"total":96636764160,"available":95866925056,"internally_reserved":0,"allocated":353591296,"data_stored":353314798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":416214701},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633920,"kb_used_data":313520,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93737920,"statfs":{"total":96636764160,"available":95987630080,"internally_reserved":0,"allocated":321044480,"data_stored":320763910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,1,0,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":353030144,"data_stored":353026531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289349632,"data_stored":289284139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":320327680,"data_stored":320319995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:28.355 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:28.885 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:28.885 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:28.897 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-10T15:59:28.562966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":376025614,"num_objects":91807,"num_object_clones":0,"num_object_copies":183614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91807,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201247,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12470,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160927,"ondisk_log_size":160927,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1998160,"kb_used_data":924504,"kb_used_omap":12,"kb_used_meta":1073587,"kb_avail":281117360,"statfs":{"total":289910292480,"available":287864176640,"internally_reserved":0,"allocated":946692096,"data_stored":945796650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1099353093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,4,7,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19873792,"num_objects":-4852,"num_object_clones":0,"num_object_copies":-9704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4852,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004601"},"pg_stats":[{"pgid":"2.f","version":"48'15093","reported_seq":15895,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.191603+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:25.191603+0000","last_peered":"2024-09-10T15:59:25.191603+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:25.191603+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23252992,"num_objects":5677,"num_object_clones":0,"num_object_copies":11354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15093,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2065}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15223","reported_seq":15335,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.490186+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:25.490186+0000","last_peered":"2024-09-10T15:59:25.490186+0000","last_clean":"2024-09-10T15:59:25.490186+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:25.490186+0000","last_undegraded":"2024-09-10T15:59:25.490186+0000","last_fullsized":"2024-09-10T15:59:25.490186+0000","mapping_epoch":36,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23965696,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13559,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14854","reported_seq":15330,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.409430+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:25.409430+0000","last_peered":"2024-09-10T15:59:25.409430+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:25.409430+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23654424,"num_objects":5776,"num_object_clones":0,"num_object_copies":11552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5776,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13122,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2147}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14953","reported_seq":15038,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.512443+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:25.512443+0000","last_peered":"2024-09-10T15:59:25.512443+0000","last_clean":"2024-09-10T15:59:25.512443+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:25.512443+0000","last_undegraded":"2024-09-10T15:59:25.512443+0000","last_fullsized":"2024-09-10T15:59:25.512443+0000","mapping_epoch":36,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23465984,"num_objects":5729,"num_object_clones":0,"num_object_copies":11458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14953,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14730","reported_seq":14862,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.219508+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:25.219508+0000","last_peered":"2024-09-10T15:59:25.219508+0000","last_clean":"2024-09-10T15:59:25.219508+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:25.219508+0000","last_undegraded":"2024-09-10T15:59:25.219508+0000","last_fullsized":"2024-09-10T15:59:25.219508+0000","mapping_epoch":35,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22941696,"num_objects":5601,"num_object_clones":0,"num_object_copies":11202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11801,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15013","reported_seq":22906,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.512141+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:25.512141+0000","last_peered":"2024-09-10T15:59:25.512141+0000","last_clean":"2024-09-10T15:59:25.512141+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:25.512141+0000","last_undegraded":"2024-09-10T15:59:25.512141+0000","last_fullsized":"2024-09-10T15:59:25.512141+0000","mapping_epoch":47,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23977984,"num_objects":5854,"num_object_clones":0,"num_object_copies":11708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11731,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14697","reported_seq":15463,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.215044+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:25.215044+0000","last_peered":"2024-09-10T15:59:25.215044+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:25.215044+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22892544,"num_objects":5589,"num_object_clones":0,"num_object_copies":11178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11347,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15128","reported_seq":15264,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.145951+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:25.145951+0000","last_peered":"2024-09-10T15:59:25.145951+0000","last_clean":"2024-09-10T15:59:25.145951+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:25.145951+0000","last_undegraded":"2024-09-10T15:59:25.145951+0000","last_fullsized":"2024-09-10T15:59:25.145951+0000","mapping_epoch":35,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23699456,"num_objects":5786,"num_object_clones":0,"num_object_copies":11572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11887,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15102","reported_seq":23234,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.495892+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:25.495892+0000","last_peered":"2024-09-10T15:59:25.495892+0000","last_clean":"2024-09-10T15:59:25.495892+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:25.495892+0000","last_undegraded":"2024-09-10T15:59:25.495892+0000","last_fullsized":"2024-09-10T15:59:25.495892+0000","mapping_epoch":45,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23007232,"num_objects":5617,"num_object_clones":0,"num_object_copies":11234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13079,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14879","reported_seq":15699,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.190353+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:25.190353+0000","last_peered":"2024-09-10T15:59:25.190353+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:25.190353+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23343104,"num_objects":5699,"num_object_clones":0,"num_object_copies":11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11743,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2180}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14960","reported_seq":19039,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.505937+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:25.505937+0000","last_peered":"2024-09-10T15:59:25.505937+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:25.505937+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23859200,"num_objects":5825,"num_object_clones":0,"num_object_copies":11650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12934,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3965}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14780","reported_seq":15197,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.500651+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:25.500651+0000","last_peered":"2024-09-10T15:59:25.500651+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:25.500651+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23248896,"num_objects":5676,"num_object_clones":0,"num_object_copies":11352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12649,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1990}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14998","reported_seq":15841,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.040610+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:25.040610+0000","last_peered":"2024-09-10T15:59:25.040610+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:25.040610+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986176,"num_objects":5856,"num_object_clones":0,"num_object_copies":11712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12190,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2378}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15046","reported_seq":15796,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.239061+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:25.239061+0000","last_peered":"2024-09-10T15:59:25.239061+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:25.239061+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11464,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2046}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14997","reported_seq":15824,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.189304+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:25.189304+0000","last_peered":"2024-09-10T15:59:25.189304+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:25.189304+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23629824,"num_objects":5769,"num_object_clones":0,"num_object_copies":11538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11603,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15271","reported_seq":16033,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.213303+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:25.213303+0000","last_peered":"2024-09-10T15:59:25.213303+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:25.213303+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23867875,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12090,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":376025595,"num_objects":91805,"num_object_clones":0,"num_object_copies":183610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201245,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12468,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":942817280,"data_stored":942740489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160924,"ondisk_log_size":160924,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626509,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610728,"kb_used_data":276440,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93761112,"statfs":{"total":96636764160,"available":96011378688,"internally_reserved":0,"allocated":283074560,"data_stored":282736182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342290094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149791,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754160,"kb_used_data":339352,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93617680,"statfs":{"total":96636764160,"available":95864504320,"internally_reserved":0,"allocated":347496448,"data_stored":347219950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":424734381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633272,"kb_used_data":308712,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93738568,"statfs":{"total":96636764160,"available":95988293632,"internally_reserved":0,"allocated":316121088,"data_stored":315840518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":345624576,"data_stored":345620963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":281767936,"data_stored":281702443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":315424768,"data_stored":315417083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:28.898 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:29.437 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:29.439 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:29.469 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-10T15:59:28.562966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":376025614,"num_objects":91807,"num_object_clones":0,"num_object_copies":183614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91807,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201247,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12470,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160927,"ondisk_log_size":160927,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1998160,"kb_used_data":924504,"kb_used_omap":12,"kb_used_meta":1073587,"kb_avail":281117360,"statfs":{"total":289910292480,"available":287864176640,"internally_reserved":0,"allocated":946692096,"data_stored":945796650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1099353093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,4,7,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19873792,"num_objects":-4852,"num_object_clones":0,"num_object_copies":-9704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4852,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004601"},"pg_stats":[{"pgid":"2.f","version":"48'15093","reported_seq":15895,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.191603+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:25.191603+0000","last_peered":"2024-09-10T15:59:25.191603+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:25.191603+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23252992,"num_objects":5677,"num_object_clones":0,"num_object_copies":11354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15093,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2065}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15223","reported_seq":15335,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.490186+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:25.490186+0000","last_peered":"2024-09-10T15:59:25.490186+0000","last_clean":"2024-09-10T15:59:25.490186+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:25.490186+0000","last_undegraded":"2024-09-10T15:59:25.490186+0000","last_fullsized":"2024-09-10T15:59:25.490186+0000","mapping_epoch":36,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23965696,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13559,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14854","reported_seq":15330,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.409430+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:25.409430+0000","last_peered":"2024-09-10T15:59:25.409430+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:25.409430+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23654424,"num_objects":5776,"num_object_clones":0,"num_object_copies":11552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5776,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13122,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2147}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'14953","reported_seq":15038,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.512443+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:25.512443+0000","last_peered":"2024-09-10T15:59:25.512443+0000","last_clean":"2024-09-10T15:59:25.512443+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:25.512443+0000","last_undegraded":"2024-09-10T15:59:25.512443+0000","last_fullsized":"2024-09-10T15:59:25.512443+0000","mapping_epoch":36,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23465984,"num_objects":5729,"num_object_clones":0,"num_object_copies":11458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14953,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14730","reported_seq":14862,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.219508+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:25.219508+0000","last_peered":"2024-09-10T15:59:25.219508+0000","last_clean":"2024-09-10T15:59:25.219508+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:25.219508+0000","last_undegraded":"2024-09-10T15:59:25.219508+0000","last_fullsized":"2024-09-10T15:59:25.219508+0000","mapping_epoch":35,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22941696,"num_objects":5601,"num_object_clones":0,"num_object_copies":11202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11801,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15013","reported_seq":22906,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.512141+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:25.512141+0000","last_peered":"2024-09-10T15:59:25.512141+0000","last_clean":"2024-09-10T15:59:25.512141+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:25.512141+0000","last_undegraded":"2024-09-10T15:59:25.512141+0000","last_fullsized":"2024-09-10T15:59:25.512141+0000","mapping_epoch":47,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23977984,"num_objects":5854,"num_object_clones":0,"num_object_copies":11708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11731,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14697","reported_seq":15463,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.215044+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:25.215044+0000","last_peered":"2024-09-10T15:59:25.215044+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:25.215044+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22892544,"num_objects":5589,"num_object_clones":0,"num_object_copies":11178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11347,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15128","reported_seq":15264,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.145951+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:25.145951+0000","last_peered":"2024-09-10T15:59:25.145951+0000","last_clean":"2024-09-10T15:59:25.145951+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:25.145951+0000","last_undegraded":"2024-09-10T15:59:25.145951+0000","last_fullsized":"2024-09-10T15:59:25.145951+0000","mapping_epoch":35,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23699456,"num_objects":5786,"num_object_clones":0,"num_object_copies":11572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11887,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15102","reported_seq":23234,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.495892+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:25.495892+0000","last_peered":"2024-09-10T15:59:25.495892+0000","last_clean":"2024-09-10T15:59:25.495892+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:25.495892+0000","last_undegraded":"2024-09-10T15:59:25.495892+0000","last_fullsized":"2024-09-10T15:59:25.495892+0000","mapping_epoch":45,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23007232,"num_objects":5617,"num_object_clones":0,"num_object_copies":11234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13079,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'14879","reported_seq":15699,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.190353+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:25.190353+0000","last_peered":"2024-09-10T15:59:25.190353+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:25.190353+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23343104,"num_objects":5699,"num_object_clones":0,"num_object_copies":11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11743,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2180}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'14960","reported_seq":19039,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.505937+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:25.505937+0000","last_peered":"2024-09-10T15:59:25.505937+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:25.505937+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23859200,"num_objects":5825,"num_object_clones":0,"num_object_copies":11650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12934,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3965}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14780","reported_seq":15197,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.500651+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:25.500651+0000","last_peered":"2024-09-10T15:59:25.500651+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:25.500651+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23248896,"num_objects":5676,"num_object_clones":0,"num_object_copies":11352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12649,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1990}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'14998","reported_seq":15841,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.040610+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:25.040610+0000","last_peered":"2024-09-10T15:59:25.040610+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:25.040610+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986176,"num_objects":5856,"num_object_clones":0,"num_object_copies":11712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12190,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2378}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15046","reported_seq":15796,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.239061+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:25.239061+0000","last_peered":"2024-09-10T15:59:25.239061+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:25.239061+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11464,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2046}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'14997","reported_seq":15824,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.189304+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:25.189304+0000","last_peered":"2024-09-10T15:59:25.189304+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:25.189304+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23629824,"num_objects":5769,"num_object_clones":0,"num_object_copies":11538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11603,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15271","reported_seq":16033,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:25.213303+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:25.213303+0000","last_peered":"2024-09-10T15:59:25.213303+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:25.213303+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23867875,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12090,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":376025595,"num_objects":91805,"num_object_clones":0,"num_object_copies":183610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201245,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12468,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":942817280,"data_stored":942740489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160924,"ondisk_log_size":160924,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626509,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610728,"kb_used_data":276440,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93761112,"statfs":{"total":96636764160,"available":96011378688,"internally_reserved":0,"allocated":283074560,"data_stored":282736182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342290094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149791,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754160,"kb_used_data":339352,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93617680,"statfs":{"total":96636764160,"available":95864504320,"internally_reserved":0,"allocated":347496448,"data_stored":347219950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":424734381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633272,"kb_used_data":308712,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93738568,"statfs":{"total":96636764160,"available":95988293632,"internally_reserved":0,"allocated":316121088,"data_stored":315840518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":345624576,"data_stored":345620963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":281767936,"data_stored":281702443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":315424768,"data_stored":315417083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:29.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:29.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'15093", 'reported_seq': 15895, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:25.191603+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:59:25.191603+0000', 'last_peered': '2024-09-10T15:59:25.191603+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:59:25.191603+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'5000", 'ondisk_log_start': "44'5000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23252992, 'num_objects': 5677, 'num_object_clones': 0, 'num_object_copies': 11354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5677, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15093, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2065}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:29.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:29.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'14854", 'reported_seq': 15330, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:25.409430+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:59:25.409430+0000', 'last_peered': '2024-09-10T15:59:25.409430+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:59:25.409430+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'4800", 'ondisk_log_start': "44'4800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23654424, 'num_objects': 5776, 'num_object_clones': 0, 'num_object_copies': 11552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5776, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13122, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:29.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:29.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'14697", 'reported_seq': 15463, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:25.215044+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:59:25.215044+0000', 'last_peered': '2024-09-10T15:59:25.215044+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:59:25.215044+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'4600", 'ondisk_log_start': "44'4600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22892544, 'num_objects': 5589, 'num_object_clones': 0, 'num_object_copies': 11178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5589, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11347, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:29.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:29.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'14879", 'reported_seq': 15699, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:25.190353+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:59:25.190353+0000', 'last_peered': '2024-09-10T15:59:25.190353+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:59:25.190353+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'4800", 'ondisk_log_start': "44'4800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23343104, 'num_objects': 5699, 'num_object_clones': 0, 'num_object_copies': 11398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5699, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11743, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2180}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:29.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:29.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'14960", 'reported_seq': 19039, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:25.505937+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:59:25.505937+0000', 'last_peered': '2024-09-10T15:59:25.505937+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:59:25.505937+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'4900", 'ondisk_log_start': "44'4900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23859200, 'num_objects': 5825, 'num_object_clones': 0, 'num_object_copies': 11650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1860, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5825, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12934, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1974, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3965}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:29.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:29.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'14780", 'reported_seq': 15197, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:25.500651+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:59:25.500651+0000', 'last_peered': '2024-09-10T15:59:25.500651+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:59:25.500651+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'4700", 'ondisk_log_start': "44'4700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23248896, 'num_objects': 5676, 'num_object_clones': 0, 'num_object_copies': 11352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5676, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12649, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1990}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:29.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:29.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'14998", 'reported_seq': 15841, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:25.040610+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:59:25.040610+0000', 'last_peered': '2024-09-10T15:59:25.040610+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:59:25.040610+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'4900", 'ondisk_log_start': "44'4900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23986176, 'num_objects': 5856, 'num_object_clones': 0, 'num_object_copies': 11712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5856, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12190, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2378}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:29.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:29.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'15046", 'reported_seq': 15796, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:25.239061+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:59:25.239061+0000', 'last_peered': '2024-09-10T15:59:25.239061+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:59:25.239061+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'5000", 'ondisk_log_start': "44'5000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23232512, 'num_objects': 5672, 'num_object_clones': 0, 'num_object_copies': 11344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11464, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2046}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:29.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:29.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'14997", 'reported_seq': 15824, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:25.189304+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:59:25.189304+0000', 'last_peered': '2024-09-10T15:59:25.189304+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:59:25.189304+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'4900", 'ondisk_log_start': "44'4900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23629824, 'num_objects': 5769, 'num_object_clones': 0, 'num_object_copies': 11538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5769, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11603, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2233}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:29.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:29.473 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'15271", 'reported_seq': 16033, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:25.213303+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:59:25.213303+0000', 'last_peered': '2024-09-10T15:59:25.213303+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:59:25.213303+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'5200", 'ondisk_log_start': "44'5200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23867875, 'num_objects': 5828, 'num_object_clones': 0, 'num_object_copies': 11656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5828, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12090, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2134}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:29.473 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:30.156 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:30.156 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34180/183614 objects degraded (18.615%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":496,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91807,"data_bytes":376025614,"bytes_used":2046115840,"bytes_avail":287864176640,"bytes_total":289910292480,"degraded_objects":34180,"degraded_total":183614,"degraded_ratio":0.18615138279216181,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":187,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":404},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:59:30.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:30.179 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:30.751 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:30.751 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:30.778 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-10T15:59:30.563547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":367387150,"num_objects":89698,"num_object_clones":0,"num_object_copies":179396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89698,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203356,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12574,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1993148,"kb_used_data":911180,"kb_used_omap":12,"kb_used_meta":1081907,"kb_avail":281122372,"statfs":{"total":289910292480,"available":287869308928,"internally_reserved":0,"allocated":933048320,"data_stored":932152874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1107872773},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,6,10,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28512256,"num_objects":-6961,"num_object_clones":0,"num_object_copies":-13922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6961,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004304"},"pg_stats":[{"pgid":"2.f","version":"48'15235","reported_seq":16037,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.222373+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:30.222373+0000","last_peered":"2024-09-10T15:59:30.222373+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:30.222373+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22671360,"num_objects":5535,"num_object_clones":0,"num_object_copies":11070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15235,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1923}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15379","reported_seq":15491,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.500459+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:30.500459+0000","last_peered":"2024-09-10T15:59:30.500459+0000","last_clean":"2024-09-10T15:59:30.500459+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:30.500459+0000","last_undegraded":"2024-09-10T15:59:30.500459+0000","last_fullsized":"2024-09-10T15:59:30.500459+0000","mapping_epoch":36,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23326720,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13715,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14996","reported_seq":15472,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.395721+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:30.395721+0000","last_peered":"2024-09-10T15:59:30.395721+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:30.395721+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23072792,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13264,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2005}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15087","reported_seq":15172,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.426234+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:30.426234+0000","last_peered":"2024-09-10T15:59:30.426234+0000","last_clean":"2024-09-10T15:59:30.426234+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:30.426234+0000","last_undegraded":"2024-09-10T15:59:30.426234+0000","last_fullsized":"2024-09-10T15:59:30.426234+0000","mapping_epoch":36,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22917120,"num_objects":5595,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15087,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14868","reported_seq":15000,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.071442+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:30.071442+0000","last_peered":"2024-09-10T15:59:30.071442+0000","last_clean":"2024-09-10T15:59:30.071442+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:30.071442+0000","last_undegraded":"2024-09-10T15:59:30.071442+0000","last_fullsized":"2024-09-10T15:59:30.071442+0000","mapping_epoch":35,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22376448,"num_objects":5463,"num_object_clones":0,"num_object_copies":10926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11939,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15013","reported_seq":22906,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.512141+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:25.512141+0000","last_peered":"2024-09-10T15:59:25.512141+0000","last_clean":"2024-09-10T15:59:25.512141+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:25.512141+0000","last_undegraded":"2024-09-10T15:59:25.512141+0000","last_fullsized":"2024-09-10T15:59:25.512141+0000","mapping_epoch":47,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23977984,"num_objects":5854,"num_object_clones":0,"num_object_copies":11708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11731,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14843","reported_seq":15609,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.226886+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:30.226886+0000","last_peered":"2024-09-10T15:59:30.226886+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:30.226886+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22294528,"num_objects":5443,"num_object_clones":0,"num_object_copies":10886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11493,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1919}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15276","reported_seq":15412,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.175237+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:30.175237+0000","last_peered":"2024-09-10T15:59:30.175237+0000","last_clean":"2024-09-10T15:59:30.175237+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:30.175237+0000","last_undegraded":"2024-09-10T15:59:30.175237+0000","last_fullsized":"2024-09-10T15:59:30.175237+0000","mapping_epoch":35,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23093248,"num_objects":5638,"num_object_clones":0,"num_object_copies":11276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12035,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15102","reported_seq":23234,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:25.495892+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:25.495892+0000","last_peered":"2024-09-10T15:59:25.495892+0000","last_clean":"2024-09-10T15:59:25.495892+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:25.495892+0000","last_undegraded":"2024-09-10T15:59:25.495892+0000","last_fullsized":"2024-09-10T15:59:25.495892+0000","mapping_epoch":45,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23007232,"num_objects":5617,"num_object_clones":0,"num_object_copies":11234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13079,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15035","reported_seq":15853,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.209180+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:30.209180+0000","last_peered":"2024-09-10T15:59:30.209180+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:30.209180+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11899,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2024}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15106","reported_seq":19393,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.431060+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:30.431060+0000","last_peered":"2024-09-10T15:59:30.431060+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:30.431060+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23261184,"num_objects":5679,"num_object_clones":0,"num_object_copies":11358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1756,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13080,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2078,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3923}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14942","reported_seq":15359,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.498500+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:30.498500+0000","last_peered":"2024-09-10T15:59:30.498500+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:30.498500+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22585344,"num_objects":5514,"num_object_clones":0,"num_object_copies":11028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12811,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15155","reported_seq":15998,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.234629+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:30.234629+0000","last_peered":"2024-09-10T15:59:30.234629+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:30.234629+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23343104,"num_objects":5699,"num_object_clones":0,"num_object_copies":11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12347,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15204","reported_seq":15955,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.240820+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:30.240820+0000","last_peered":"2024-09-10T15:59:30.240820+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:30.240820+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22585344,"num_objects":5514,"num_object_clones":0,"num_object_copies":11028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11622,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15152","reported_seq":15978,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.182811+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:30.182811+0000","last_peered":"2024-09-10T15:59:30.182811+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:30.182811+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22994944,"num_objects":5614,"num_object_clones":0,"num_object_copies":11228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11758,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2078}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15440","reported_seq":16201,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.244855+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:30.244855+0000","last_peered":"2024-09-10T15:59:30.244855+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:30.244855+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23175651,"num_objects":5659,"num_object_clones":0,"num_object_copies":11318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12259,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1965}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":367387131,"num_objects":89696,"num_object_clones":0,"num_object_copies":179392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89696,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203354,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12572,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":930545664,"data_stored":930468873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626509,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610728,"kb_used_data":276440,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93761112,"statfs":{"total":96636764160,"available":96011378688,"internally_reserved":0,"allocated":283074560,"data_stored":282736182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342290094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149792,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":749752,"kb_used_data":330792,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93622088,"statfs":{"total":96636764160,"available":95869018112,"internally_reserved":0,"allocated":338731008,"data_stored":338454510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":428994221},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632668,"kb_used_data":303948,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93739172,"statfs":{"total":96636764160,"available":95988912128,"internally_reserved":0,"allocated":311242752,"data_stored":310962182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":338210816,"data_stored":338207203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":281767936,"data_stored":281702443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310566912,"data_stored":310559227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:33.780 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:34.347 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:34.347 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:34.369 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-10T15:59:32.564131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":366109198,"num_objects":89386,"num_object_clones":0,"num_object_copies":178772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89386,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203668,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12574,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1994124,"kb_used_data":903812,"kb_used_omap":12,"kb_used_meta":1090227,"kb_avail":281121396,"statfs":{"total":289910292480,"available":287868309504,"internally_reserved":0,"allocated":925503488,"data_stored":924608042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1116392453},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,6,10,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19828736,"num_objects":-4841,"num_object_clones":0,"num_object_copies":-9682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4841,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004274"},"pg_stats":[{"pgid":"2.f","version":"48'15235","reported_seq":16037,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.222373+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:30.222373+0000","last_peered":"2024-09-10T15:59:30.222373+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:30.222373+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22671360,"num_objects":5535,"num_object_clones":0,"num_object_copies":11070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15235,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1923}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15379","reported_seq":15491,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.500459+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:30.500459+0000","last_peered":"2024-09-10T15:59:30.500459+0000","last_clean":"2024-09-10T15:59:30.500459+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:30.500459+0000","last_undegraded":"2024-09-10T15:59:30.500459+0000","last_fullsized":"2024-09-10T15:59:30.500459+0000","mapping_epoch":36,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23326720,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13715,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14996","reported_seq":15472,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.395721+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:30.395721+0000","last_peered":"2024-09-10T15:59:30.395721+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:30.395721+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23072792,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13264,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2005}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15087","reported_seq":15172,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.426234+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:30.426234+0000","last_peered":"2024-09-10T15:59:30.426234+0000","last_clean":"2024-09-10T15:59:30.426234+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:30.426234+0000","last_undegraded":"2024-09-10T15:59:30.426234+0000","last_fullsized":"2024-09-10T15:59:30.426234+0000","mapping_epoch":36,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22917120,"num_objects":5595,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15087,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14868","reported_seq":15000,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.071442+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:30.071442+0000","last_peered":"2024-09-10T15:59:30.071442+0000","last_clean":"2024-09-10T15:59:30.071442+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:30.071442+0000","last_undegraded":"2024-09-10T15:59:30.071442+0000","last_fullsized":"2024-09-10T15:59:30.071442+0000","mapping_epoch":35,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22376448,"num_objects":5463,"num_object_clones":0,"num_object_copies":10926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11939,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15157","reported_seq":23050,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.496492+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:30.496492+0000","last_peered":"2024-09-10T15:59:30.496492+0000","last_clean":"2024-09-10T15:59:30.496492+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:30.496492+0000","last_undegraded":"2024-09-10T15:59:30.496492+0000","last_fullsized":"2024-09-10T15:59:30.496492+0000","mapping_epoch":47,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23388160,"num_objects":5710,"num_object_clones":0,"num_object_copies":11420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11875,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14843","reported_seq":15609,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.226886+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:30.226886+0000","last_peered":"2024-09-10T15:59:30.226886+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:30.226886+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22294528,"num_objects":5443,"num_object_clones":0,"num_object_copies":10886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11493,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1919}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15276","reported_seq":15412,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.175237+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:30.175237+0000","last_peered":"2024-09-10T15:59:30.175237+0000","last_clean":"2024-09-10T15:59:30.175237+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:30.175237+0000","last_undegraded":"2024-09-10T15:59:30.175237+0000","last_fullsized":"2024-09-10T15:59:30.175237+0000","mapping_epoch":35,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23093248,"num_objects":5638,"num_object_clones":0,"num_object_copies":11276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12035,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15270","reported_seq":23402,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.600157+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:30.600157+0000","last_peered":"2024-09-10T15:59:30.600157+0000","last_clean":"2024-09-10T15:59:30.600157+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:30.600157+0000","last_undegraded":"2024-09-10T15:59:30.600157+0000","last_fullsized":"2024-09-10T15:59:30.600157+0000","mapping_epoch":45,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22319104,"num_objects":5449,"num_object_clones":0,"num_object_copies":10898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13247,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15035","reported_seq":15853,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.209180+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:30.209180+0000","last_peered":"2024-09-10T15:59:30.209180+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:30.209180+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11899,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2024}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15106","reported_seq":19393,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.431060+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:30.431060+0000","last_peered":"2024-09-10T15:59:30.431060+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:30.431060+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23261184,"num_objects":5679,"num_object_clones":0,"num_object_copies":11358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1756,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13080,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2078,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3923}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14942","reported_seq":15359,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.498500+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:30.498500+0000","last_peered":"2024-09-10T15:59:30.498500+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:30.498500+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22585344,"num_objects":5514,"num_object_clones":0,"num_object_copies":11028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12811,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15155","reported_seq":15998,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.234629+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:30.234629+0000","last_peered":"2024-09-10T15:59:30.234629+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:30.234629+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23343104,"num_objects":5699,"num_object_clones":0,"num_object_copies":11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12347,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15204","reported_seq":15955,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.240820+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:30.240820+0000","last_peered":"2024-09-10T15:59:30.240820+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:30.240820+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22585344,"num_objects":5514,"num_object_clones":0,"num_object_copies":11028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11622,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15152","reported_seq":15978,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.182811+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:30.182811+0000","last_peered":"2024-09-10T15:59:30.182811+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:30.182811+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22994944,"num_objects":5614,"num_object_clones":0,"num_object_copies":11228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11758,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2078}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15440","reported_seq":16201,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.244855+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:30.244855+0000","last_peered":"2024-09-10T15:59:30.244855+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:30.244855+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23175651,"num_objects":5659,"num_object_clones":0,"num_object_copies":11318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12259,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1965}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":366109179,"num_objects":89384,"num_object_clones":0,"num_object_copies":178768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89384,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203666,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12572,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":922890240,"data_stored":922813449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626510,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611704,"kb_used_data":269072,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93760136,"statfs":{"total":96636764160,"available":96010379264,"internally_reserved":0,"allocated":275529728,"data_stored":275191350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350809774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149792,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":749752,"kb_used_data":330792,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93622088,"statfs":{"total":96636764160,"available":95869018112,"internally_reserved":0,"allocated":338731008,"data_stored":338454510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":428994221},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632668,"kb_used_data":303948,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93739172,"statfs":{"total":96636764160,"available":95988912128,"internally_reserved":0,"allocated":311242752,"data_stored":310962182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":338210816,"data_stored":338207203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274112512,"data_stored":274047019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310566912,"data_stored":310559227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:34.371 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:35.584 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:35.586 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:35.612 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-10T15:59:34.564866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":366109198,"num_objects":89386,"num_object_clones":0,"num_object_copies":178772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89386,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203668,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12574,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1994124,"kb_used_data":903812,"kb_used_omap":12,"kb_used_meta":1090227,"kb_avail":281121396,"statfs":{"total":289910292480,"available":287868309504,"internally_reserved":0,"allocated":925503488,"data_stored":924608042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1116392453},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,6,10,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19828736,"num_objects":-4841,"num_object_clones":0,"num_object_copies":-9682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4841,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004124"},"pg_stats":[{"pgid":"2.f","version":"48'15235","reported_seq":16037,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.222373+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:30.222373+0000","last_peered":"2024-09-10T15:59:30.222373+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:30.222373+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22671360,"num_objects":5535,"num_object_clones":0,"num_object_copies":11070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15235,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1923}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15379","reported_seq":15491,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.500459+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:30.500459+0000","last_peered":"2024-09-10T15:59:30.500459+0000","last_clean":"2024-09-10T15:59:30.500459+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:30.500459+0000","last_undegraded":"2024-09-10T15:59:30.500459+0000","last_fullsized":"2024-09-10T15:59:30.500459+0000","mapping_epoch":36,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23326720,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13715,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'14996","reported_seq":15472,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.395721+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:30.395721+0000","last_peered":"2024-09-10T15:59:30.395721+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:30.395721+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23072792,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13264,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2005}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15087","reported_seq":15172,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.426234+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:30.426234+0000","last_peered":"2024-09-10T15:59:30.426234+0000","last_clean":"2024-09-10T15:59:30.426234+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:30.426234+0000","last_undegraded":"2024-09-10T15:59:30.426234+0000","last_fullsized":"2024-09-10T15:59:30.426234+0000","mapping_epoch":36,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22917120,"num_objects":5595,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15087,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'14868","reported_seq":15000,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.071442+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:30.071442+0000","last_peered":"2024-09-10T15:59:30.071442+0000","last_clean":"2024-09-10T15:59:30.071442+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:30.071442+0000","last_undegraded":"2024-09-10T15:59:30.071442+0000","last_fullsized":"2024-09-10T15:59:30.071442+0000","mapping_epoch":35,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22376448,"num_objects":5463,"num_object_clones":0,"num_object_copies":10926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11939,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15157","reported_seq":23050,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.496492+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:30.496492+0000","last_peered":"2024-09-10T15:59:30.496492+0000","last_clean":"2024-09-10T15:59:30.496492+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:30.496492+0000","last_undegraded":"2024-09-10T15:59:30.496492+0000","last_fullsized":"2024-09-10T15:59:30.496492+0000","mapping_epoch":47,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23388160,"num_objects":5710,"num_object_clones":0,"num_object_copies":11420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11875,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'14843","reported_seq":15609,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.226886+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:30.226886+0000","last_peered":"2024-09-10T15:59:30.226886+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:30.226886+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22294528,"num_objects":5443,"num_object_clones":0,"num_object_copies":10886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11493,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1919}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15276","reported_seq":15412,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.175237+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:30.175237+0000","last_peered":"2024-09-10T15:59:30.175237+0000","last_clean":"2024-09-10T15:59:30.175237+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:30.175237+0000","last_undegraded":"2024-09-10T15:59:30.175237+0000","last_fullsized":"2024-09-10T15:59:30.175237+0000","mapping_epoch":35,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23093248,"num_objects":5638,"num_object_clones":0,"num_object_copies":11276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12035,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15270","reported_seq":23402,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:30.600157+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:30.600157+0000","last_peered":"2024-09-10T15:59:30.600157+0000","last_clean":"2024-09-10T15:59:30.600157+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:30.600157+0000","last_undegraded":"2024-09-10T15:59:30.600157+0000","last_fullsized":"2024-09-10T15:59:30.600157+0000","mapping_epoch":45,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22319104,"num_objects":5449,"num_object_clones":0,"num_object_copies":10898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13247,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15035","reported_seq":15853,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.209180+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:30.209180+0000","last_peered":"2024-09-10T15:59:30.209180+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:30.209180+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11899,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2024}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15106","reported_seq":19393,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.431060+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:30.431060+0000","last_peered":"2024-09-10T15:59:30.431060+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:30.431060+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23261184,"num_objects":5679,"num_object_clones":0,"num_object_copies":11358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1756,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13080,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2078,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3923}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746356+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746356+0000","last_peered":"2024-09-10T15:58:12.746356+0000","last_clean":"2024-09-10T15:58:12.746356+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746356+0000","last_undegraded":"2024-09-10T15:58:12.746356+0000","last_fullsized":"2024-09-10T15:58:12.746356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'14942","reported_seq":15359,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.498500+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:30.498500+0000","last_peered":"2024-09-10T15:59:30.498500+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:30.498500+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22585344,"num_objects":5514,"num_object_clones":0,"num_object_copies":11028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12811,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15155","reported_seq":15998,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.234629+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:30.234629+0000","last_peered":"2024-09-10T15:59:30.234629+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:30.234629+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23343104,"num_objects":5699,"num_object_clones":0,"num_object_copies":11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12347,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747204+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747204+0000","last_peered":"2024-09-10T15:58:12.747204+0000","last_clean":"2024-09-10T15:58:12.747204+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747204+0000","last_undegraded":"2024-09-10T15:58:12.747204+0000","last_fullsized":"2024-09-10T15:58:12.747204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15204","reported_seq":15955,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.240820+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:30.240820+0000","last_peered":"2024-09-10T15:59:30.240820+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:30.240820+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22585344,"num_objects":5514,"num_object_clones":0,"num_object_copies":11028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11622,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15152","reported_seq":15978,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.182811+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:30.182811+0000","last_peered":"2024-09-10T15:59:30.182811+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:30.182811+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22994944,"num_objects":5614,"num_object_clones":0,"num_object_copies":11228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11758,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2078}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15440","reported_seq":16201,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:30.244855+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:30.244855+0000","last_peered":"2024-09-10T15:59:30.244855+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:30.244855+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23175651,"num_objects":5659,"num_object_clones":0,"num_object_copies":11318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12259,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1965}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":366109179,"num_objects":89384,"num_object_clones":0,"num_object_copies":178768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89384,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203666,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12572,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":922890240,"data_stored":922813449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626510,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611704,"kb_used_data":269072,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93760136,"statfs":{"total":96636764160,"available":96010379264,"internally_reserved":0,"allocated":275529728,"data_stored":275191350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350809774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149792,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":749752,"kb_used_data":330792,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93622088,"statfs":{"total":96636764160,"available":95869018112,"internally_reserved":0,"allocated":338731008,"data_stored":338454510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":428994221},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632668,"kb_used_data":303948,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93739172,"statfs":{"total":96636764160,"available":95988912128,"internally_reserved":0,"allocated":311242752,"data_stored":310962182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":338210816,"data_stored":338207203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274112512,"data_stored":274047019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310566912,"data_stored":310559227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:35.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:35.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'15235", 'reported_seq': 16037, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:30.222373+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:59:30.222373+0000', 'last_peered': '2024-09-10T15:59:30.222373+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:59:30.222373+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'5200", 'ondisk_log_start': "44'5200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22671360, 'num_objects': 5535, 'num_object_clones': 0, 'num_object_copies': 11070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5535, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15235, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1923}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:35.614 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:35.614 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'14996", 'reported_seq': 15472, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:30.395721+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:59:30.395721+0000', 'last_peered': '2024-09-10T15:59:30.395721+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:59:30.395721+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'4900", 'ondisk_log_start': "44'4900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23072792, 'num_objects': 5634, 'num_object_clones': 0, 'num_object_copies': 11268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5634, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13264, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2005}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:35.614 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:35.614 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'14843", 'reported_seq': 15609, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:30.226886+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:59:30.226886+0000', 'last_peered': '2024-09-10T15:59:30.226886+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:59:30.226886+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'4800", 'ondisk_log_start': "44'4800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22294528, 'num_objects': 5443, 'num_object_clones': 0, 'num_object_copies': 10886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5443, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11493, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1919}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:35.614 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:35.614 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'15035", 'reported_seq': 15853, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:30.209180+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:59:30.209180+0000', 'last_peered': '2024-09-10T15:59:30.209180+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:59:30.209180+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'5000", 'ondisk_log_start': "44'5000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22704128, 'num_objects': 5543, 'num_object_clones': 0, 'num_object_copies': 11086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5543, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11899, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2024}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:35.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:35.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'15106", 'reported_seq': 19393, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:30.431060+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:59:30.431060+0000', 'last_peered': '2024-09-10T15:59:30.431060+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:59:30.431060+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'5100", 'ondisk_log_start': "44'5100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23261184, 'num_objects': 5679, 'num_object_clones': 0, 'num_object_copies': 11358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1756, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5679, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13080, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2078, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3923}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:35.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:35.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'14942", 'reported_seq': 15359, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:30.498500+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:59:30.498500+0000', 'last_peered': '2024-09-10T15:59:30.498500+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:59:30.498500+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'4900", 'ondisk_log_start': "44'4900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22585344, 'num_objects': 5514, 'num_object_clones': 0, 'num_object_copies': 11028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5514, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12811, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1828}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:35.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:35.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'15155", 'reported_seq': 15998, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:30.234629+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:59:30.234629+0000', 'last_peered': '2024-09-10T15:59:30.234629+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:59:30.234629+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'5100", 'ondisk_log_start': "44'5100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23343104, 'num_objects': 5699, 'num_object_clones': 0, 'num_object_copies': 11398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5699, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12347, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2221}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:35.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:35.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'15204", 'reported_seq': 15955, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:30.240820+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:59:30.240820+0000', 'last_peered': '2024-09-10T15:59:30.240820+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:59:30.240820+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'5200", 'ondisk_log_start': "44'5200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22585344, 'num_objects': 5514, 'num_object_clones': 0, 'num_object_copies': 11028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5514, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11622, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1888}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:35.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:35.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'15152", 'reported_seq': 15978, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:30.182811+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:59:30.182811+0000', 'last_peered': '2024-09-10T15:59:30.182811+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:59:30.182811+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'5100", 'ondisk_log_start': "44'5100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22994944, 'num_objects': 5614, 'num_object_clones': 0, 'num_object_copies': 11228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5614, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11758, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2078}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:35.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:35.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'15440", 'reported_seq': 16201, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:30.244855+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:59:30.244855+0000', 'last_peered': '2024-09-10T15:59:30.244855+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:59:30.244855+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'5400", 'ondisk_log_start': "44'5400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23175651, 'num_objects': 5659, 'num_object_clones': 0, 'num_object_copies': 11318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5659, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12259, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1965}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:35.616 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:36.295 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:36.295 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34076/178772 objects degraded (19.061%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":502,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":89386,"data_bytes":366109198,"bytes_used":2041982976,"bytes_avail":287868309504,"bytes_total":289910292480,"degraded_objects":34076,"degraded_total":178772,"degraded_ratio":0.19061150515740721,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":198,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":403},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T15:57:26.517121+0000","services":{}},"progress_events":{}} 2024-09-10T15:59:36.319 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:36.319 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:36.873 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:36.873 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:36.892 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-10T15:59:36.565590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":355861006,"num_objects":86884,"num_object_clones":0,"num_object_copies":173768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86884,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206170,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12679,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160950,"ondisk_log_size":160950,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1957708,"kb_used_data":884056,"kb_used_omap":12,"kb_used_meta":1073587,"kb_avail":281157812,"statfs":{"total":289910292480,"available":287905599488,"internally_reserved":0,"allocated":905273344,"data_stored":904377898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1099353093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,4,4,7,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30076928,"num_objects":-7343,"num_object_clones":0,"num_object_copies":-14686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7343,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004143"},"pg_stats":[{"pgid":"2.f","version":"48'15390","reported_seq":16192,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.302608+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:35.302608+0000","last_peered":"2024-09-10T15:59:35.302608+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:35.302608+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22036480,"num_objects":5380,"num_object_clones":0,"num_object_copies":10760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15390,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1768}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15556","reported_seq":15668,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.489046+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:35.489046+0000","last_peered":"2024-09-10T15:59:35.489046+0000","last_clean":"2024-09-10T15:59:35.489046+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:35.489046+0000","last_undegraded":"2024-09-10T15:59:35.489046+0000","last_fullsized":"2024-09-10T15:59:35.489046+0000","mapping_epoch":36,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13892,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15179","reported_seq":15653,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.556259+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:35.556259+0000","last_peered":"2024-09-10T15:59:35.556259+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:35.556259+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22323224,"num_objects":5451,"num_object_clones":0,"num_object_copies":10902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13447,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1822}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15229","reported_seq":15313,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.521322+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:35.521322+0000","last_peered":"2024-09-10T15:59:35.521322+0000","last_clean":"2024-09-10T15:59:35.521322+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:35.521322+0000","last_undegraded":"2024-09-10T15:59:35.521322+0000","last_fullsized":"2024-09-10T15:59:35.521322+0000","mapping_epoch":36,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22335488,"num_objects":5453,"num_object_clones":0,"num_object_copies":10906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15229,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15003","reported_seq":15135,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.301358+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:35.301358+0000","last_peered":"2024-09-10T15:59:35.301358+0000","last_clean":"2024-09-10T15:59:35.301358+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:35.301358+0000","last_undegraded":"2024-09-10T15:59:35.301358+0000","last_fullsized":"2024-09-10T15:59:35.301358+0000","mapping_epoch":35,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823488,"num_objects":5328,"num_object_clones":0,"num_object_copies":10656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12074,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15326","reported_seq":23219,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.602076+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:35.602076+0000","last_peered":"2024-09-10T15:59:35.602076+0000","last_clean":"2024-09-10T15:59:35.602076+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:35.602076+0000","last_undegraded":"2024-09-10T15:59:35.602076+0000","last_fullsized":"2024-09-10T15:59:35.602076+0000","mapping_epoch":47,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22695936,"num_objects":5541,"num_object_clones":0,"num_object_copies":11082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12044,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15000","reported_seq":15766,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.285932+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:35.285932+0000","last_peered":"2024-09-10T15:59:35.285932+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:35.285932+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21651456,"num_objects":5286,"num_object_clones":0,"num_object_copies":10572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11650,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1762}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15435","reported_seq":15571,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.302560+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:35.302560+0000","last_peered":"2024-09-10T15:59:35.302560+0000","last_clean":"2024-09-10T15:59:35.302560+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:35.302560+0000","last_undegraded":"2024-09-10T15:59:35.302560+0000","last_fullsized":"2024-09-10T15:59:35.302560+0000","mapping_epoch":35,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22441984,"num_objects":5479,"num_object_clones":0,"num_object_copies":10958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12194,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15414","reported_seq":23547,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.643058+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:35.643058+0000","last_peered":"2024-09-10T15:59:35.643058+0000","last_clean":"2024-09-10T15:59:35.643058+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:35.643058+0000","last_undegraded":"2024-09-10T15:59:35.643058+0000","last_fullsized":"2024-09-10T15:59:35.643058+0000","mapping_epoch":45,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21729280,"num_objects":5305,"num_object_clones":0,"num_object_copies":10610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13391,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15200","reported_seq":16016,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.292662+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:35.292662+0000","last_peered":"2024-09-10T15:59:35.292662+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:35.292662+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22028288,"num_objects":5378,"num_object_clones":0,"num_object_copies":10756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12064,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1859}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15257","reported_seq":19754,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.509627+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:35.509627+0000","last_peered":"2024-09-10T15:59:35.509627+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:35.509627+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22642688,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13231,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2183,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3877}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15079","reported_seq":15496,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.530095+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:35.530095+0000","last_peered":"2024-09-10T15:59:35.530095+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:35.530095+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12948,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1691}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15339","reported_seq":16182,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.260939+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:35.260939+0000","last_peered":"2024-09-10T15:59:35.260939+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:35.260939+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22589440,"num_objects":5515,"num_object_clones":0,"num_object_copies":11030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12531,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2037}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15361","reported_seq":16112,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.269374+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:35.269374+0000","last_peered":"2024-09-10T15:59:35.269374+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:35.269374+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21942272,"num_objects":5357,"num_object_clones":0,"num_object_copies":10714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11779,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1731}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15283","reported_seq":16110,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.316344+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:35.316344+0000","last_peered":"2024-09-10T15:59:35.316344+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:35.316344+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22458368,"num_objects":5483,"num_object_clones":0,"num_object_copies":10966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11889,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1947}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15596","reported_seq":16358,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.201124+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:35.201124+0000","last_peered":"2024-09-10T15:59:35.201124+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:35.201124+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22536675,"num_objects":5503,"num_object_clones":0,"num_object_copies":11006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12415,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":355860987,"num_objects":86882,"num_object_clones":0,"num_object_copies":173764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206168,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12677,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":902426624,"data_stored":902349833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160947,"ondisk_log_size":160947,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626511,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608364,"kb_used_data":261596,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93763476,"statfs":{"total":96636764160,"available":96013799424,"internally_reserved":0,"allocated":267874304,"data_stored":267535926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355069614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149793,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":734012,"kb_used_data":323372,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93637828,"statfs":{"total":96636764160,"available":95885135872,"internally_reserved":0,"allocated":331132928,"data_stored":330856430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":420474541},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615332,"kb_used_data":299088,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93756508,"statfs":{"total":96636764160,"available":96006664192,"internally_reserved":0,"allocated":306266112,"data_stored":305985542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":330510336,"data_stored":330506723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":266395648,"data_stored":266330155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":305520640,"data_stored":305512955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:39.895 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:40.450 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:40.452 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:40.476 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-10T15:59:38.566539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":355861006,"num_objects":86884,"num_object_clones":0,"num_object_copies":173768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86884,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206170,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12679,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160950,"ondisk_log_size":160950,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1957708,"kb_used_data":884056,"kb_used_omap":12,"kb_used_meta":1073587,"kb_avail":281157812,"statfs":{"total":289910292480,"available":287905599488,"internally_reserved":0,"allocated":905273344,"data_stored":904377898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1099353093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,4,4,7,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20164608,"num_objects":-4923,"num_object_clones":0,"num_object_copies":-9846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4923,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004255"},"pg_stats":[{"pgid":"2.f","version":"48'15390","reported_seq":16192,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.302608+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:35.302608+0000","last_peered":"2024-09-10T15:59:35.302608+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:35.302608+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22036480,"num_objects":5380,"num_object_clones":0,"num_object_copies":10760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15390,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1768}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15556","reported_seq":15668,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.489046+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:35.489046+0000","last_peered":"2024-09-10T15:59:35.489046+0000","last_clean":"2024-09-10T15:59:35.489046+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:35.489046+0000","last_undegraded":"2024-09-10T15:59:35.489046+0000","last_fullsized":"2024-09-10T15:59:35.489046+0000","mapping_epoch":36,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13892,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15179","reported_seq":15653,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.556259+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:35.556259+0000","last_peered":"2024-09-10T15:59:35.556259+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:35.556259+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22323224,"num_objects":5451,"num_object_clones":0,"num_object_copies":10902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13447,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1822}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15229","reported_seq":15313,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.521322+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:35.521322+0000","last_peered":"2024-09-10T15:59:35.521322+0000","last_clean":"2024-09-10T15:59:35.521322+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:35.521322+0000","last_undegraded":"2024-09-10T15:59:35.521322+0000","last_fullsized":"2024-09-10T15:59:35.521322+0000","mapping_epoch":36,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22335488,"num_objects":5453,"num_object_clones":0,"num_object_copies":10906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15229,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15003","reported_seq":15135,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.301358+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:35.301358+0000","last_peered":"2024-09-10T15:59:35.301358+0000","last_clean":"2024-09-10T15:59:35.301358+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:35.301358+0000","last_undegraded":"2024-09-10T15:59:35.301358+0000","last_fullsized":"2024-09-10T15:59:35.301358+0000","mapping_epoch":35,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823488,"num_objects":5328,"num_object_clones":0,"num_object_copies":10656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12074,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15326","reported_seq":23219,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.602076+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:35.602076+0000","last_peered":"2024-09-10T15:59:35.602076+0000","last_clean":"2024-09-10T15:59:35.602076+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:35.602076+0000","last_undegraded":"2024-09-10T15:59:35.602076+0000","last_fullsized":"2024-09-10T15:59:35.602076+0000","mapping_epoch":47,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22695936,"num_objects":5541,"num_object_clones":0,"num_object_copies":11082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12044,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15000","reported_seq":15766,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.285932+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:35.285932+0000","last_peered":"2024-09-10T15:59:35.285932+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:35.285932+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21651456,"num_objects":5286,"num_object_clones":0,"num_object_copies":10572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11650,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1762}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15435","reported_seq":15571,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.302560+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:35.302560+0000","last_peered":"2024-09-10T15:59:35.302560+0000","last_clean":"2024-09-10T15:59:35.302560+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:35.302560+0000","last_undegraded":"2024-09-10T15:59:35.302560+0000","last_fullsized":"2024-09-10T15:59:35.302560+0000","mapping_epoch":35,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22441984,"num_objects":5479,"num_object_clones":0,"num_object_copies":10958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12194,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15414","reported_seq":23547,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.643058+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:35.643058+0000","last_peered":"2024-09-10T15:59:35.643058+0000","last_clean":"2024-09-10T15:59:35.643058+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:35.643058+0000","last_undegraded":"2024-09-10T15:59:35.643058+0000","last_fullsized":"2024-09-10T15:59:35.643058+0000","mapping_epoch":45,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21729280,"num_objects":5305,"num_object_clones":0,"num_object_copies":10610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13391,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15200","reported_seq":16016,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.292662+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:35.292662+0000","last_peered":"2024-09-10T15:59:35.292662+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:35.292662+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22028288,"num_objects":5378,"num_object_clones":0,"num_object_copies":10756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12064,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1859}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15257","reported_seq":19754,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.509627+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:35.509627+0000","last_peered":"2024-09-10T15:59:35.509627+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:35.509627+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22642688,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13231,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2183,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3877}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15079","reported_seq":15496,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.530095+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:35.530095+0000","last_peered":"2024-09-10T15:59:35.530095+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:35.530095+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12948,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1691}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15339","reported_seq":16182,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.260939+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:35.260939+0000","last_peered":"2024-09-10T15:59:35.260939+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:35.260939+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22589440,"num_objects":5515,"num_object_clones":0,"num_object_copies":11030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12531,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2037}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15361","reported_seq":16112,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.269374+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:35.269374+0000","last_peered":"2024-09-10T15:59:35.269374+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:35.269374+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21942272,"num_objects":5357,"num_object_clones":0,"num_object_copies":10714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11779,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1731}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15283","reported_seq":16110,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.316344+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:35.316344+0000","last_peered":"2024-09-10T15:59:35.316344+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:35.316344+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22458368,"num_objects":5483,"num_object_clones":0,"num_object_copies":10966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11889,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1947}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15596","reported_seq":16358,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.201124+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:35.201124+0000","last_peered":"2024-09-10T15:59:35.201124+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:35.201124+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22536675,"num_objects":5503,"num_object_clones":0,"num_object_copies":11006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12415,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":355860987,"num_objects":86882,"num_object_clones":0,"num_object_copies":173764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206168,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12677,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":902426624,"data_stored":902349833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160947,"ondisk_log_size":160947,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626511,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608364,"kb_used_data":261596,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93763476,"statfs":{"total":96636764160,"available":96013799424,"internally_reserved":0,"allocated":267874304,"data_stored":267535926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355069614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149793,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":734012,"kb_used_data":323372,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93637828,"statfs":{"total":96636764160,"available":95885135872,"internally_reserved":0,"allocated":331132928,"data_stored":330856430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":420474541},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615332,"kb_used_data":299088,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93756508,"statfs":{"total":96636764160,"available":96006664192,"internally_reserved":0,"allocated":306266112,"data_stored":305985542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":330510336,"data_stored":330506723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":266395648,"data_stored":266330155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":305520640,"data_stored":305512955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:40.477 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:41.051 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:41.052 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:41.075 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-10T15:59:40.567031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":350179854,"num_objects":85497,"num_object_clones":0,"num_object_copies":170994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85497,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207557,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12679,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160637,"ondisk_log_size":160637,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1954376,"kb_used_data":876560,"kb_used_omap":12,"kb_used_meta":1077747,"kb_avail":281161144,"statfs":{"total":289910292480,"available":287909011456,"internally_reserved":0,"allocated":897597440,"data_stored":896701994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1103612933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,18,2,9,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25845760,"num_objects":-6310,"num_object_clones":0,"num_object_copies":-12620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6310,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004065"},"pg_stats":[{"pgid":"2.f","version":"48'15545","reported_seq":16348,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.308529+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:40.308529+0000","last_peered":"2024-09-10T15:59:40.308529+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:40.308529+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21401600,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15545,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1613}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15556","reported_seq":15668,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.489046+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:35.489046+0000","last_peered":"2024-09-10T15:59:35.489046+0000","last_clean":"2024-09-10T15:59:35.489046+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:35.489046+0000","last_undegraded":"2024-09-10T15:59:35.489046+0000","last_fullsized":"2024-09-10T15:59:35.489046+0000","mapping_epoch":36,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13892,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15179","reported_seq":15653,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.556259+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:35.556259+0000","last_peered":"2024-09-10T15:59:35.556259+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:35.556259+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22323224,"num_objects":5451,"num_object_clones":0,"num_object_copies":10902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13447,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1822}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15229","reported_seq":15313,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.521322+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:35.521322+0000","last_peered":"2024-09-10T15:59:35.521322+0000","last_clean":"2024-09-10T15:59:35.521322+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:35.521322+0000","last_undegraded":"2024-09-10T15:59:35.521322+0000","last_fullsized":"2024-09-10T15:59:35.521322+0000","mapping_epoch":36,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22335488,"num_objects":5453,"num_object_clones":0,"num_object_copies":10906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15229,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15159","reported_seq":15291,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.347249+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:40.347249+0000","last_peered":"2024-09-10T15:59:40.347249+0000","last_clean":"2024-09-10T15:59:40.347249+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:40.347249+0000","last_undegraded":"2024-09-10T15:59:40.347249+0000","last_fullsized":"2024-09-10T15:59:40.347249+0000","mapping_epoch":35,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21184512,"num_objects":5172,"num_object_clones":0,"num_object_copies":10344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12230,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15326","reported_seq":23219,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.602076+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:35.602076+0000","last_peered":"2024-09-10T15:59:35.602076+0000","last_clean":"2024-09-10T15:59:35.602076+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:35.602076+0000","last_undegraded":"2024-09-10T15:59:35.602076+0000","last_fullsized":"2024-09-10T15:59:35.602076+0000","mapping_epoch":47,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22695936,"num_objects":5541,"num_object_clones":0,"num_object_copies":11082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12044,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15149","reported_seq":15915,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.254491+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:40.254491+0000","last_peered":"2024-09-10T15:59:40.254491+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:40.254491+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21041152,"num_objects":5137,"num_object_clones":0,"num_object_copies":10274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11799,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1613}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15607","reported_seq":15743,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.382842+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:40.382842+0000","last_peered":"2024-09-10T15:59:40.382842+0000","last_clean":"2024-09-10T15:59:40.382842+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:40.382842+0000","last_undegraded":"2024-09-10T15:59:40.382842+0000","last_fullsized":"2024-09-10T15:59:40.382842+0000","mapping_epoch":35,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21737472,"num_objects":5307,"num_object_clones":0,"num_object_copies":10614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12366,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15414","reported_seq":23547,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.643058+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:35.643058+0000","last_peered":"2024-09-10T15:59:35.643058+0000","last_clean":"2024-09-10T15:59:35.643058+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:35.643058+0000","last_undegraded":"2024-09-10T15:59:35.643058+0000","last_fullsized":"2024-09-10T15:59:35.643058+0000","mapping_epoch":45,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21729280,"num_objects":5305,"num_object_clones":0,"num_object_copies":10610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13391,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15341","reported_seq":16161,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.350215+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:40.350215+0000","last_peered":"2024-09-10T15:59:40.350215+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:40.350215+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21450752,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12205,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1718}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15257","reported_seq":19754,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.509627+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:35.509627+0000","last_peered":"2024-09-10T15:59:35.509627+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:35.509627+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22642688,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13231,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2183,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3877}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15079","reported_seq":15496,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.530095+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:35.530095+0000","last_peered":"2024-09-10T15:59:35.530095+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:35.530095+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12948,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1691}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15507","reported_seq":16349,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.360515+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:40.360515+0000","last_peered":"2024-09-10T15:59:40.360515+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:40.360515+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12699,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1869}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15506","reported_seq":16257,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.391227+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:40.391227+0000","last_peered":"2024-09-10T15:59:40.391227+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:40.391227+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21348352,"num_objects":5212,"num_object_clones":0,"num_object_copies":10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11924,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15435","reported_seq":16262,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.383842+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:40.383842+0000","last_peered":"2024-09-10T15:59:40.383842+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:40.383842+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21835776,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12041,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1795}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15745","reported_seq":16506,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.274834+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:40.274834+0000","last_peered":"2024-09-10T15:59:40.274834+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:40.274834+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21926371,"num_objects":5354,"num_object_clones":0,"num_object_copies":10708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12564,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":350179835,"num_objects":85495,"num_object_clones":0,"num_object_copies":170990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207555,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12677,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":894750720,"data_stored":894673929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160634,"ondisk_log_size":160634,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626511,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608364,"kb_used_data":261596,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93763476,"statfs":{"total":96636764160,"available":96013799424,"internally_reserved":0,"allocated":267874304,"data_stored":267535926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355069614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149794,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730680,"kb_used_data":315876,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93641160,"statfs":{"total":96636764160,"available":95888547840,"internally_reserved":0,"allocated":323457024,"data_stored":323180526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":424734381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,16,0,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615332,"kb_used_data":299088,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93756508,"statfs":{"total":96636764160,"available":96006664192,"internally_reserved":0,"allocated":306266112,"data_stored":305985542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322834432,"data_stored":322830819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":266395648,"data_stored":266330155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":305520640,"data_stored":305512955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:41.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:41.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'15545", 'reported_seq': 16348, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.308529+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:59:40.308529+0000', 'last_peered': '2024-09-10T15:59:40.308529+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:59:40.308529+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'5500", 'ondisk_log_start': "44'5500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21401600, 'num_objects': 5225, 'num_object_clones': 0, 'num_object_copies': 10450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5225, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15545, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1613}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:41.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'15179", 'reported_seq': 15653, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:35.556259+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:59:35.556259+0000', 'last_peered': '2024-09-10T15:59:35.556259+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:59:35.556259+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'5100", 'ondisk_log_start': "44'5100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22323224, 'num_objects': 5451, 'num_object_clones': 0, 'num_object_copies': 10902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5451, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13447, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1822}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:41.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:41.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'15149", 'reported_seq': 15915, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.254491+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:59:40.254491+0000', 'last_peered': '2024-09-10T15:59:40.254491+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:59:40.254491+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'5100", 'ondisk_log_start': "44'5100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21041152, 'num_objects': 5137, 'num_object_clones': 0, 'num_object_copies': 10274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5137, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11799, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1613}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:41.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'15341", 'reported_seq': 16161, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.350215+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:59:40.350215+0000', 'last_peered': '2024-09-10T15:59:40.350215+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:59:40.350215+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'5300", 'ondisk_log_start': "44'5300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21450752, 'num_objects': 5237, 'num_object_clones': 0, 'num_object_copies': 10474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12205, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1718}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:41.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'15257", 'reported_seq': 19754, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:35.509627+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:59:35.509627+0000', 'last_peered': '2024-09-10T15:59:35.509627+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:59:35.509627+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'5200", 'ondisk_log_start': "44'5200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22642688, 'num_objects': 5528, 'num_object_clones': 0, 'num_object_copies': 11056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1651, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13231, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2183, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:41.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:41.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'15079", 'reported_seq': 15496, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:35.530095+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:59:35.530095+0000', 'last_peered': '2024-09-10T15:59:35.530095+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:59:35.530095+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'5000", 'ondisk_log_start': "44'5000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22024192, 'num_objects': 5377, 'num_object_clones': 0, 'num_object_copies': 10754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12948, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1691}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:41.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:41.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'15507", 'reported_seq': 16349, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.360515+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:59:40.360515+0000', 'last_peered': '2024-09-10T15:59:40.360515+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:59:40.360515+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'5500", 'ondisk_log_start': "44'5500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21901312, 'num_objects': 5347, 'num_object_clones': 0, 'num_object_copies': 10694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5347, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12699, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1869}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:41.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'15506", 'reported_seq': 16257, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.391227+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:59:40.391227+0000', 'last_peered': '2024-09-10T15:59:40.391227+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:59:40.391227+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'5500", 'ondisk_log_start': "44'5500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21348352, 'num_objects': 5212, 'num_object_clones': 0, 'num_object_copies': 10424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5212, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11924, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1586}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:41.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'15435", 'reported_seq': 16262, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.383842+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:59:40.383842+0000', 'last_peered': '2024-09-10T15:59:40.383842+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:59:40.383842+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'5400", 'ondisk_log_start': "44'5400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21835776, 'num_objects': 5331, 'num_object_clones': 0, 'num_object_copies': 10662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5331, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12041, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1795}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:41.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'15745", 'reported_seq': 16506, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.274834+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:59:40.274834+0000', 'last_peered': '2024-09-10T15:59:40.274834+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:59:40.274834+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'5700", 'ondisk_log_start': "44'5700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21926371, 'num_objects': 5354, 'num_object_clones': 0, 'num_object_copies': 10708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5354, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12564, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1660}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:41.080 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:41.771 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:41.771 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33971/170994 objects degraded (19.867%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":507,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":85497,"data_bytes":350179854,"bytes_used":2001281024,"bytes_avail":287909011456,"bytes_total":289910292480,"degraded_objects":33971,"degraded_total":170994,"degraded_ratio":0.19866778951308234,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":525},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T15:59:36.565732+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:59:41.794 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:41.794 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:42.322 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:42.323 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:42.336 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-10T15:59:40.567031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":350179854,"num_objects":85497,"num_object_clones":0,"num_object_copies":170994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85497,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207557,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12679,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160637,"ondisk_log_size":160637,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1954376,"kb_used_data":876560,"kb_used_omap":12,"kb_used_meta":1077747,"kb_avail":281161144,"statfs":{"total":289910292480,"available":287909011456,"internally_reserved":0,"allocated":897597440,"data_stored":896701994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1103612933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,18,2,9,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25845760,"num_objects":-6310,"num_object_clones":0,"num_object_copies":-12620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6310,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004065"},"pg_stats":[{"pgid":"2.f","version":"48'15545","reported_seq":16348,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.308529+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:40.308529+0000","last_peered":"2024-09-10T15:59:40.308529+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:40.308529+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21401600,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15545,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1613}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15556","reported_seq":15668,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.489046+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:35.489046+0000","last_peered":"2024-09-10T15:59:35.489046+0000","last_clean":"2024-09-10T15:59:35.489046+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:35.489046+0000","last_undegraded":"2024-09-10T15:59:35.489046+0000","last_fullsized":"2024-09-10T15:59:35.489046+0000","mapping_epoch":36,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13892,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15179","reported_seq":15653,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.556259+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:35.556259+0000","last_peered":"2024-09-10T15:59:35.556259+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:35.556259+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22323224,"num_objects":5451,"num_object_clones":0,"num_object_copies":10902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13447,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1822}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15229","reported_seq":15313,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.521322+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:35.521322+0000","last_peered":"2024-09-10T15:59:35.521322+0000","last_clean":"2024-09-10T15:59:35.521322+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:35.521322+0000","last_undegraded":"2024-09-10T15:59:35.521322+0000","last_fullsized":"2024-09-10T15:59:35.521322+0000","mapping_epoch":36,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22335488,"num_objects":5453,"num_object_clones":0,"num_object_copies":10906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15229,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15159","reported_seq":15291,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.347249+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:40.347249+0000","last_peered":"2024-09-10T15:59:40.347249+0000","last_clean":"2024-09-10T15:59:40.347249+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:40.347249+0000","last_undegraded":"2024-09-10T15:59:40.347249+0000","last_fullsized":"2024-09-10T15:59:40.347249+0000","mapping_epoch":35,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21184512,"num_objects":5172,"num_object_clones":0,"num_object_copies":10344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12230,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15326","reported_seq":23219,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.602076+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:35.602076+0000","last_peered":"2024-09-10T15:59:35.602076+0000","last_clean":"2024-09-10T15:59:35.602076+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:35.602076+0000","last_undegraded":"2024-09-10T15:59:35.602076+0000","last_fullsized":"2024-09-10T15:59:35.602076+0000","mapping_epoch":47,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22695936,"num_objects":5541,"num_object_clones":0,"num_object_copies":11082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12044,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15149","reported_seq":15915,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.254491+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:40.254491+0000","last_peered":"2024-09-10T15:59:40.254491+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:40.254491+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21041152,"num_objects":5137,"num_object_clones":0,"num_object_copies":10274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11799,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1613}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15607","reported_seq":15743,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.382842+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:40.382842+0000","last_peered":"2024-09-10T15:59:40.382842+0000","last_clean":"2024-09-10T15:59:40.382842+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:40.382842+0000","last_undegraded":"2024-09-10T15:59:40.382842+0000","last_fullsized":"2024-09-10T15:59:40.382842+0000","mapping_epoch":35,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21737472,"num_objects":5307,"num_object_clones":0,"num_object_copies":10614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12366,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15414","reported_seq":23547,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:35.643058+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:35.643058+0000","last_peered":"2024-09-10T15:59:35.643058+0000","last_clean":"2024-09-10T15:59:35.643058+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:35.643058+0000","last_undegraded":"2024-09-10T15:59:35.643058+0000","last_fullsized":"2024-09-10T15:59:35.643058+0000","mapping_epoch":45,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21729280,"num_objects":5305,"num_object_clones":0,"num_object_copies":10610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13391,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15341","reported_seq":16161,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.350215+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:40.350215+0000","last_peered":"2024-09-10T15:59:40.350215+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:40.350215+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21450752,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12205,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1718}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15257","reported_seq":19754,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.509627+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:35.509627+0000","last_peered":"2024-09-10T15:59:35.509627+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:35.509627+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22642688,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13231,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2183,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3877}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15079","reported_seq":15496,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:35.530095+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:35.530095+0000","last_peered":"2024-09-10T15:59:35.530095+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:35.530095+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12948,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1691}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15507","reported_seq":16349,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.360515+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:40.360515+0000","last_peered":"2024-09-10T15:59:40.360515+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:40.360515+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12699,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1869}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15506","reported_seq":16257,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.391227+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:40.391227+0000","last_peered":"2024-09-10T15:59:40.391227+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:40.391227+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21348352,"num_objects":5212,"num_object_clones":0,"num_object_copies":10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11924,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15435","reported_seq":16262,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.383842+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:40.383842+0000","last_peered":"2024-09-10T15:59:40.383842+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:40.383842+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21835776,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12041,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1795}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746545+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746545+0000","last_peered":"2024-09-10T15:58:12.746545+0000","last_clean":"2024-09-10T15:58:12.746545+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746545+0000","last_undegraded":"2024-09-10T15:58:12.746545+0000","last_fullsized":"2024-09-10T15:58:12.746545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15745","reported_seq":16506,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.274834+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:40.274834+0000","last_peered":"2024-09-10T15:59:40.274834+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:40.274834+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21926371,"num_objects":5354,"num_object_clones":0,"num_object_copies":10708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12564,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":350179835,"num_objects":85495,"num_object_clones":0,"num_object_copies":170990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207555,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12677,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":894750720,"data_stored":894673929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160634,"ondisk_log_size":160634,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626511,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608364,"kb_used_data":261596,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93763476,"statfs":{"total":96636764160,"available":96013799424,"internally_reserved":0,"allocated":267874304,"data_stored":267535926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355069614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149794,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730680,"kb_used_data":315876,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93641160,"statfs":{"total":96636764160,"available":95888547840,"internally_reserved":0,"allocated":323457024,"data_stored":323180526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":424734381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,16,0,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615332,"kb_used_data":299088,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93756508,"statfs":{"total":96636764160,"available":96006664192,"internally_reserved":0,"allocated":306266112,"data_stored":305985542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322834432,"data_stored":322830819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":266395648,"data_stored":266330155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":305520640,"data_stored":305512955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:45.339 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:45.891 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:45.891 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:45.917 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-10T15:59:44.568517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345899534,"num_objects":84452,"num_object_clones":0,"num_object_copies":168904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208602,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12780,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160582,"ondisk_log_size":160582,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1938044,"kb_used_data":864388,"kb_used_omap":12,"kb_used_meta":1073587,"kb_avail":281177476,"statfs":{"total":289910292480,"available":287925735424,"internally_reserved":0,"allocated":885133312,"data_stored":884237866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1099353093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,2,18,4,11,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20209664,"num_objects":-4934,"num_object_clones":0,"num_object_copies":-9868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4934,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004385"},"pg_stats":[{"pgid":"2.f","version":"48'15545","reported_seq":16348,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.308529+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:40.308529+0000","last_peered":"2024-09-10T15:59:40.308529+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:40.308529+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21401600,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15545,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1613}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15720","reported_seq":15833,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.521647+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:40.521647+0000","last_peered":"2024-09-10T15:59:40.521647+0000","last_clean":"2024-09-10T15:59:40.521647+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:40.521647+0000","last_undegraded":"2024-09-10T15:59:40.521647+0000","last_fullsized":"2024-09-10T15:59:40.521647+0000","mapping_epoch":36,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21929984,"num_objects":5354,"num_object_clones":0,"num_object_copies":10708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14056,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15311","reported_seq":15787,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.526452+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:40.526452+0000","last_peered":"2024-09-10T15:59:40.526452+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:40.526452+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21782552,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13579,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1690}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15368","reported_seq":15453,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.304105+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:40.304105+0000","last_peered":"2024-09-10T15:59:40.304105+0000","last_clean":"2024-09-10T15:59:40.304105+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:40.304105+0000","last_undegraded":"2024-09-10T15:59:40.304105+0000","last_fullsized":"2024-09-10T15:59:40.304105+0000","mapping_epoch":36,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21766144,"num_objects":5314,"num_object_clones":0,"num_object_copies":10628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15368,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15159","reported_seq":15291,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.347249+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:40.347249+0000","last_peered":"2024-09-10T15:59:40.347249+0000","last_clean":"2024-09-10T15:59:40.347249+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:40.347249+0000","last_undegraded":"2024-09-10T15:59:40.347249+0000","last_fullsized":"2024-09-10T15:59:40.347249+0000","mapping_epoch":35,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21184512,"num_objects":5172,"num_object_clones":0,"num_object_copies":10344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12230,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15497","reported_seq":23390,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.689496+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:40.689496+0000","last_peered":"2024-09-10T15:59:40.689496+0000","last_clean":"2024-09-10T15:59:40.689496+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:40.689496+0000","last_undegraded":"2024-09-10T15:59:40.689496+0000","last_fullsized":"2024-09-10T15:59:40.689496+0000","mapping_epoch":47,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21995520,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12215,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15149","reported_seq":15915,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.254491+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:40.254491+0000","last_peered":"2024-09-10T15:59:40.254491+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:40.254491+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21041152,"num_objects":5137,"num_object_clones":0,"num_object_copies":10274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11799,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1613}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15607","reported_seq":15743,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.382842+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:40.382842+0000","last_peered":"2024-09-10T15:59:40.382842+0000","last_clean":"2024-09-10T15:59:40.382842+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:40.382842+0000","last_undegraded":"2024-09-10T15:59:40.382842+0000","last_fullsized":"2024-09-10T15:59:40.382842+0000","mapping_epoch":35,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21737472,"num_objects":5307,"num_object_clones":0,"num_object_copies":10614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12366,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15557","reported_seq":23690,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.736264+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:40.736264+0000","last_peered":"2024-09-10T15:59:40.736264+0000","last_clean":"2024-09-10T15:59:40.736264+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:40.736264+0000","last_undegraded":"2024-09-10T15:59:40.736264+0000","last_fullsized":"2024-09-10T15:59:40.736264+0000","mapping_epoch":45,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21143552,"num_objects":5162,"num_object_clones":0,"num_object_copies":10324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13534,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15341","reported_seq":16161,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.350215+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:40.350215+0000","last_peered":"2024-09-10T15:59:40.350215+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:40.350215+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21450752,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12205,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1718}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15423","reported_seq":20122,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.561369+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:40.561369+0000","last_peered":"2024-09-10T15:59:40.561369+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:40.561369+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21962752,"num_objects":5362,"num_object_clones":0,"num_object_copies":10724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1550,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13397,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2284,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3812}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15209","reported_seq":15627,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.535885+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:40.535885+0000","last_peered":"2024-09-10T15:59:40.535885+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:40.535885+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21491712,"num_objects":5247,"num_object_clones":0,"num_object_copies":10494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13078,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1561}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15507","reported_seq":16349,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.360515+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:40.360515+0000","last_peered":"2024-09-10T15:59:40.360515+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:40.360515+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12699,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1869}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15506","reported_seq":16257,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.391227+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:40.391227+0000","last_peered":"2024-09-10T15:59:40.391227+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:40.391227+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21348352,"num_objects":5212,"num_object_clones":0,"num_object_copies":10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11924,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15435","reported_seq":16262,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.383842+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:40.383842+0000","last_peered":"2024-09-10T15:59:40.383842+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:40.383842+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21835776,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12041,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1795}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15745","reported_seq":16506,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.274834+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:40.274834+0000","last_peered":"2024-09-10T15:59:40.274834+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:40.274834+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21926371,"num_objects":5354,"num_object_clones":0,"num_object_copies":10708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12564,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":345899515,"num_objects":84450,"num_object_clones":0,"num_object_copies":168900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84450,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208600,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12778,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":882434048,"data_stored":882357257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160579,"ondisk_log_size":160579,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626513,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592676,"kb_used_data":254228,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93779164,"statfs":{"total":96636764160,"available":96029863936,"internally_reserved":0,"allocated":260329472,"data_stored":259991094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346549934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149794,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730680,"kb_used_data":315876,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93641160,"statfs":{"total":96636764160,"available":95888547840,"internally_reserved":0,"allocated":323457024,"data_stored":323180526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":424734381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,16,0,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614688,"kb_used_data":294284,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93757152,"statfs":{"total":96636764160,"available":96007323648,"internally_reserved":0,"allocated":301346816,"data_stored":301066246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322834432,"data_stored":322830819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":258977792,"data_stored":258912299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":300621824,"data_stored":300614139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:45.918 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:46.494 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:46.494 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:46.519 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-10T15:59:44.568517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345899534,"num_objects":84452,"num_object_clones":0,"num_object_copies":168904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208602,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12780,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160582,"ondisk_log_size":160582,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1938044,"kb_used_data":864388,"kb_used_omap":12,"kb_used_meta":1073587,"kb_avail":281177476,"statfs":{"total":289910292480,"available":287925735424,"internally_reserved":0,"allocated":885133312,"data_stored":884237866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1099353093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,2,18,4,11,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20209664,"num_objects":-4934,"num_object_clones":0,"num_object_copies":-9868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4934,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004385"},"pg_stats":[{"pgid":"2.f","version":"48'15545","reported_seq":16348,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.308529+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:40.308529+0000","last_peered":"2024-09-10T15:59:40.308529+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:40.308529+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21401600,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15545,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1613}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15720","reported_seq":15833,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.521647+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:40.521647+0000","last_peered":"2024-09-10T15:59:40.521647+0000","last_clean":"2024-09-10T15:59:40.521647+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:40.521647+0000","last_undegraded":"2024-09-10T15:59:40.521647+0000","last_fullsized":"2024-09-10T15:59:40.521647+0000","mapping_epoch":36,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21929984,"num_objects":5354,"num_object_clones":0,"num_object_copies":10708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14056,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15311","reported_seq":15787,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.526452+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:40.526452+0000","last_peered":"2024-09-10T15:59:40.526452+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:40.526452+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21782552,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13579,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1690}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15368","reported_seq":15453,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.304105+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:40.304105+0000","last_peered":"2024-09-10T15:59:40.304105+0000","last_clean":"2024-09-10T15:59:40.304105+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:40.304105+0000","last_undegraded":"2024-09-10T15:59:40.304105+0000","last_fullsized":"2024-09-10T15:59:40.304105+0000","mapping_epoch":36,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21766144,"num_objects":5314,"num_object_clones":0,"num_object_copies":10628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15368,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15159","reported_seq":15291,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.347249+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:40.347249+0000","last_peered":"2024-09-10T15:59:40.347249+0000","last_clean":"2024-09-10T15:59:40.347249+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:40.347249+0000","last_undegraded":"2024-09-10T15:59:40.347249+0000","last_fullsized":"2024-09-10T15:59:40.347249+0000","mapping_epoch":35,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21184512,"num_objects":5172,"num_object_clones":0,"num_object_copies":10344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12230,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15497","reported_seq":23390,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.689496+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:40.689496+0000","last_peered":"2024-09-10T15:59:40.689496+0000","last_clean":"2024-09-10T15:59:40.689496+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:40.689496+0000","last_undegraded":"2024-09-10T15:59:40.689496+0000","last_fullsized":"2024-09-10T15:59:40.689496+0000","mapping_epoch":47,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21995520,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12215,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15149","reported_seq":15915,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.254491+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:40.254491+0000","last_peered":"2024-09-10T15:59:40.254491+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:40.254491+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21041152,"num_objects":5137,"num_object_clones":0,"num_object_copies":10274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11799,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1613}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15607","reported_seq":15743,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.382842+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:40.382842+0000","last_peered":"2024-09-10T15:59:40.382842+0000","last_clean":"2024-09-10T15:59:40.382842+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:40.382842+0000","last_undegraded":"2024-09-10T15:59:40.382842+0000","last_fullsized":"2024-09-10T15:59:40.382842+0000","mapping_epoch":35,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21737472,"num_objects":5307,"num_object_clones":0,"num_object_copies":10614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12366,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15557","reported_seq":23690,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:40.736264+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:40.736264+0000","last_peered":"2024-09-10T15:59:40.736264+0000","last_clean":"2024-09-10T15:59:40.736264+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:40.736264+0000","last_undegraded":"2024-09-10T15:59:40.736264+0000","last_fullsized":"2024-09-10T15:59:40.736264+0000","mapping_epoch":45,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21143552,"num_objects":5162,"num_object_clones":0,"num_object_copies":10324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13534,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15341","reported_seq":16161,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.350215+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:40.350215+0000","last_peered":"2024-09-10T15:59:40.350215+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:40.350215+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21450752,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12205,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1718}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15423","reported_seq":20122,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.561369+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:40.561369+0000","last_peered":"2024-09-10T15:59:40.561369+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:40.561369+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21962752,"num_objects":5362,"num_object_clones":0,"num_object_copies":10724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1550,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13397,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2284,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3812}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15209","reported_seq":15627,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.535885+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:40.535885+0000","last_peered":"2024-09-10T15:59:40.535885+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:40.535885+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21491712,"num_objects":5247,"num_object_clones":0,"num_object_copies":10494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13078,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1561}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15507","reported_seq":16349,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.360515+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:40.360515+0000","last_peered":"2024-09-10T15:59:40.360515+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:40.360515+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12699,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1869}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15506","reported_seq":16257,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.391227+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:40.391227+0000","last_peered":"2024-09-10T15:59:40.391227+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:40.391227+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21348352,"num_objects":5212,"num_object_clones":0,"num_object_copies":10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11924,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15435","reported_seq":16262,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.383842+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:40.383842+0000","last_peered":"2024-09-10T15:59:40.383842+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:40.383842+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21835776,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12041,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1795}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746547+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746547+0000","last_peered":"2024-09-10T15:58:12.746547+0000","last_clean":"2024-09-10T15:58:12.746547+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746547+0000","last_undegraded":"2024-09-10T15:58:12.746547+0000","last_fullsized":"2024-09-10T15:58:12.746547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15745","reported_seq":16506,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:40.274834+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:40.274834+0000","last_peered":"2024-09-10T15:59:40.274834+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:40.274834+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21926371,"num_objects":5354,"num_object_clones":0,"num_object_copies":10708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12564,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":345899515,"num_objects":84450,"num_object_clones":0,"num_object_copies":168900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33870,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84450,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208600,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12778,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":882434048,"data_stored":882357257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160579,"ondisk_log_size":160579,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626513,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592676,"kb_used_data":254228,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93779164,"statfs":{"total":96636764160,"available":96029863936,"internally_reserved":0,"allocated":260329472,"data_stored":259991094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346549934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149794,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730680,"kb_used_data":315876,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93641160,"statfs":{"total":96636764160,"available":95888547840,"internally_reserved":0,"allocated":323457024,"data_stored":323180526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":424734381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,16,0,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614688,"kb_used_data":294284,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93757152,"statfs":{"total":96636764160,"available":96007323648,"internally_reserved":0,"allocated":301346816,"data_stored":301066246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322834432,"data_stored":322830819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":258977792,"data_stored":258912299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":300621824,"data_stored":300614139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:46.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:46.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'15545", 'reported_seq': 16348, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.308529+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:59:40.308529+0000', 'last_peered': '2024-09-10T15:59:40.308529+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:59:40.308529+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'5500", 'ondisk_log_start': "44'5500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21401600, 'num_objects': 5225, 'num_object_clones': 0, 'num_object_copies': 10450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5225, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15545, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1613}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:46.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:46.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'15311", 'reported_seq': 15787, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.526452+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:59:40.526452+0000', 'last_peered': '2024-09-10T15:59:40.526452+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:59:40.526452+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'5300", 'ondisk_log_start': "44'5300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21782552, 'num_objects': 5319, 'num_object_clones': 0, 'num_object_copies': 10638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5319, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13579, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1690}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:46.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:46.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'15149", 'reported_seq': 15915, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.254491+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:59:40.254491+0000', 'last_peered': '2024-09-10T15:59:40.254491+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:59:40.254491+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'5100", 'ondisk_log_start': "44'5100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21041152, 'num_objects': 5137, 'num_object_clones': 0, 'num_object_copies': 10274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5137, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11799, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1613}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:46.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:46.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'15341", 'reported_seq': 16161, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.350215+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:59:40.350215+0000', 'last_peered': '2024-09-10T15:59:40.350215+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:59:40.350215+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'5300", 'ondisk_log_start': "44'5300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21450752, 'num_objects': 5237, 'num_object_clones': 0, 'num_object_copies': 10474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12205, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1718}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:46.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:46.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'15423", 'reported_seq': 20122, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.561369+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:59:40.561369+0000', 'last_peered': '2024-09-10T15:59:40.561369+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:59:40.561369+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'5400", 'ondisk_log_start': "44'5400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21962752, 'num_objects': 5362, 'num_object_clones': 0, 'num_object_copies': 10724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1550, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5362, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13397, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2284, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3812}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:46.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:46.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'15209", 'reported_seq': 15627, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.535885+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:59:40.535885+0000', 'last_peered': '2024-09-10T15:59:40.535885+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:59:40.535885+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'5200", 'ondisk_log_start': "44'5200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21491712, 'num_objects': 5247, 'num_object_clones': 0, 'num_object_copies': 10494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13078, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1561}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:46.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:46.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'15507", 'reported_seq': 16349, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.360515+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:59:40.360515+0000', 'last_peered': '2024-09-10T15:59:40.360515+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:59:40.360515+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'5500", 'ondisk_log_start': "44'5500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21901312, 'num_objects': 5347, 'num_object_clones': 0, 'num_object_copies': 10694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5347, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12699, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1869}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:46.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:46.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'15506", 'reported_seq': 16257, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.391227+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:59:40.391227+0000', 'last_peered': '2024-09-10T15:59:40.391227+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:59:40.391227+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'5500", 'ondisk_log_start': "44'5500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21348352, 'num_objects': 5212, 'num_object_clones': 0, 'num_object_copies': 10424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5212, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11924, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1586}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:46.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:46.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'15435", 'reported_seq': 16262, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.383842+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:59:40.383842+0000', 'last_peered': '2024-09-10T15:59:40.383842+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:59:40.383842+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'5400", 'ondisk_log_start': "44'5400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21835776, 'num_objects': 5331, 'num_object_clones': 0, 'num_object_copies': 10662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5331, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12041, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1795}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:46.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:46.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'15745", 'reported_seq': 16506, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:40.274834+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:59:40.274834+0000', 'last_peered': '2024-09-10T15:59:40.274834+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:59:40.274834+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'5700", 'ondisk_log_start': "44'5700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21926371, 'num_objects': 5354, 'num_object_clones': 0, 'num_object_copies': 10708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5354, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12564, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1660}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:46.523 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:47.181 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:47.181 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33870/168904 objects degraded (20.053%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":513,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":84452,"data_bytes":345899534,"bytes_used":1984557056,"bytes_avail":287925735424,"bytes_total":289910292480,"degraded_objects":33870,"degraded_total":168904,"degraded_ratio":0.20052811064273196,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":411},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T15:59:36.565732+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:59:47.205 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:47.205 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:47.767 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:47.767 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:47.793 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-10T15:59:46.569151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":336085518,"num_objects":82056,"num_object_clones":0,"num_object_copies":164112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33737,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":210998,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12913,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160978,"ondisk_log_size":160978,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1931444,"kb_used_data":845288,"kb_used_omap":12,"kb_used_meta":1086067,"kb_avail":281184076,"statfs":{"total":289910292480,"available":287932493824,"internally_reserved":0,"allocated":865574912,"data_stored":864679466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1112132613},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,4,14,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30023680,"num_objects":-7330,"num_object_clones":0,"num_object_copies":-14660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7330,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004285"},"pg_stats":[{"pgid":"2.f","version":"48'15672","reported_seq":16476,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.305560+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:45.305560+0000","last_peered":"2024-09-10T15:59:45.305560+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:45.305560+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20881408,"num_objects":5098,"num_object_clones":0,"num_object_copies":10196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15672,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1486}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15883","reported_seq":15996,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.569383+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:45.569383+0000","last_peered":"2024-09-10T15:59:45.569383+0000","last_clean":"2024-09-10T15:59:45.569383+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:45.569383+0000","last_undegraded":"2024-09-10T15:59:45.569383+0000","last_fullsized":"2024-09-10T15:59:45.569383+0000","mapping_epoch":36,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21262336,"num_objects":5191,"num_object_clones":0,"num_object_copies":10382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14219,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15445","reported_seq":15920,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.501269+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:45.501269+0000","last_peered":"2024-09-10T15:59:45.501269+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:45.501269+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233688,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13713,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15527","reported_seq":15612,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.419988+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:45.419988+0000","last_peered":"2024-09-10T15:59:45.419988+0000","last_clean":"2024-09-10T15:59:45.419988+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:45.419988+0000","last_undegraded":"2024-09-10T15:59:45.419988+0000","last_fullsized":"2024-09-10T15:59:45.419988+0000","mapping_epoch":36,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21114880,"num_objects":5155,"num_object_clones":0,"num_object_copies":10310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15527,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15322","reported_seq":15454,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.385341+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:45.385341+0000","last_peered":"2024-09-10T15:59:45.385341+0000","last_clean":"2024-09-10T15:59:45.385341+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:45.385341+0000","last_undegraded":"2024-09-10T15:59:45.385341+0000","last_fullsized":"2024-09-10T15:59:45.385341+0000","mapping_epoch":35,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20516864,"num_objects":5009,"num_object_clones":0,"num_object_copies":10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12393,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15673","reported_seq":23566,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.769168+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:45.769168+0000","last_peered":"2024-09-10T15:59:45.769168+0000","last_clean":"2024-09-10T15:59:45.769168+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:45.769168+0000","last_undegraded":"2024-09-10T15:59:45.769168+0000","last_fullsized":"2024-09-10T15:59:45.769168+0000","mapping_epoch":47,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21274624,"num_objects":5194,"num_object_clones":0,"num_object_copies":10388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12391,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15298","reported_seq":16064,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.325512+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:45.325512+0000","last_peered":"2024-09-10T15:59:45.325512+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:45.325512+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20430848,"num_objects":4988,"num_object_clones":0,"num_object_copies":9976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11948,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1464}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15738","reported_seq":15875,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.333132+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:45.333132+0000","last_peered":"2024-09-10T15:59:45.333132+0000","last_clean":"2024-09-10T15:59:45.333132+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:45.333132+0000","last_undegraded":"2024-09-10T15:59:45.333132+0000","last_fullsized":"2024-09-10T15:59:45.333132+0000","mapping_epoch":35,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21200896,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12497,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15718","reported_seq":23851,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.713468+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:45.713468+0000","last_peered":"2024-09-10T15:59:45.713468+0000","last_clean":"2024-09-10T15:59:45.713468+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:45.713468+0000","last_undegraded":"2024-09-10T15:59:45.713468+0000","last_fullsized":"2024-09-10T15:59:45.713468+0000","mapping_epoch":45,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20484096,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15491","reported_seq":16311,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.310971+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:45.310971+0000","last_peered":"2024-09-10T15:59:45.310971+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:45.310971+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20836352,"num_objects":5087,"num_object_clones":0,"num_object_copies":10174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12355,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1568}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15578","reported_seq":20543,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.572036+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:45.572036+0000","last_peered":"2024-09-10T15:59:45.572036+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:45.572036+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21327872,"num_objects":5207,"num_object_clones":0,"num_object_copies":10414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13552,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2417,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3790}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15366","reported_seq":15784,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.483356+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:45.483356+0000","last_peered":"2024-09-10T15:59:45.483356+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:45.483356+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20848640,"num_objects":5090,"num_object_clones":0,"num_object_copies":10180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13235,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1404}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15656","reported_seq":16498,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.354648+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:45.354648+0000","last_peered":"2024-09-10T15:59:45.354648+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:45.354648+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21291008,"num_objects":5198,"num_object_clones":0,"num_object_copies":10396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12848,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1720}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15647","reported_seq":16398,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.395998+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:45.395998+0000","last_peered":"2024-09-10T15:59:45.395998+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:45.395998+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20770816,"num_objects":5071,"num_object_clones":0,"num_object_copies":10142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12065,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1445}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15592","reported_seq":16419,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.380253+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:45.380253+0000","last_peered":"2024-09-10T15:59:45.380253+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:45.380253+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21192704,"num_objects":5174,"num_object_clones":0,"num_object_copies":10348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12198,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'15869","reported_seq":16631,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.383954+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:45.383954+0000","last_peered":"2024-09-10T15:59:45.383954+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:45.383954+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21418467,"num_objects":5230,"num_object_clones":0,"num_object_copies":10460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12688,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":336085499,"num_objects":82054,"num_object_clones":0,"num_object_copies":164108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33737,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82054,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":210996,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12911,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":862822400,"data_stored":862745609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160975,"ondisk_log_size":160975,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626514,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589948,"kb_used_data":247328,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93781892,"statfs":{"total":96636764160,"available":96032657408,"internally_reserved":0,"allocated":253263872,"data_stored":252925494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350809774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149795,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":727588,"kb_used_data":308620,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93644252,"statfs":{"total":96636764160,"available":95891714048,"internally_reserved":0,"allocated":316026880,"data_stored":315750382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":428994221},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613908,"kb_used_data":289340,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93757932,"statfs":{"total":96636764160,"available":96008122368,"internally_reserved":0,"allocated":296284160,"data_stored":296003590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315518976,"data_stored":315515363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":251691008,"data_stored":251625515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":295612416,"data_stored":295604731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:50.795 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:51.359 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:51.359 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:51.383 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-10T15:59:50.573426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":330727950,"num_objects":80748,"num_object_clones":0,"num_object_copies":161496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33737,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212306,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12913,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160786,"ondisk_log_size":160786,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1916208,"kb_used_data":838372,"kb_used_omap":12,"kb_used_meta":1077747,"kb_avail":281199312,"statfs":{"total":289910292480,"available":287948095488,"internally_reserved":0,"allocated":858492928,"data_stored":857597482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1103612933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,10,6,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25133056,"num_objects":-6136,"num_object_clones":0,"num_object_copies":-12272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6136,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006887"},"pg_stats":[{"pgid":"2.f","version":"48'15808","reported_seq":16612,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.433239+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:50.433239+0000","last_peered":"2024-09-10T15:59:50.433239+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:50.433239+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20324352,"num_objects":4962,"num_object_clones":0,"num_object_copies":9924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15808,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15883","reported_seq":15996,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.569383+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:45.569383+0000","last_peered":"2024-09-10T15:59:45.569383+0000","last_clean":"2024-09-10T15:59:45.569383+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:45.569383+0000","last_undegraded":"2024-09-10T15:59:45.569383+0000","last_fullsized":"2024-09-10T15:59:45.569383+0000","mapping_epoch":36,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21262336,"num_objects":5191,"num_object_clones":0,"num_object_copies":10382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14219,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15445","reported_seq":15920,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.501269+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:45.501269+0000","last_peered":"2024-09-10T15:59:45.501269+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:45.501269+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233688,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13713,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15527","reported_seq":15612,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.419988+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:45.419988+0000","last_peered":"2024-09-10T15:59:45.419988+0000","last_clean":"2024-09-10T15:59:45.419988+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:45.419988+0000","last_undegraded":"2024-09-10T15:59:45.419988+0000","last_fullsized":"2024-09-10T15:59:45.419988+0000","mapping_epoch":36,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21114880,"num_objects":5155,"num_object_clones":0,"num_object_copies":10310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15527,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15462","reported_seq":15594,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:50.370505+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:50.370505+0000","last_peered":"2024-09-10T15:59:50.370505+0000","last_clean":"2024-09-10T15:59:50.370505+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:50.370505+0000","last_undegraded":"2024-09-10T15:59:50.370505+0000","last_fullsized":"2024-09-10T15:59:50.370505+0000","mapping_epoch":35,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19943424,"num_objects":4869,"num_object_clones":0,"num_object_copies":9738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12533,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15673","reported_seq":23566,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.769168+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:45.769168+0000","last_peered":"2024-09-10T15:59:45.769168+0000","last_clean":"2024-09-10T15:59:45.769168+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:45.769168+0000","last_undegraded":"2024-09-10T15:59:45.769168+0000","last_fullsized":"2024-09-10T15:59:45.769168+0000","mapping_epoch":47,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21274624,"num_objects":5194,"num_object_clones":0,"num_object_copies":10388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12391,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15432","reported_seq":16198,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.230242+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:50.230242+0000","last_peered":"2024-09-10T15:59:50.230242+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:50.230242+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19881984,"num_objects":4854,"num_object_clones":0,"num_object_copies":9708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12082,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1330}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15890","reported_seq":16027,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:50.408985+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:50.408985+0000","last_peered":"2024-09-10T15:59:50.408985+0000","last_clean":"2024-09-10T15:59:50.408985+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:50.408985+0000","last_undegraded":"2024-09-10T15:59:50.408985+0000","last_fullsized":"2024-09-10T15:59:50.408985+0000","mapping_epoch":35,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":5024,"num_object_clones":0,"num_object_copies":10048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12649,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15718","reported_seq":23851,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.713468+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:45.713468+0000","last_peered":"2024-09-10T15:59:45.713468+0000","last_clean":"2024-09-10T15:59:45.713468+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:45.713468+0000","last_undegraded":"2024-09-10T15:59:45.713468+0000","last_fullsized":"2024-09-10T15:59:45.713468+0000","mapping_epoch":45,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20484096,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15626","reported_seq":16445,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.344089+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:50.344089+0000","last_peered":"2024-09-10T15:59:50.344089+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:50.344089+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12490,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1433}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15578","reported_seq":20543,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.572036+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:45.572036+0000","last_peered":"2024-09-10T15:59:45.572036+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:45.572036+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21327872,"num_objects":5207,"num_object_clones":0,"num_object_copies":10414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13552,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2417,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3790}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15366","reported_seq":15784,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.483356+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:45.483356+0000","last_peered":"2024-09-10T15:59:45.483356+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:45.483356+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20848640,"num_objects":5090,"num_object_clones":0,"num_object_copies":10180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13235,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1404}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15828","reported_seq":16671,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.385900+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:50.385900+0000","last_peered":"2024-09-10T15:59:50.385900+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:50.385900+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20586496,"num_objects":5026,"num_object_clones":0,"num_object_copies":10052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13020,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1548}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15783","reported_seq":16534,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.379310+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:50.379310+0000","last_peered":"2024-09-10T15:59:50.379310+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:50.379310+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20213760,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12201,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15727","reported_seq":16554,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.372619+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:50.372619+0000","last_peered":"2024-09-10T15:59:50.372619+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:50.372619+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20639744,"num_objects":5039,"num_object_clones":0,"num_object_copies":10078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12333,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16037","reported_seq":16797,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.371113+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:50.371113+0000","last_peered":"2024-09-10T15:59:50.371113+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:50.371113+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20730339,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12856,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1368}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":330727931,"num_objects":80746,"num_object_clones":0,"num_object_copies":161492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33737,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212304,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12911,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":855666688,"data_stored":855589897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626514,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589948,"kb_used_data":247328,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93781892,"statfs":{"total":96636764160,"available":96032657408,"internally_reserved":0,"allocated":253263872,"data_stored":252925494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350809774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149796,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712352,"kb_used_data":301704,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93659488,"statfs":{"total":96636764160,"available":95907315712,"internally_reserved":0,"allocated":308944896,"data_stored":308668398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":420474541},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613908,"kb_used_data":289340,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93757932,"statfs":{"total":96636764160,"available":96008122368,"internally_reserved":0,"allocated":296284160,"data_stored":296003590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":308363264,"data_stored":308359651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":251691008,"data_stored":251625515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":295612416,"data_stored":295604731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:51.384 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:51.955 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:51.956 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:51.976 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-10T15:59:50.573426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":330727950,"num_objects":80748,"num_object_clones":0,"num_object_copies":161496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33737,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212306,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12913,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160786,"ondisk_log_size":160786,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1916208,"kb_used_data":838372,"kb_used_omap":12,"kb_used_meta":1077747,"kb_avail":281199312,"statfs":{"total":289910292480,"available":287948095488,"internally_reserved":0,"allocated":858492928,"data_stored":857597482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1103612933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,10,6,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25133056,"num_objects":-6136,"num_object_clones":0,"num_object_copies":-12272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6136,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006887"},"pg_stats":[{"pgid":"2.f","version":"48'15808","reported_seq":16612,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.433239+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:50.433239+0000","last_peered":"2024-09-10T15:59:50.433239+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:50.433239+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20324352,"num_objects":4962,"num_object_clones":0,"num_object_copies":9924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15808,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'15883","reported_seq":15996,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.569383+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:45.569383+0000","last_peered":"2024-09-10T15:59:45.569383+0000","last_clean":"2024-09-10T15:59:45.569383+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:45.569383+0000","last_undegraded":"2024-09-10T15:59:45.569383+0000","last_fullsized":"2024-09-10T15:59:45.569383+0000","mapping_epoch":36,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21262336,"num_objects":5191,"num_object_clones":0,"num_object_copies":10382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14219,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15445","reported_seq":15920,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.501269+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:45.501269+0000","last_peered":"2024-09-10T15:59:45.501269+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:45.501269+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233688,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13713,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15527","reported_seq":15612,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.419988+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:45.419988+0000","last_peered":"2024-09-10T15:59:45.419988+0000","last_clean":"2024-09-10T15:59:45.419988+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:45.419988+0000","last_undegraded":"2024-09-10T15:59:45.419988+0000","last_fullsized":"2024-09-10T15:59:45.419988+0000","mapping_epoch":36,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21114880,"num_objects":5155,"num_object_clones":0,"num_object_copies":10310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15527,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15462","reported_seq":15594,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:50.370505+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:50.370505+0000","last_peered":"2024-09-10T15:59:50.370505+0000","last_clean":"2024-09-10T15:59:50.370505+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:50.370505+0000","last_undegraded":"2024-09-10T15:59:50.370505+0000","last_fullsized":"2024-09-10T15:59:50.370505+0000","mapping_epoch":35,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19943424,"num_objects":4869,"num_object_clones":0,"num_object_copies":9738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12533,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15673","reported_seq":23566,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.769168+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:45.769168+0000","last_peered":"2024-09-10T15:59:45.769168+0000","last_clean":"2024-09-10T15:59:45.769168+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:45.769168+0000","last_undegraded":"2024-09-10T15:59:45.769168+0000","last_fullsized":"2024-09-10T15:59:45.769168+0000","mapping_epoch":47,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21274624,"num_objects":5194,"num_object_clones":0,"num_object_copies":10388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12391,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15432","reported_seq":16198,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.230242+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:50.230242+0000","last_peered":"2024-09-10T15:59:50.230242+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:50.230242+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19881984,"num_objects":4854,"num_object_clones":0,"num_object_copies":9708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12082,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1330}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15890","reported_seq":16027,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:50.408985+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:50.408985+0000","last_peered":"2024-09-10T15:59:50.408985+0000","last_clean":"2024-09-10T15:59:50.408985+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:50.408985+0000","last_undegraded":"2024-09-10T15:59:50.408985+0000","last_fullsized":"2024-09-10T15:59:50.408985+0000","mapping_epoch":35,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":5024,"num_object_clones":0,"num_object_copies":10048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12649,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15718","reported_seq":23851,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:45.713468+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:45.713468+0000","last_peered":"2024-09-10T15:59:45.713468+0000","last_clean":"2024-09-10T15:59:45.713468+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:45.713468+0000","last_undegraded":"2024-09-10T15:59:45.713468+0000","last_fullsized":"2024-09-10T15:59:45.713468+0000","mapping_epoch":45,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20484096,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15626","reported_seq":16445,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.344089+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:50.344089+0000","last_peered":"2024-09-10T15:59:50.344089+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:50.344089+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12490,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1433}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15578","reported_seq":20543,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.572036+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:45.572036+0000","last_peered":"2024-09-10T15:59:45.572036+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:45.572036+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21327872,"num_objects":5207,"num_object_clones":0,"num_object_copies":10414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13552,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2417,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3790}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15366","reported_seq":15784,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:45.483356+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:45.483356+0000","last_peered":"2024-09-10T15:59:45.483356+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:45.483356+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20848640,"num_objects":5090,"num_object_clones":0,"num_object_copies":10180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13235,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1404}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15828","reported_seq":16671,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.385900+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:50.385900+0000","last_peered":"2024-09-10T15:59:50.385900+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:50.385900+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20586496,"num_objects":5026,"num_object_clones":0,"num_object_copies":10052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13020,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1548}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15783","reported_seq":16534,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.379310+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:50.379310+0000","last_peered":"2024-09-10T15:59:50.379310+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:50.379310+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20213760,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12201,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15727","reported_seq":16554,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.372619+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:50.372619+0000","last_peered":"2024-09-10T15:59:50.372619+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:50.372619+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20639744,"num_objects":5039,"num_object_clones":0,"num_object_copies":10078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12333,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16037","reported_seq":16797,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.371113+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:50.371113+0000","last_peered":"2024-09-10T15:59:50.371113+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:50.371113+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20730339,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12856,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1368}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":330727931,"num_objects":80746,"num_object_clones":0,"num_object_copies":161492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33737,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212304,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12911,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":855666688,"data_stored":855589897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626514,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589948,"kb_used_data":247328,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93781892,"statfs":{"total":96636764160,"available":96032657408,"internally_reserved":0,"allocated":253263872,"data_stored":252925494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350809774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149796,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712352,"kb_used_data":301704,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93659488,"statfs":{"total":96636764160,"available":95907315712,"internally_reserved":0,"allocated":308944896,"data_stored":308668398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":420474541},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613908,"kb_used_data":289340,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93757932,"statfs":{"total":96636764160,"available":96008122368,"internally_reserved":0,"allocated":296284160,"data_stored":296003590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":308363264,"data_stored":308359651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":251691008,"data_stored":251625515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":295612416,"data_stored":295604731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:51.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:51.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'15808", 'reported_seq': 16612, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:50.433239+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:59:50.433239+0000', 'last_peered': '2024-09-10T15:59:50.433239+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:59:50.433239+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'5800", 'ondisk_log_start': "44'5800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20324352, 'num_objects': 4962, 'num_object_clones': 0, 'num_object_copies': 9924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4962, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15808, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1350}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:51.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:51.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'15445", 'reported_seq': 15920, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:45.501269+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:59:45.501269+0000', 'last_peered': '2024-09-10T15:59:45.501269+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:59:45.501269+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'5400", 'ondisk_log_start': "44'5400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21233688, 'num_objects': 5185, 'num_object_clones': 0, 'num_object_copies': 10370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5185, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13713, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1556}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:51.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:51.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'15432", 'reported_seq': 16198, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:50.230242+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:59:50.230242+0000', 'last_peered': '2024-09-10T15:59:50.230242+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:59:50.230242+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'5400", 'ondisk_log_start': "44'5400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19881984, 'num_objects': 4854, 'num_object_clones': 0, 'num_object_copies': 9708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4854, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12082, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1330}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:51.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:51.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'15626", 'reported_seq': 16445, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:50.344089+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:59:50.344089+0000', 'last_peered': '2024-09-10T15:59:50.344089+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:59:50.344089+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'5600", 'ondisk_log_start': "44'5600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20283392, 'num_objects': 4952, 'num_object_clones': 0, 'num_object_copies': 9904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12490, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1433}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:51.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:51.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'15578", 'reported_seq': 20543, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:45.572036+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:59:45.572036+0000', 'last_peered': '2024-09-10T15:59:45.572036+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:59:45.572036+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'5500", 'ondisk_log_start': "44'5500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21327872, 'num_objects': 5207, 'num_object_clones': 0, 'num_object_copies': 10414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1417, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13552, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2417, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3790}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:51.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:51.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'15366", 'reported_seq': 15784, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:45.483356+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:59:45.483356+0000', 'last_peered': '2024-09-10T15:59:45.483356+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:59:45.483356+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'5300", 'ondisk_log_start': "44'5300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20848640, 'num_objects': 5090, 'num_object_clones': 0, 'num_object_copies': 10180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5090, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13235, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1404}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:51.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:51.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'15828", 'reported_seq': 16671, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:50.385900+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:59:50.385900+0000', 'last_peered': '2024-09-10T15:59:50.385900+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:59:50.385900+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'5800", 'ondisk_log_start': "44'5800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20586496, 'num_objects': 5026, 'num_object_clones': 0, 'num_object_copies': 10052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13020, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1548}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:51.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:51.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'15783", 'reported_seq': 16534, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:50.379310+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:59:50.379310+0000', 'last_peered': '2024-09-10T15:59:50.379310+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:59:50.379310+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'5700", 'ondisk_log_start': "44'5700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20213760, 'num_objects': 4935, 'num_object_clones': 0, 'num_object_copies': 9870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4935, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12201, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1309}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:51.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:51.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'15727", 'reported_seq': 16554, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:50.372619+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:59:50.372619+0000', 'last_peered': '2024-09-10T15:59:50.372619+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:59:50.372619+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'5700", 'ondisk_log_start': "44'5700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20639744, 'num_objects': 5039, 'num_object_clones': 0, 'num_object_copies': 10078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5039, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12333, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1503}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:51.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:51.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'16037", 'reported_seq': 16797, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:50.371113+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:59:50.371113+0000', 'last_peered': '2024-09-10T15:59:50.371113+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:59:50.371113+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'6000", 'ondisk_log_start': "44'6000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20730339, 'num_objects': 5062, 'num_object_clones': 0, 'num_object_copies': 10124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5062, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12856, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1368}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:51.979 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:52.679 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:52.679 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33603/159372 objects degraded (21.085%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":518,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":79686,"data_bytes":326377998,"bytes_used":1964105728,"bytes_avail":287946186752,"bytes_total":289910292480,"degraded_objects":33603,"degraded_total":159372,"degraded_ratio":0.21084632181311652,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":368,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":483},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T15:59:36.565732+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:59:52.698 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:52.698 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:53.251 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:53.253 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:53.271 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-10T15:59:52.573897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":326377998,"num_objects":79686,"num_object_clones":0,"num_object_copies":159372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213368,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13047,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160648,"ondisk_log_size":160648,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1918072,"kb_used_data":826360,"kb_used_omap":12,"kb_used_meta":1091635,"kb_avail":281197448,"statfs":{"total":289910292480,"available":287946186752,"internally_reserved":0,"allocated":846192640,"data_stored":845297194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1117834245},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,3,2,2,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23801856,"num_objects":-5811,"num_object_clones":0,"num_object_copies":-11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5811,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006866"},"pg_stats":[{"pgid":"2.f","version":"48'15808","reported_seq":16612,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.433239+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:50.433239+0000","last_peered":"2024-09-10T15:59:50.433239+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:50.433239+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20324352,"num_objects":4962,"num_object_clones":0,"num_object_copies":9924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15808,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16046","reported_seq":16158,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:50.582286+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:50.582286+0000","last_peered":"2024-09-10T15:59:50.582286+0000","last_clean":"2024-09-10T15:59:50.582286+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:50.582286+0000","last_undegraded":"2024-09-10T15:59:50.582286+0000","last_fullsized":"2024-09-10T15:59:50.582286+0000","mapping_epoch":36,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20594688,"num_objects":5028,"num_object_clones":0,"num_object_copies":10056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14382,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15610","reported_seq":16085,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.535365+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:50.535365+0000","last_peered":"2024-09-10T15:59:50.535365+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:50.535365+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20557848,"num_objects":5020,"num_object_clones":0,"num_object_copies":10040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13878,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1391}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15657","reported_seq":15742,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:50.495150+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:50.495150+0000","last_peered":"2024-09-10T15:59:50.495150+0000","last_clean":"2024-09-10T15:59:50.495150+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:50.495150+0000","last_undegraded":"2024-09-10T15:59:50.495150+0000","last_fullsized":"2024-09-10T15:59:50.495150+0000","mapping_epoch":36,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20582400,"num_objects":5025,"num_object_clones":0,"num_object_copies":10050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15657,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15462","reported_seq":15594,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:50.370505+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:50.370505+0000","last_peered":"2024-09-10T15:59:50.370505+0000","last_clean":"2024-09-10T15:59:50.370505+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:50.370505+0000","last_undegraded":"2024-09-10T15:59:50.370505+0000","last_fullsized":"2024-09-10T15:59:50.370505+0000","mapping_epoch":35,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19943424,"num_objects":4869,"num_object_clones":0,"num_object_copies":9738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12533,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15817","reported_seq":23710,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:50.751291+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:50.751291+0000","last_peered":"2024-09-10T15:59:50.751291+0000","last_clean":"2024-09-10T15:59:50.751291+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:50.751291+0000","last_undegraded":"2024-09-10T15:59:50.751291+0000","last_fullsized":"2024-09-10T15:59:50.751291+0000","mapping_epoch":47,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20684800,"num_objects":5050,"num_object_clones":0,"num_object_copies":10100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12535,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15432","reported_seq":16198,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.230242+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:50.230242+0000","last_peered":"2024-09-10T15:59:50.230242+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:50.230242+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19881984,"num_objects":4854,"num_object_clones":0,"num_object_copies":9708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12082,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1330}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'15890","reported_seq":16027,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:50.408985+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:50.408985+0000","last_peered":"2024-09-10T15:59:50.408985+0000","last_clean":"2024-09-10T15:59:50.408985+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:50.408985+0000","last_undegraded":"2024-09-10T15:59:50.408985+0000","last_fullsized":"2024-09-10T15:59:50.408985+0000","mapping_epoch":35,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":5024,"num_object_clones":0,"num_object_copies":10048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12649,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'15867","reported_seq":24000,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:50.789370+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:50.789370+0000","last_peered":"2024-09-10T15:59:50.789370+0000","last_clean":"2024-09-10T15:59:50.789370+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:50.789370+0000","last_undegraded":"2024-09-10T15:59:50.789370+0000","last_fullsized":"2024-09-10T15:59:50.789370+0000","mapping_epoch":45,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19873792,"num_objects":4852,"num_object_clones":0,"num_object_copies":9704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13844,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15626","reported_seq":16445,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.344089+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:50.344089+0000","last_peered":"2024-09-10T15:59:50.344089+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:50.344089+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12490,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1433}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15749","reported_seq":20982,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.588204+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:50.588204+0000","last_peered":"2024-09-10T15:59:50.588204+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:50.588204+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20627456,"num_objects":5036,"num_object_clones":0,"num_object_copies":10072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13723,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2551,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3753}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15506","reported_seq":15923,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.577348+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:50.577348+0000","last_peered":"2024-09-10T15:59:50.577348+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:50.577348+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20275200,"num_objects":4950,"num_object_clones":0,"num_object_copies":9900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13375,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1264}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15828","reported_seq":16671,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.385900+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:50.385900+0000","last_peered":"2024-09-10T15:59:50.385900+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:50.385900+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20586496,"num_objects":5026,"num_object_clones":0,"num_object_copies":10052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13020,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1548}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15783","reported_seq":16534,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.379310+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:50.379310+0000","last_peered":"2024-09-10T15:59:50.379310+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:50.379310+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20213760,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12201,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15727","reported_seq":16554,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.372619+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:50.372619+0000","last_peered":"2024-09-10T15:59:50.372619+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:50.372619+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20639744,"num_objects":5039,"num_object_clones":0,"num_object_copies":10078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12333,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16037","reported_seq":16797,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:50.371113+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:50.371113+0000","last_peered":"2024-09-10T15:59:50.371113+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:50.371113+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20730339,"num_objects":5062,"num_object_clones":0,"num_object_copies":10124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12856,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1368}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":326377979,"num_objects":79684,"num_object_clones":0,"num_object_copies":159368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33603,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79684,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213366,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13045,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":843366400,"data_stored":843289609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160645,"ondisk_log_size":160645,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626515,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592512,"kb_used_data":240172,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93779328,"statfs":{"total":96636764160,"available":96030031872,"internally_reserved":0,"allocated":245936128,"data_stored":245597750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":360771246},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149796,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712352,"kb_used_data":301704,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93659488,"statfs":{"total":96636764160,"available":95907315712,"internally_reserved":0,"allocated":308944896,"data_stored":308668398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":420474541},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613208,"kb_used_data":284484,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93758632,"statfs":{"total":96636764160,"available":96008839168,"internally_reserved":0,"allocated":291311616,"data_stored":291031046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":308363264,"data_stored":308359651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244322304,"data_stored":244256811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290680832,"data_stored":290673147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:56.273 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:56.881 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:56.883 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:56.903 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-10T15:59:56.575039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":316617230,"num_objects":77303,"num_object_clones":0,"num_object_copies":154606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77303,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215751,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13194,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1894972,"kb_used_data":807420,"kb_used_omap":12,"kb_used_meta":1087475,"kb_avail":281220548,"statfs":{"total":289910292480,"available":287969841152,"internally_reserved":0,"allocated":826798080,"data_stored":825902634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1113574405},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,3,7,7,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29282304,"num_objects":-7149,"num_object_clones":0,"num_object_copies":-14298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7149,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006522"},"pg_stats":[{"pgid":"2.f","version":"48'15947","reported_seq":16751,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.457550+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:55.457550+0000","last_peered":"2024-09-10T15:59:55.457550+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:55.457550+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19755008,"num_objects":4823,"num_object_clones":0,"num_object_copies":9646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15947,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16208","reported_seq":16321,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.535365+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:55.535365+0000","last_peered":"2024-09-10T15:59:55.535365+0000","last_clean":"2024-09-10T15:59:55.535365+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:55.535365+0000","last_undegraded":"2024-09-10T15:59:55.535365+0000","last_fullsized":"2024-09-10T15:59:55.535365+0000","mapping_epoch":36,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19931136,"num_objects":4866,"num_object_clones":0,"num_object_copies":9732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14544,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15745","reported_seq":16221,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.551263+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:55.551263+0000","last_peered":"2024-09-10T15:59:55.551263+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:55.551263+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20004888,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14013,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1256}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15815","reported_seq":15900,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.572772+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:55.572772+0000","last_peered":"2024-09-10T15:59:55.572772+0000","last_clean":"2024-09-10T15:59:55.572772+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:55.572772+0000","last_undegraded":"2024-09-10T15:59:55.572772+0000","last_fullsized":"2024-09-10T15:59:55.572772+0000","mapping_epoch":36,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935232,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15815,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15604","reported_seq":15736,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.387794+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:55.387794+0000","last_peered":"2024-09-10T15:59:55.387794+0000","last_clean":"2024-09-10T15:59:55.387794+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:55.387794+0000","last_undegraded":"2024-09-10T15:59:55.387794+0000","last_fullsized":"2024-09-10T15:59:55.387794+0000","mapping_epoch":35,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19361792,"num_objects":4727,"num_object_clones":0,"num_object_copies":9454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12675,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15978","reported_seq":23871,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.839213+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:55.839213+0000","last_peered":"2024-09-10T15:59:55.839213+0000","last_clean":"2024-09-10T15:59:55.839213+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:55.839213+0000","last_undegraded":"2024-09-10T15:59:55.839213+0000","last_fullsized":"2024-09-10T15:59:55.839213+0000","mapping_epoch":47,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12696,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15566","reported_seq":16330,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.411208+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:55.411208+0000","last_peered":"2024-09-10T15:59:55.411208+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:55.411208+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19333120,"num_objects":4720,"num_object_clones":0,"num_object_copies":9440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12216,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16024","reported_seq":16161,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.407397+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:55.407397+0000","last_peered":"2024-09-10T15:59:55.407397+0000","last_clean":"2024-09-10T15:59:55.407397+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:55.407397+0000","last_undegraded":"2024-09-10T15:59:55.407397+0000","last_fullsized":"2024-09-10T15:59:55.407397+0000","mapping_epoch":35,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20029440,"num_objects":4890,"num_object_clones":0,"num_object_copies":9780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12783,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16008","reported_seq":24141,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.765420+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:55.765420+0000","last_peered":"2024-09-10T15:59:55.765420+0000","last_clean":"2024-09-10T15:59:55.765420+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:55.765420+0000","last_undegraded":"2024-09-10T15:59:55.765420+0000","last_fullsized":"2024-09-10T15:59:55.765420+0000","mapping_epoch":45,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19296256,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13985,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15768","reported_seq":16588,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.411182+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:55.411182+0000","last_peered":"2024-09-10T15:59:55.411182+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:55.411182+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19701760,"num_objects":4810,"num_object_clones":0,"num_object_copies":9620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12632,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1291}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15894","reported_seq":21421,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.563734+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:55.563734+0000","last_peered":"2024-09-10T15:59:55.563734+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:55.563734+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20033536,"num_objects":4891,"num_object_clones":0,"num_object_copies":9782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13868,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2698,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3755}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15656","reported_seq":16074,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.568013+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:55.568013+0000","last_peered":"2024-09-10T15:59:55.568013+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:55.568013+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19660800,"num_objects":4800,"num_object_clones":0,"num_object_copies":9600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13525,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1114}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15990","reported_seq":16833,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.445329+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:55.445329+0000","last_peered":"2024-09-10T15:59:55.445329+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:55.445329+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19922944,"num_objects":4864,"num_object_clones":0,"num_object_copies":9728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13182,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15955","reported_seq":16706,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.432920+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:55.432920+0000","last_peered":"2024-09-10T15:59:55.432920+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:55.432920+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19509248,"num_objects":4763,"num_object_clones":0,"num_object_copies":9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12373,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15878","reported_seq":16705,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.464691+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:55.464691+0000","last_peered":"2024-09-10T15:59:55.464691+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:55.464691+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021248,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12484,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1352}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16192","reported_seq":16954,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.423794+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:55.423794+0000","last_peered":"2024-09-10T15:59:55.423794+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:55.423794+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20095459,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13011,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":316617211,"num_objects":77301,"num_object_clones":0,"num_object_copies":154602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215749,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13192,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":823824384,"data_stored":823747593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589556,"kb_used_data":233056,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93782284,"statfs":{"total":96636764160,"available":96033058816,"internally_reserved":0,"allocated":238649344,"data_stored":238310966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365031086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149797,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709352,"kb_used_data":294544,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93662488,"statfs":{"total":96636764160,"available":95910387712,"internally_reserved":0,"allocated":301613056,"data_stored":301336558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":424734381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596064,"kb_used_data":279820,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93775776,"statfs":{"total":96636764160,"available":96026394624,"internally_reserved":0,"allocated":286535680,"data_stored":286255110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":300949504,"data_stored":300945891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":236965888,"data_stored":236900395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":285908992,"data_stored":285901307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:56.905 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:57.462 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:57.465 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:57.482 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-10T15:59:56.575039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":316617230,"num_objects":77303,"num_object_clones":0,"num_object_copies":154606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77303,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215751,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13194,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1894972,"kb_used_data":807420,"kb_used_omap":12,"kb_used_meta":1087475,"kb_avail":281220548,"statfs":{"total":289910292480,"available":287969841152,"internally_reserved":0,"allocated":826798080,"data_stored":825902634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1113574405},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,3,7,7,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29282304,"num_objects":-7149,"num_object_clones":0,"num_object_copies":-14298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7149,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006522"},"pg_stats":[{"pgid":"2.f","version":"48'15947","reported_seq":16751,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.457550+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:55.457550+0000","last_peered":"2024-09-10T15:59:55.457550+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:55.457550+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19755008,"num_objects":4823,"num_object_clones":0,"num_object_copies":9646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15947,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16208","reported_seq":16321,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.535365+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:55.535365+0000","last_peered":"2024-09-10T15:59:55.535365+0000","last_clean":"2024-09-10T15:59:55.535365+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:55.535365+0000","last_undegraded":"2024-09-10T15:59:55.535365+0000","last_fullsized":"2024-09-10T15:59:55.535365+0000","mapping_epoch":36,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19931136,"num_objects":4866,"num_object_clones":0,"num_object_copies":9732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14544,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15745","reported_seq":16221,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.551263+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:55.551263+0000","last_peered":"2024-09-10T15:59:55.551263+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:55.551263+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20004888,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14013,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1256}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15815","reported_seq":15900,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.572772+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:55.572772+0000","last_peered":"2024-09-10T15:59:55.572772+0000","last_clean":"2024-09-10T15:59:55.572772+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:55.572772+0000","last_undegraded":"2024-09-10T15:59:55.572772+0000","last_fullsized":"2024-09-10T15:59:55.572772+0000","mapping_epoch":36,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935232,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15815,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15604","reported_seq":15736,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.387794+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:55.387794+0000","last_peered":"2024-09-10T15:59:55.387794+0000","last_clean":"2024-09-10T15:59:55.387794+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:55.387794+0000","last_undegraded":"2024-09-10T15:59:55.387794+0000","last_fullsized":"2024-09-10T15:59:55.387794+0000","mapping_epoch":35,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19361792,"num_objects":4727,"num_object_clones":0,"num_object_copies":9454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12675,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15978","reported_seq":23871,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.839213+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:55.839213+0000","last_peered":"2024-09-10T15:59:55.839213+0000","last_clean":"2024-09-10T15:59:55.839213+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:55.839213+0000","last_undegraded":"2024-09-10T15:59:55.839213+0000","last_fullsized":"2024-09-10T15:59:55.839213+0000","mapping_epoch":47,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12696,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15566","reported_seq":16330,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.411208+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:55.411208+0000","last_peered":"2024-09-10T15:59:55.411208+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:55.411208+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19333120,"num_objects":4720,"num_object_clones":0,"num_object_copies":9440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12216,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16024","reported_seq":16161,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.407397+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:55.407397+0000","last_peered":"2024-09-10T15:59:55.407397+0000","last_clean":"2024-09-10T15:59:55.407397+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:55.407397+0000","last_undegraded":"2024-09-10T15:59:55.407397+0000","last_fullsized":"2024-09-10T15:59:55.407397+0000","mapping_epoch":35,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20029440,"num_objects":4890,"num_object_clones":0,"num_object_copies":9780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12783,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16008","reported_seq":24141,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.765420+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:55.765420+0000","last_peered":"2024-09-10T15:59:55.765420+0000","last_clean":"2024-09-10T15:59:55.765420+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:55.765420+0000","last_undegraded":"2024-09-10T15:59:55.765420+0000","last_fullsized":"2024-09-10T15:59:55.765420+0000","mapping_epoch":45,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19296256,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13985,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15768","reported_seq":16588,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.411182+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:55.411182+0000","last_peered":"2024-09-10T15:59:55.411182+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:55.411182+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19701760,"num_objects":4810,"num_object_clones":0,"num_object_copies":9620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12632,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1291}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15894","reported_seq":21421,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.563734+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:55.563734+0000","last_peered":"2024-09-10T15:59:55.563734+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:55.563734+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20033536,"num_objects":4891,"num_object_clones":0,"num_object_copies":9782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13868,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2698,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3755}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15656","reported_seq":16074,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.568013+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:55.568013+0000","last_peered":"2024-09-10T15:59:55.568013+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:55.568013+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19660800,"num_objects":4800,"num_object_clones":0,"num_object_copies":9600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13525,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1114}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15990","reported_seq":16833,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.445329+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:55.445329+0000","last_peered":"2024-09-10T15:59:55.445329+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:55.445329+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19922944,"num_objects":4864,"num_object_clones":0,"num_object_copies":9728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13182,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15955","reported_seq":16706,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.432920+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:55.432920+0000","last_peered":"2024-09-10T15:59:55.432920+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:55.432920+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19509248,"num_objects":4763,"num_object_clones":0,"num_object_copies":9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12373,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15878","reported_seq":16705,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.464691+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:55.464691+0000","last_peered":"2024-09-10T15:59:55.464691+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:55.464691+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021248,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12484,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1352}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16192","reported_seq":16954,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.423794+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:55.423794+0000","last_peered":"2024-09-10T15:59:55.423794+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:55.423794+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20095459,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13011,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":316617211,"num_objects":77301,"num_object_clones":0,"num_object_copies":154602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215749,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13192,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":823824384,"data_stored":823747593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589556,"kb_used_data":233056,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93782284,"statfs":{"total":96636764160,"available":96033058816,"internally_reserved":0,"allocated":238649344,"data_stored":238310966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365031086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149797,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709352,"kb_used_data":294544,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93662488,"statfs":{"total":96636764160,"available":95910387712,"internally_reserved":0,"allocated":301613056,"data_stored":301336558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":424734381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596064,"kb_used_data":279820,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93775776,"statfs":{"total":96636764160,"available":96026394624,"internally_reserved":0,"allocated":286535680,"data_stored":286255110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":300949504,"data_stored":300945891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":236965888,"data_stored":236900395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":285908992,"data_stored":285901307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T15:59:57.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T15:59:57.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'15947", 'reported_seq': 16751, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:55.457550+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T15:59:55.457550+0000', 'last_peered': '2024-09-10T15:59:55.457550+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T15:59:55.457550+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'5900", 'ondisk_log_start': "44'5900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19755008, 'num_objects': 4823, 'num_object_clones': 0, 'num_object_copies': 9646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4823, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15947, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1211}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:57.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T15:59:57.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'15745", 'reported_seq': 16221, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:55.551263+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T15:59:55.551263+0000', 'last_peered': '2024-09-10T15:59:55.551263+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T15:59:55.551263+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'5700", 'ondisk_log_start': "44'5700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20004888, 'num_objects': 4885, 'num_object_clones': 0, 'num_object_copies': 9770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4885, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14013, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1256}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:57.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T15:59:57.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'15566", 'reported_seq': 16330, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:55.411208+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T15:59:55.411208+0000', 'last_peered': '2024-09-10T15:59:55.411208+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T15:59:55.411208+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'5500", 'ondisk_log_start': "44'5500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19333120, 'num_objects': 4720, 'num_object_clones': 0, 'num_object_copies': 9440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4720, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12216, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1196}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:57.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T15:59:57.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'15768", 'reported_seq': 16588, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:55.411182+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T15:59:55.411182+0000', 'last_peered': '2024-09-10T15:59:55.411182+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T15:59:55.411182+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'5700", 'ondisk_log_start': "44'5700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19701760, 'num_objects': 4810, 'num_object_clones': 0, 'num_object_copies': 9620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4810, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12632, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1291}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:57.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T15:59:57.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'15894", 'reported_seq': 21421, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:55.563734+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T15:59:55.563734+0000', 'last_peered': '2024-09-10T15:59:55.563734+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T15:59:55.563734+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'5800", 'ondisk_log_start': "44'5800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20033536, 'num_objects': 4891, 'num_object_clones': 0, 'num_object_copies': 9782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1136, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4891, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13868, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2698, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3755}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:57.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T15:59:57.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'15656", 'reported_seq': 16074, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:55.568013+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T15:59:55.568013+0000', 'last_peered': '2024-09-10T15:59:55.568013+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T15:59:55.568013+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'5600", 'ondisk_log_start': "44'5600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19660800, 'num_objects': 4800, 'num_object_clones': 0, 'num_object_copies': 9600, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4800, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13525, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1114}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T15:59:57.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T15:59:57.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'15990", 'reported_seq': 16833, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:55.445329+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T15:59:55.445329+0000', 'last_peered': '2024-09-10T15:59:55.445329+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T15:59:55.445329+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'5900", 'ondisk_log_start': "44'5900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19922944, 'num_objects': 4864, 'num_object_clones': 0, 'num_object_copies': 9728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4864, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13182, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1386}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:57.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T15:59:57.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'15955", 'reported_seq': 16706, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:55.432920+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T15:59:55.432920+0000', 'last_peered': '2024-09-10T15:59:55.432920+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T15:59:55.432920+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'5900", 'ondisk_log_start': "44'5900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19509248, 'num_objects': 4763, 'num_object_clones': 0, 'num_object_copies': 9526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4763, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12373, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1137}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:57.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T15:59:57.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'15878", 'reported_seq': 16705, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:55.464691+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T15:59:55.464691+0000', 'last_peered': '2024-09-10T15:59:55.464691+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T15:59:55.464691+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'5800", 'ondisk_log_start': "44'5800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20021248, 'num_objects': 4888, 'num_object_clones': 0, 'num_object_copies': 9776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12484, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1352}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:57.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T15:59:57.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'16192", 'reported_seq': 16954, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T15:59:55.423794+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T15:59:55.423794+0000', 'last_peered': '2024-09-10T15:59:55.423794+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T15:59:55.423794+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20095459, 'num_objects': 4907, 'num_object_clones': 0, 'num_object_copies': 9814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4907, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13011, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1213}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T15:59:57.485 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T15:59:58.162 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:58.163 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33456/154606 objects degraded (21.640%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":524,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":77303,"data_bytes":316617230,"bytes_used":1940451328,"bytes_avail":287969841152,"bytes_total":289910292480,"degraded_objects":33456,"degraded_total":154606,"degraded_ratio":0.21639522398871971,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":414,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":595},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T15:59:36.565732+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T15:59:58.191 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T15:59:58.192 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T15:59:58.749 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T15:59:58.751 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T15:59:58.771 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-10T15:59:58.575652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":316617230,"num_objects":77303,"num_object_clones":0,"num_object_copies":154606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77303,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215751,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13194,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1894972,"kb_used_data":807420,"kb_used_omap":12,"kb_used_meta":1087475,"kb_avail":281220548,"statfs":{"total":289910292480,"available":287969841152,"internally_reserved":0,"allocated":826798080,"data_stored":825902634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1113574405},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,3,7,7,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19468288,"num_objects":-4753,"num_object_clones":0,"num_object_copies":-9506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4753,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006500"},"pg_stats":[{"pgid":"2.f","version":"48'15947","reported_seq":16751,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.457550+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T15:59:55.457550+0000","last_peered":"2024-09-10T15:59:55.457550+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T15:59:55.457550+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19755008,"num_objects":4823,"num_object_clones":0,"num_object_copies":9646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15947,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16208","reported_seq":16321,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.535365+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:55.535365+0000","last_peered":"2024-09-10T15:59:55.535365+0000","last_clean":"2024-09-10T15:59:55.535365+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:55.535365+0000","last_undegraded":"2024-09-10T15:59:55.535365+0000","last_fullsized":"2024-09-10T15:59:55.535365+0000","mapping_epoch":36,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19931136,"num_objects":4866,"num_object_clones":0,"num_object_copies":9732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14544,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15745","reported_seq":16221,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.551263+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:55.551263+0000","last_peered":"2024-09-10T15:59:55.551263+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:55.551263+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20004888,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14013,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1256}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15815","reported_seq":15900,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.572772+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:55.572772+0000","last_peered":"2024-09-10T15:59:55.572772+0000","last_clean":"2024-09-10T15:59:55.572772+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:55.572772+0000","last_undegraded":"2024-09-10T15:59:55.572772+0000","last_fullsized":"2024-09-10T15:59:55.572772+0000","mapping_epoch":36,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935232,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15815,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15604","reported_seq":15736,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.387794+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T15:59:55.387794+0000","last_peered":"2024-09-10T15:59:55.387794+0000","last_clean":"2024-09-10T15:59:55.387794+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T15:59:55.387794+0000","last_undegraded":"2024-09-10T15:59:55.387794+0000","last_fullsized":"2024-09-10T15:59:55.387794+0000","mapping_epoch":35,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19361792,"num_objects":4727,"num_object_clones":0,"num_object_copies":9454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12675,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15978","reported_seq":23871,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.839213+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:55.839213+0000","last_peered":"2024-09-10T15:59:55.839213+0000","last_clean":"2024-09-10T15:59:55.839213+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:55.839213+0000","last_undegraded":"2024-09-10T15:59:55.839213+0000","last_fullsized":"2024-09-10T15:59:55.839213+0000","mapping_epoch":47,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12696,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15566","reported_seq":16330,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.411208+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T15:59:55.411208+0000","last_peered":"2024-09-10T15:59:55.411208+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T15:59:55.411208+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19333120,"num_objects":4720,"num_object_clones":0,"num_object_copies":9440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12216,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16024","reported_seq":16161,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.407397+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T15:59:55.407397+0000","last_peered":"2024-09-10T15:59:55.407397+0000","last_clean":"2024-09-10T15:59:55.407397+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T15:59:55.407397+0000","last_undegraded":"2024-09-10T15:59:55.407397+0000","last_fullsized":"2024-09-10T15:59:55.407397+0000","mapping_epoch":35,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20029440,"num_objects":4890,"num_object_clones":0,"num_object_copies":9780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12783,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16008","reported_seq":24141,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.765420+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:55.765420+0000","last_peered":"2024-09-10T15:59:55.765420+0000","last_clean":"2024-09-10T15:59:55.765420+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:55.765420+0000","last_undegraded":"2024-09-10T15:59:55.765420+0000","last_fullsized":"2024-09-10T15:59:55.765420+0000","mapping_epoch":45,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19296256,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13985,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15768","reported_seq":16588,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.411182+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T15:59:55.411182+0000","last_peered":"2024-09-10T15:59:55.411182+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T15:59:55.411182+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19701760,"num_objects":4810,"num_object_clones":0,"num_object_copies":9620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12632,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1291}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15894","reported_seq":21421,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.563734+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:55.563734+0000","last_peered":"2024-09-10T15:59:55.563734+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:55.563734+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20033536,"num_objects":4891,"num_object_clones":0,"num_object_copies":9782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13868,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2698,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3755}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15656","reported_seq":16074,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.568013+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:55.568013+0000","last_peered":"2024-09-10T15:59:55.568013+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:55.568013+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19660800,"num_objects":4800,"num_object_clones":0,"num_object_copies":9600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13525,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1114}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'15990","reported_seq":16833,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.445329+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T15:59:55.445329+0000","last_peered":"2024-09-10T15:59:55.445329+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T15:59:55.445329+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19922944,"num_objects":4864,"num_object_clones":0,"num_object_copies":9728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13182,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'15955","reported_seq":16706,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.432920+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T15:59:55.432920+0000","last_peered":"2024-09-10T15:59:55.432920+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T15:59:55.432920+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19509248,"num_objects":4763,"num_object_clones":0,"num_object_copies":9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12373,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15878","reported_seq":16705,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.464691+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T15:59:55.464691+0000","last_peered":"2024-09-10T15:59:55.464691+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T15:59:55.464691+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021248,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12484,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1352}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16192","reported_seq":16954,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.423794+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T15:59:55.423794+0000","last_peered":"2024-09-10T15:59:55.423794+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T15:59:55.423794+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20095459,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13011,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":316617211,"num_objects":77301,"num_object_clones":0,"num_object_copies":154602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215749,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13192,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":823824384,"data_stored":823747593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589556,"kb_used_data":233056,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93782284,"statfs":{"total":96636764160,"available":96033058816,"internally_reserved":0,"allocated":238649344,"data_stored":238310966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365031086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149797,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709352,"kb_used_data":294544,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93662488,"statfs":{"total":96636764160,"available":95910387712,"internally_reserved":0,"allocated":301613056,"data_stored":301336558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":424734381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596064,"kb_used_data":279820,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93775776,"statfs":{"total":96636764160,"available":96026394624,"internally_reserved":0,"allocated":286535680,"data_stored":286255110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":300949504,"data_stored":300945891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":236965888,"data_stored":236900395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":285908992,"data_stored":285901307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:01.775 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:02.239 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:02.240 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:02.253 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-10T16:00:00.576435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":311104014,"num_objects":75957,"num_object_clones":0,"num_object_copies":151914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75957,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217097,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13194,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160677,"ondisk_log_size":160677,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1909964,"kb_used_data":800208,"kb_used_omap":12,"kb_used_meta":1109683,"kb_avail":281205556,"statfs":{"total":289910292480,"available":287954489344,"internally_reserved":0,"allocated":819412992,"data_stored":818517546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1136315397},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,7,6,2,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24981504,"num_objects":-6099,"num_object_clones":0,"num_object_copies":-12198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6099,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003465"},"pg_stats":[{"pgid":"2.f","version":"48'16116","reported_seq":16920,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.459996+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:00.459996+0000","last_peered":"2024-09-10T16:00:00.459996+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:00.459996+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19062784,"num_objects":4654,"num_object_clones":0,"num_object_copies":9308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16116,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1042}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16208","reported_seq":16321,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.535365+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T15:59:55.535365+0000","last_peered":"2024-09-10T15:59:55.535365+0000","last_clean":"2024-09-10T15:59:55.535365+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T15:59:55.535365+0000","last_undegraded":"2024-09-10T15:59:55.535365+0000","last_fullsized":"2024-09-10T15:59:55.535365+0000","mapping_epoch":36,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19931136,"num_objects":4866,"num_object_clones":0,"num_object_copies":9732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14544,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15745","reported_seq":16221,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.551263+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T15:59:55.551263+0000","last_peered":"2024-09-10T15:59:55.551263+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T15:59:55.551263+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20004888,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14013,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1256}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15815","reported_seq":15900,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.572772+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T15:59:55.572772+0000","last_peered":"2024-09-10T15:59:55.572772+0000","last_clean":"2024-09-10T15:59:55.572772+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T15:59:55.572772+0000","last_undegraded":"2024-09-10T15:59:55.572772+0000","last_fullsized":"2024-09-10T15:59:55.572772+0000","mapping_epoch":36,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935232,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15815,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15740","reported_seq":15872,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.447417+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:00.447417+0000","last_peered":"2024-09-10T16:00:00.447417+0000","last_clean":"2024-09-10T16:00:00.447417+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:00.447417+0000","last_undegraded":"2024-09-10T16:00:00.447417+0000","last_fullsized":"2024-09-10T16:00:00.447417+0000","mapping_epoch":35,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18804736,"num_objects":4591,"num_object_clones":0,"num_object_copies":9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12811,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'15978","reported_seq":23871,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.839213+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T15:59:55.839213+0000","last_peered":"2024-09-10T15:59:55.839213+0000","last_clean":"2024-09-10T15:59:55.839213+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T15:59:55.839213+0000","last_undegraded":"2024-09-10T15:59:55.839213+0000","last_fullsized":"2024-09-10T15:59:55.839213+0000","mapping_epoch":47,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12696,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15709","reported_seq":16475,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.435032+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:00.435032+0000","last_peered":"2024-09-10T16:00:00.435032+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:00.435032+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18747392,"num_objects":4577,"num_object_clones":0,"num_object_copies":9154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1053}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16177","reported_seq":16313,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.478237+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:00.478237+0000","last_peered":"2024-09-10T16:00:00.478237+0000","last_clean":"2024-09-10T16:00:00.478237+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:00.478237+0000","last_undegraded":"2024-09-10T16:00:00.478237+0000","last_fullsized":"2024-09-10T16:00:00.478237+0000","mapping_epoch":35,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19402752,"num_objects":4737,"num_object_clones":0,"num_object_copies":9474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12936,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16008","reported_seq":24141,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:59:55.765420+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T15:59:55.765420+0000","last_peered":"2024-09-10T15:59:55.765420+0000","last_clean":"2024-09-10T15:59:55.765420+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T15:59:55.765420+0000","last_undegraded":"2024-09-10T15:59:55.765420+0000","last_fullsized":"2024-09-10T15:59:55.765420+0000","mapping_epoch":45,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19296256,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13985,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15921","reported_seq":16740,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.476930+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:00.476930+0000","last_peered":"2024-09-10T16:00:00.476930+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:00.476930+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19075072,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12785,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1138}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'15894","reported_seq":21421,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.563734+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T15:59:55.563734+0000","last_peered":"2024-09-10T15:59:55.563734+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T15:59:55.563734+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20033536,"num_objects":4891,"num_object_clones":0,"num_object_copies":9782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13868,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2698,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3755}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15656","reported_seq":16074,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T15:59:55.568013+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T15:59:55.568013+0000","last_peered":"2024-09-10T15:59:55.568013+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T15:59:55.568013+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19660800,"num_objects":4800,"num_object_clones":0,"num_object_copies":9600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13525,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1114}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16139","reported_seq":16976,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.493263+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:00.493263+0000","last_peered":"2024-09-10T16:00:00.493263+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:00.493263+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19312640,"num_objects":4715,"num_object_clones":0,"num_object_copies":9430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13331,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16115","reported_seq":16866,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.489080+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:00.489080+0000","last_peered":"2024-09-10T16:00:00.489080+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:00.489080+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18853888,"num_objects":4603,"num_object_clones":0,"num_object_copies":9206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12533,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15999","reported_seq":16826,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.348419+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:00.348419+0000","last_peered":"2024-09-10T16:00:00.348419+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:00.348419+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12605,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16354","reported_seq":17116,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.480580+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:00.480580+0000","last_peered":"2024-09-10T16:00:00.480580+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:00.480580+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19431907,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13173,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":311103995,"num_objects":75955,"num_object_clones":0,"num_object_copies":151910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75955,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217095,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13192,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":816418816,"data_stored":816342025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160674,"ondisk_log_size":160674,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589556,"kb_used_data":233056,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93782284,"statfs":{"total":96636764160,"available":96033058816,"internally_reserved":0,"allocated":238649344,"data_stored":238310966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365031086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149798,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":724344,"kb_used_data":287332,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93647496,"statfs":{"total":96636764160,"available":95895035904,"internally_reserved":0,"allocated":294227968,"data_stored":293951470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":447475373},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,2,1,1,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596064,"kb_used_data":279820,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93775776,"statfs":{"total":96636764160,"available":96026394624,"internally_reserved":0,"allocated":286535680,"data_stored":286255110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293543936,"data_stored":293540323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":236965888,"data_stored":236900395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":285908992,"data_stored":285901307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:02.255 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:02.839 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:02.840 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:02.859 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-10T16:00:02.577272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":306627086,"num_objects":74864,"num_object_clones":0,"num_object_copies":149728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74864,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218190,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13311,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1910328,"kb_used_data":788096,"kb_used_omap":12,"kb_used_meta":1122163,"kb_avail":281205192,"statfs":{"total":289910292480,"available":287954116608,"internally_reserved":0,"allocated":807010304,"data_stored":806114858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1149094917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,4,6,8,3,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24100864,"num_objects":-5884,"num_object_clones":0,"num_object_copies":-11768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5884,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003845"},"pg_stats":[{"pgid":"2.f","version":"48'16116","reported_seq":16920,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.459996+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:00.459996+0000","last_peered":"2024-09-10T16:00:00.459996+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:00.459996+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19062784,"num_objects":4654,"num_object_clones":0,"num_object_copies":9308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16116,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1042}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16376","reported_seq":16489,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.542628+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:00.542628+0000","last_peered":"2024-09-10T16:00:00.542628+0000","last_clean":"2024-09-10T16:00:00.542628+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:00.542628+0000","last_undegraded":"2024-09-10T16:00:00.542628+0000","last_fullsized":"2024-09-10T16:00:00.542628+0000","mapping_epoch":36,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19243008,"num_objects":4698,"num_object_clones":0,"num_object_copies":9396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14712,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15920","reported_seq":16396,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.642167+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:00.642167+0000","last_peered":"2024-09-10T16:00:00.642167+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:00.642167+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19288088,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14188,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1081}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15956","reported_seq":16041,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.621122+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:00.621122+0000","last_peered":"2024-09-10T16:00:00.621122+0000","last_clean":"2024-09-10T16:00:00.621122+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:00.621122+0000","last_undegraded":"2024-09-10T16:00:00.621122+0000","last_fullsized":"2024-09-10T16:00:00.621122+0000","mapping_epoch":36,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19357696,"num_objects":4726,"num_object_clones":0,"num_object_copies":9452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15956,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15740","reported_seq":15872,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.447417+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:00.447417+0000","last_peered":"2024-09-10T16:00:00.447417+0000","last_clean":"2024-09-10T16:00:00.447417+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:00.447417+0000","last_undegraded":"2024-09-10T16:00:00.447417+0000","last_fullsized":"2024-09-10T16:00:00.447417+0000","mapping_epoch":35,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18804736,"num_objects":4591,"num_object_clones":0,"num_object_copies":9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12811,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16138","reported_seq":24031,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.914512+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:00.914512+0000","last_peered":"2024-09-10T16:00:00.914512+0000","last_clean":"2024-09-10T16:00:00.914512+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:00.914512+0000","last_undegraded":"2024-09-10T16:00:00.914512+0000","last_fullsized":"2024-09-10T16:00:00.914512+0000","mapping_epoch":47,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19369984,"num_objects":4729,"num_object_clones":0,"num_object_copies":9458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12856,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15709","reported_seq":16475,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.435032+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:00.435032+0000","last_peered":"2024-09-10T16:00:00.435032+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:00.435032+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18747392,"num_objects":4577,"num_object_clones":0,"num_object_copies":9154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1053}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16177","reported_seq":16313,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.478237+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:00.478237+0000","last_peered":"2024-09-10T16:00:00.478237+0000","last_clean":"2024-09-10T16:00:00.478237+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:00.478237+0000","last_undegraded":"2024-09-10T16:00:00.478237+0000","last_fullsized":"2024-09-10T16:00:00.478237+0000","mapping_epoch":35,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19402752,"num_objects":4737,"num_object_clones":0,"num_object_copies":9474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12936,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16145","reported_seq":24278,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.931518+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:00.931518+0000","last_peered":"2024-09-10T16:00:00.931518+0000","last_clean":"2024-09-10T16:00:00.931518+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:00.931518+0000","last_undegraded":"2024-09-10T16:00:00.931518+0000","last_fullsized":"2024-09-10T16:00:00.931518+0000","mapping_epoch":45,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18735104,"num_objects":4574,"num_object_clones":0,"num_object_copies":9148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14122,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15921","reported_seq":16740,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.476930+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:00.476930+0000","last_peered":"2024-09-10T16:00:00.476930+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:00.476930+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19075072,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12785,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1138}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16060","reported_seq":21820,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.655288+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:00.655288+0000","last_peered":"2024-09-10T16:00:00.655288+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:00.655288+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19353600,"num_objects":4725,"num_object_clones":0,"num_object_copies":9450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1019,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14034,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2815,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3706}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15802","reported_seq":16220,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.504189+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:00.504189+0000","last_peered":"2024-09-10T16:00:00.504189+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:00.504189+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19062784,"num_objects":4654,"num_object_clones":0,"num_object_copies":9308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13671,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16139","reported_seq":16976,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.493263+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:00.493263+0000","last_peered":"2024-09-10T16:00:00.493263+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:00.493263+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19312640,"num_objects":4715,"num_object_clones":0,"num_object_copies":9430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13331,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16115","reported_seq":16866,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.489080+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:00.489080+0000","last_peered":"2024-09-10T16:00:00.489080+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:00.489080+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18853888,"num_objects":4603,"num_object_clones":0,"num_object_copies":9206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12533,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15999","reported_seq":16826,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.348419+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:00.348419+0000","last_peered":"2024-09-10T16:00:00.348419+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:00.348419+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12605,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16354","reported_seq":17116,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.480580+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:00.480580+0000","last_peered":"2024-09-10T16:00:00.480580+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:00.480580+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19431907,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13173,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":306627067,"num_objects":74862,"num_object_clones":0,"num_object_copies":149724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218188,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13309,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":803811328,"data_stored":803734537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626517,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590560,"kb_used_data":225740,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93781280,"statfs":{"total":96636764160,"available":96032030720,"internally_reserved":0,"allocated":231157760,"data_stored":230819382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":373550766},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149798,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":724344,"kb_used_data":287332,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93647496,"statfs":{"total":96636764160,"available":95895035904,"internally_reserved":0,"allocated":294227968,"data_stored":293951470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":447475373},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,2,1,1,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595424,"kb_used_data":275024,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93776416,"statfs":{"total":96636764160,"available":96027049984,"internally_reserved":0,"allocated":281624576,"data_stored":281344006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293543936,"data_stored":293540323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":229388288,"data_stored":229322795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280879104,"data_stored":280871419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:02.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:02.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'16116", 'reported_seq': 16920, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:00.459996+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T16:00:00.459996+0000', 'last_peered': '2024-09-10T16:00:00.459996+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:00:00.459996+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19062784, 'num_objects': 4654, 'num_object_clones': 0, 'num_object_copies': 9308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4654, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16116, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1042}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:02.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:02.860 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'15920", 'reported_seq': 16396, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:00.642167+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:00:00.642167+0000', 'last_peered': '2024-09-10T16:00:00.642167+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:00:00.642167+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'5900", 'ondisk_log_start': "44'5900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19288088, 'num_objects': 4710, 'num_object_clones': 0, 'num_object_copies': 9420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4710, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14188, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1081}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:02.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:02.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'15709", 'reported_seq': 16475, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:00.435032+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:00:00.435032+0000', 'last_peered': '2024-09-10T16:00:00.435032+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:00:00.435032+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'5700", 'ondisk_log_start': "44'5700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18747392, 'num_objects': 4577, 'num_object_clones': 0, 'num_object_copies': 9154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4577, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12359, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1053}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:02.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:02.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'15921", 'reported_seq': 16740, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:00.476930+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:00:00.476930+0000', 'last_peered': '2024-09-10T16:00:00.476930+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:00:00.476930+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'5900", 'ondisk_log_start': "44'5900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19075072, 'num_objects': 4657, 'num_object_clones': 0, 'num_object_copies': 9314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12785, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1138}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:02.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:02.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'16060", 'reported_seq': 21820, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:00.655288+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T16:00:00.655288+0000', 'last_peered': '2024-09-10T16:00:00.655288+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T16:00:00.655288+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'6000", 'ondisk_log_start': "44'6000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19353600, 'num_objects': 4725, 'num_object_clones': 0, 'num_object_copies': 9450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1019, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4725, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14034, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2815, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3706}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:02.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:02.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'15802", 'reported_seq': 16220, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:00.504189+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:00:00.504189+0000', 'last_peered': '2024-09-10T16:00:00.504189+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:00:00.504189+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'5800", 'ondisk_log_start': "44'5800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19062784, 'num_objects': 4654, 'num_object_clones': 0, 'num_object_copies': 9308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4654, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13671, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 968}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:02.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:02.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'16139", 'reported_seq': 16976, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:00.493263+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:00:00.493263+0000', 'last_peered': '2024-09-10T16:00:00.493263+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:00:00.493263+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19312640, 'num_objects': 4715, 'num_object_clones': 0, 'num_object_copies': 9430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4715, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13331, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:02.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:02.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'16115", 'reported_seq': 16866, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:00.489080+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:00:00.489080+0000', 'last_peered': '2024-09-10T16:00:00.489080+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:00:00.489080+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18853888, 'num_objects': 4603, 'num_object_clones': 0, 'num_object_copies': 9206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4603, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12533, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 977}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:02.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:02.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'15999", 'reported_seq': 16826, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:00.348419+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:00:00.348419+0000', 'last_peered': '2024-09-10T16:00:00.348419+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:00:00.348419+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'5900", 'ondisk_log_start': "44'5900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19525632, 'num_objects': 4767, 'num_object_clones': 0, 'num_object_copies': 9534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4767, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12605, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1231}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:02.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:02.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'16354", 'reported_seq': 17116, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:00.480580+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:00:00.480580+0000', 'last_peered': '2024-09-10T16:00:00.480580+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:00:00.480580+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'6300", 'ondisk_log_start': "44'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19431907, 'num_objects': 4745, 'num_object_clones': 0, 'num_object_copies': 9490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4745, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13173, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1051}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:02.863 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:03.563 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:03.563 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33339/149728 objects degraded (22.266%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":529,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":74864,"data_bytes":306627086,"bytes_used":1956175872,"bytes_avail":287954116608,"bytes_total":289910292480,"degraded_objects":33339,"degraded_total":149728,"degraded_ratio":0.22266376362470613,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":398,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":490},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T15:59:36.565732+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:00:03.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:03.585 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:04.150 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:04.153 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:04.172 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-10T16:00:02.577272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":306627086,"num_objects":74864,"num_object_clones":0,"num_object_copies":149728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74864,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218190,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13311,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1910328,"kb_used_data":788096,"kb_used_omap":12,"kb_used_meta":1122163,"kb_avail":281205192,"statfs":{"total":289910292480,"available":287954116608,"internally_reserved":0,"allocated":807010304,"data_stored":806114858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1149094917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,4,6,8,3,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24100864,"num_objects":-5884,"num_object_clones":0,"num_object_copies":-11768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5884,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003845"},"pg_stats":[{"pgid":"2.f","version":"48'16116","reported_seq":16920,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.459996+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:00.459996+0000","last_peered":"2024-09-10T16:00:00.459996+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:00.459996+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19062784,"num_objects":4654,"num_object_clones":0,"num_object_copies":9308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16116,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1042}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16376","reported_seq":16489,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.542628+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:00.542628+0000","last_peered":"2024-09-10T16:00:00.542628+0000","last_clean":"2024-09-10T16:00:00.542628+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:00.542628+0000","last_undegraded":"2024-09-10T16:00:00.542628+0000","last_fullsized":"2024-09-10T16:00:00.542628+0000","mapping_epoch":36,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19243008,"num_objects":4698,"num_object_clones":0,"num_object_copies":9396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14712,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'15920","reported_seq":16396,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.642167+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:00.642167+0000","last_peered":"2024-09-10T16:00:00.642167+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:00.642167+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19288088,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14188,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1081}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'15956","reported_seq":16041,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.621122+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:00.621122+0000","last_peered":"2024-09-10T16:00:00.621122+0000","last_clean":"2024-09-10T16:00:00.621122+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:00.621122+0000","last_undegraded":"2024-09-10T16:00:00.621122+0000","last_fullsized":"2024-09-10T16:00:00.621122+0000","mapping_epoch":36,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19357696,"num_objects":4726,"num_object_clones":0,"num_object_copies":9452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15956,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15740","reported_seq":15872,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.447417+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:00.447417+0000","last_peered":"2024-09-10T16:00:00.447417+0000","last_clean":"2024-09-10T16:00:00.447417+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:00.447417+0000","last_undegraded":"2024-09-10T16:00:00.447417+0000","last_fullsized":"2024-09-10T16:00:00.447417+0000","mapping_epoch":35,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18804736,"num_objects":4591,"num_object_clones":0,"num_object_copies":9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12811,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16138","reported_seq":24031,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.914512+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:00.914512+0000","last_peered":"2024-09-10T16:00:00.914512+0000","last_clean":"2024-09-10T16:00:00.914512+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:00.914512+0000","last_undegraded":"2024-09-10T16:00:00.914512+0000","last_fullsized":"2024-09-10T16:00:00.914512+0000","mapping_epoch":47,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19369984,"num_objects":4729,"num_object_clones":0,"num_object_copies":9458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12856,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15709","reported_seq":16475,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.435032+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:00.435032+0000","last_peered":"2024-09-10T16:00:00.435032+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:00.435032+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18747392,"num_objects":4577,"num_object_clones":0,"num_object_copies":9154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1053}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16177","reported_seq":16313,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.478237+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:00.478237+0000","last_peered":"2024-09-10T16:00:00.478237+0000","last_clean":"2024-09-10T16:00:00.478237+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:00.478237+0000","last_undegraded":"2024-09-10T16:00:00.478237+0000","last_fullsized":"2024-09-10T16:00:00.478237+0000","mapping_epoch":35,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19402752,"num_objects":4737,"num_object_clones":0,"num_object_copies":9474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12936,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16145","reported_seq":24278,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:00.931518+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:00.931518+0000","last_peered":"2024-09-10T16:00:00.931518+0000","last_clean":"2024-09-10T16:00:00.931518+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:00.931518+0000","last_undegraded":"2024-09-10T16:00:00.931518+0000","last_fullsized":"2024-09-10T16:00:00.931518+0000","mapping_epoch":45,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18735104,"num_objects":4574,"num_object_clones":0,"num_object_copies":9148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14122,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'15921","reported_seq":16740,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.476930+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:00.476930+0000","last_peered":"2024-09-10T16:00:00.476930+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:00.476930+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19075072,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12785,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1138}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16060","reported_seq":21820,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.655288+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:00.655288+0000","last_peered":"2024-09-10T16:00:00.655288+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:00.655288+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19353600,"num_objects":4725,"num_object_clones":0,"num_object_copies":9450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1019,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14034,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2815,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3706}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15802","reported_seq":16220,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.504189+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:00.504189+0000","last_peered":"2024-09-10T16:00:00.504189+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:00.504189+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19062784,"num_objects":4654,"num_object_clones":0,"num_object_copies":9308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13671,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16139","reported_seq":16976,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.493263+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:00.493263+0000","last_peered":"2024-09-10T16:00:00.493263+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:00.493263+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19312640,"num_objects":4715,"num_object_clones":0,"num_object_copies":9430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13331,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16115","reported_seq":16866,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.489080+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:00.489080+0000","last_peered":"2024-09-10T16:00:00.489080+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:00.489080+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18853888,"num_objects":4603,"num_object_clones":0,"num_object_copies":9206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12533,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'15999","reported_seq":16826,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.348419+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:00.348419+0000","last_peered":"2024-09-10T16:00:00.348419+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:00.348419+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12605,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16354","reported_seq":17116,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:00.480580+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:00.480580+0000","last_peered":"2024-09-10T16:00:00.480580+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:00.480580+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19431907,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13173,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":306627067,"num_objects":74862,"num_object_clones":0,"num_object_copies":149724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218188,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13309,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":803811328,"data_stored":803734537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626517,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590560,"kb_used_data":225740,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93781280,"statfs":{"total":96636764160,"available":96032030720,"internally_reserved":0,"allocated":231157760,"data_stored":230819382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":373550766},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149798,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":724344,"kb_used_data":287332,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93647496,"statfs":{"total":96636764160,"available":95895035904,"internally_reserved":0,"allocated":294227968,"data_stored":293951470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":447475373},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,2,1,1,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595424,"kb_used_data":275024,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93776416,"statfs":{"total":96636764160,"available":96027049984,"internally_reserved":0,"allocated":281624576,"data_stored":281344006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293543936,"data_stored":293540323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":229388288,"data_stored":229322795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280879104,"data_stored":280871419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:07.174 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:07.717 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:07.717 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:07.737 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-10T16:00:06.581145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":296784398,"num_objects":72461,"num_object_clones":0,"num_object_copies":144922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220593,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13455,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160673,"ondisk_log_size":160673,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1874728,"kb_used_data":769132,"kb_used_omap":12,"kb_used_meta":1105523,"kb_avail":281240792,"statfs":{"total":289910292480,"available":287990571008,"internally_reserved":0,"allocated":787591168,"data_stored":786695722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1132055557},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,3,10,17,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29593600,"num_objects":-7225,"num_object_clones":0,"num_object_copies":-14450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7225,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006692"},"pg_stats":[{"pgid":"2.f","version":"48'16263","reported_seq":17066,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.479806+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:05.479806+0000","last_peered":"2024-09-10T16:00:05.479806+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:05.479806+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18460672,"num_objects":4507,"num_object_clones":0,"num_object_copies":9014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16263,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16523","reported_seq":16636,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.610064+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:05.610064+0000","last_peered":"2024-09-10T16:00:05.610064+0000","last_clean":"2024-09-10T16:00:05.610064+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:05.610064+0000","last_undegraded":"2024-09-10T16:00:05.610064+0000","last_fullsized":"2024-09-10T16:00:05.610064+0000","mapping_epoch":36,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18640896,"num_objects":4551,"num_object_clones":0,"num_object_copies":9102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14859,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16079","reported_seq":16555,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.659709+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:05.659709+0000","last_peered":"2024-09-10T16:00:05.659709+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:05.659709+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18636824,"num_objects":4551,"num_object_clones":0,"num_object_copies":9102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14347,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16113","reported_seq":16198,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.635698+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:05.635698+0000","last_peered":"2024-09-10T16:00:05.635698+0000","last_clean":"2024-09-10T16:00:05.635698+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:05.635698+0000","last_undegraded":"2024-09-10T16:00:05.635698+0000","last_fullsized":"2024-09-10T16:00:05.635698+0000","mapping_epoch":36,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16113,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15901","reported_seq":16033,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.534684+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:05.534684+0000","last_peered":"2024-09-10T16:00:05.534684+0000","last_clean":"2024-09-10T16:00:05.534684+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:05.534684+0000","last_undegraded":"2024-09-10T16:00:05.534684+0000","last_fullsized":"2024-09-10T16:00:05.534684+0000","mapping_epoch":35,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18145280,"num_objects":4430,"num_object_clones":0,"num_object_copies":8860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12972,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16302","reported_seq":24195,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.964775+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:05.964775+0000","last_peered":"2024-09-10T16:00:05.964775+0000","last_clean":"2024-09-10T16:00:05.964775+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:05.964775+0000","last_undegraded":"2024-09-10T16:00:05.964775+0000","last_fullsized":"2024-09-10T16:00:05.964775+0000","mapping_epoch":47,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18698240,"num_objects":4565,"num_object_clones":0,"num_object_copies":9130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13020,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15858","reported_seq":16621,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.532437+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:05.532437+0000","last_peered":"2024-09-10T16:00:05.532437+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:05.532437+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18137088,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12508,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":904}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16333","reported_seq":16470,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.461873+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:05.461873+0000","last_peered":"2024-09-10T16:00:05.461873+0000","last_clean":"2024-09-10T16:00:05.461873+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:05.461873+0000","last_undegraded":"2024-09-10T16:00:05.461873+0000","last_fullsized":"2024-09-10T16:00:05.461873+0000","mapping_epoch":35,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13092,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16287","reported_seq":24420,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.944631+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:05.944631+0000","last_peered":"2024-09-10T16:00:05.944631+0000","last_clean":"2024-09-10T16:00:05.944631+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:05.944631+0000","last_undegraded":"2024-09-10T16:00:05.944631+0000","last_fullsized":"2024-09-10T16:00:05.944631+0000","mapping_epoch":45,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18153472,"num_objects":4432,"num_object_clones":0,"num_object_copies":8864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14264,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16053","reported_seq":16873,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.540471+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:05.540471+0000","last_peered":"2024-09-10T16:00:05.540471+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:05.540471+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12917,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1006}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16212","reported_seq":22261,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.620973+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:05.620973+0000","last_peered":"2024-09-10T16:00:05.620973+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:05.620973+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18731008,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14186,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2959,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3698}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15946","reported_seq":16359,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.631700+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:05.631700+0000","last_peered":"2024-09-10T16:00:05.631700+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:05.631700+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18472960,"num_objects":4510,"num_object_clones":0,"num_object_copies":9020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13815,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":824}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16294","reported_seq":17137,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.477162+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:05.477162+0000","last_peered":"2024-09-10T16:00:05.477162+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:05.477162+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18677760,"num_objects":4560,"num_object_clones":0,"num_object_copies":9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13486,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16250","reported_seq":17001,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.513082+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:05.513082+0000","last_peered":"2024-09-10T16:00:05.513082+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:05.513082+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18300928,"num_objects":4468,"num_object_clones":0,"num_object_copies":8936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12668,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16140","reported_seq":16967,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.497144+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:05.497144+0000","last_peered":"2024-09-10T16:00:05.497144+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:05.497144+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18948096,"num_objects":4626,"num_object_clones":0,"num_object_copies":9252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12746,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16516","reported_seq":17278,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.464818+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:05.464818+0000","last_peered":"2024-09-10T16:00:05.464818+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:05.464818+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18768355,"num_objects":4583,"num_object_clones":0,"num_object_copies":9166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13335,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":889}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":296784379,"num_objects":72459,"num_object_clones":0,"num_object_copies":144918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220591,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13453,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":784142336,"data_stored":784065545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626518,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571120,"kb_used_data":218780,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93800720,"statfs":{"total":96636764160,"available":96051937280,"internally_reserved":0,"allocated":224030720,"data_stored":223692342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":360771246},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149799,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708844,"kb_used_data":280148,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93662996,"statfs":{"total":96636764160,"available":95910907904,"internally_reserved":0,"allocated":286871552,"data_stored":286595054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":438955693},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594764,"kb_used_data":270204,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93777076,"statfs":{"total":96636764160,"available":96027725824,"internally_reserved":0,"allocated":276688896,"data_stored":276408326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":286138368,"data_stored":286134755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222052352,"data_stored":221986859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":275951616,"data_stored":275943931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:07.738 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:08.307 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:08.307 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:08.334 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-10T16:00:06.581145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":296784398,"num_objects":72461,"num_object_clones":0,"num_object_copies":144922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220593,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13455,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160673,"ondisk_log_size":160673,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1874728,"kb_used_data":769132,"kb_used_omap":12,"kb_used_meta":1105523,"kb_avail":281240792,"statfs":{"total":289910292480,"available":287990571008,"internally_reserved":0,"allocated":787591168,"data_stored":786695722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1132055557},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,3,10,17,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29593600,"num_objects":-7225,"num_object_clones":0,"num_object_copies":-14450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7225,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006692"},"pg_stats":[{"pgid":"2.f","version":"48'16263","reported_seq":17066,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.479806+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:05.479806+0000","last_peered":"2024-09-10T16:00:05.479806+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:05.479806+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18460672,"num_objects":4507,"num_object_clones":0,"num_object_copies":9014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16263,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16523","reported_seq":16636,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.610064+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:05.610064+0000","last_peered":"2024-09-10T16:00:05.610064+0000","last_clean":"2024-09-10T16:00:05.610064+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:05.610064+0000","last_undegraded":"2024-09-10T16:00:05.610064+0000","last_fullsized":"2024-09-10T16:00:05.610064+0000","mapping_epoch":36,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18640896,"num_objects":4551,"num_object_clones":0,"num_object_copies":9102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14859,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16079","reported_seq":16555,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.659709+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:05.659709+0000","last_peered":"2024-09-10T16:00:05.659709+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:05.659709+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18636824,"num_objects":4551,"num_object_clones":0,"num_object_copies":9102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14347,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16113","reported_seq":16198,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.635698+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:05.635698+0000","last_peered":"2024-09-10T16:00:05.635698+0000","last_clean":"2024-09-10T16:00:05.635698+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:05.635698+0000","last_undegraded":"2024-09-10T16:00:05.635698+0000","last_fullsized":"2024-09-10T16:00:05.635698+0000","mapping_epoch":36,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16113,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15901","reported_seq":16033,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.534684+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:05.534684+0000","last_peered":"2024-09-10T16:00:05.534684+0000","last_clean":"2024-09-10T16:00:05.534684+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:05.534684+0000","last_undegraded":"2024-09-10T16:00:05.534684+0000","last_fullsized":"2024-09-10T16:00:05.534684+0000","mapping_epoch":35,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18145280,"num_objects":4430,"num_object_clones":0,"num_object_copies":8860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12972,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16302","reported_seq":24195,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.964775+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:05.964775+0000","last_peered":"2024-09-10T16:00:05.964775+0000","last_clean":"2024-09-10T16:00:05.964775+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:05.964775+0000","last_undegraded":"2024-09-10T16:00:05.964775+0000","last_fullsized":"2024-09-10T16:00:05.964775+0000","mapping_epoch":47,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18698240,"num_objects":4565,"num_object_clones":0,"num_object_copies":9130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13020,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15858","reported_seq":16621,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.532437+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:05.532437+0000","last_peered":"2024-09-10T16:00:05.532437+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:05.532437+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18137088,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12508,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":904}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16333","reported_seq":16470,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.461873+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:05.461873+0000","last_peered":"2024-09-10T16:00:05.461873+0000","last_clean":"2024-09-10T16:00:05.461873+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:05.461873+0000","last_undegraded":"2024-09-10T16:00:05.461873+0000","last_fullsized":"2024-09-10T16:00:05.461873+0000","mapping_epoch":35,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13092,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16287","reported_seq":24420,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.944631+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:05.944631+0000","last_peered":"2024-09-10T16:00:05.944631+0000","last_clean":"2024-09-10T16:00:05.944631+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:05.944631+0000","last_undegraded":"2024-09-10T16:00:05.944631+0000","last_fullsized":"2024-09-10T16:00:05.944631+0000","mapping_epoch":45,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18153472,"num_objects":4432,"num_object_clones":0,"num_object_copies":8864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14264,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16053","reported_seq":16873,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.540471+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:05.540471+0000","last_peered":"2024-09-10T16:00:05.540471+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:05.540471+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12917,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1006}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16212","reported_seq":22261,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.620973+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:05.620973+0000","last_peered":"2024-09-10T16:00:05.620973+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:05.620973+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18731008,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14186,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2959,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3698}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15946","reported_seq":16359,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.631700+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:05.631700+0000","last_peered":"2024-09-10T16:00:05.631700+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:05.631700+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18472960,"num_objects":4510,"num_object_clones":0,"num_object_copies":9020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13815,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":824}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16294","reported_seq":17137,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.477162+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:05.477162+0000","last_peered":"2024-09-10T16:00:05.477162+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:05.477162+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18677760,"num_objects":4560,"num_object_clones":0,"num_object_copies":9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13486,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16250","reported_seq":17001,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.513082+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:05.513082+0000","last_peered":"2024-09-10T16:00:05.513082+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:05.513082+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18300928,"num_objects":4468,"num_object_clones":0,"num_object_copies":8936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12668,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16140","reported_seq":16967,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.497144+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:05.497144+0000","last_peered":"2024-09-10T16:00:05.497144+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:05.497144+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18948096,"num_objects":4626,"num_object_clones":0,"num_object_copies":9252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12746,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16516","reported_seq":17278,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.464818+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:05.464818+0000","last_peered":"2024-09-10T16:00:05.464818+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:05.464818+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18768355,"num_objects":4583,"num_object_clones":0,"num_object_copies":9166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13335,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":889}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":296784379,"num_objects":72459,"num_object_clones":0,"num_object_copies":144918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220591,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13453,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":784142336,"data_stored":784065545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626518,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571120,"kb_used_data":218780,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93800720,"statfs":{"total":96636764160,"available":96051937280,"internally_reserved":0,"allocated":224030720,"data_stored":223692342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":360771246},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149799,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708844,"kb_used_data":280148,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93662996,"statfs":{"total":96636764160,"available":95910907904,"internally_reserved":0,"allocated":286871552,"data_stored":286595054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":438955693},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594764,"kb_used_data":270204,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93777076,"statfs":{"total":96636764160,"available":96027725824,"internally_reserved":0,"allocated":276688896,"data_stored":276408326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":286138368,"data_stored":286134755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222052352,"data_stored":221986859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":275951616,"data_stored":275943931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:08.335 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:08.335 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'16263", 'reported_seq': 17066, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:05.479806+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T16:00:05.479806+0000', 'last_peered': '2024-09-10T16:00:05.479806+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:00:05.479806+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'6200", 'ondisk_log_start': "44'6200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18460672, 'num_objects': 4507, 'num_object_clones': 0, 'num_object_copies': 9014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4507, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16263, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 895}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:08.335 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:08.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'16079", 'reported_seq': 16555, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:05.659709+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:00:05.659709+0000', 'last_peered': '2024-09-10T16:00:05.659709+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:00:05.659709+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'6000", 'ondisk_log_start': "44'6000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18636824, 'num_objects': 4551, 'num_object_clones': 0, 'num_object_copies': 9102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4551, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14347, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 922}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:08.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:08.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'15858", 'reported_seq': 16621, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:05.532437+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:00:05.532437+0000', 'last_peered': '2024-09-10T16:00:05.532437+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:00:05.532437+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'5800", 'ondisk_log_start': "44'5800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18137088, 'num_objects': 4428, 'num_object_clones': 0, 'num_object_copies': 8856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12508, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 904}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:08.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:08.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'16053", 'reported_seq': 16873, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:05.540471+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:00:05.540471+0000', 'last_peered': '2024-09-10T16:00:05.540471+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:00:05.540471+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'6000", 'ondisk_log_start': "44'6000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18534400, 'num_objects': 4525, 'num_object_clones': 0, 'num_object_copies': 9050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4525, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12917, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1006}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:08.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:08.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'16212", 'reported_seq': 22261, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:05.620973+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T16:00:05.620973+0000', 'last_peered': '2024-09-10T16:00:05.620973+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T16:00:05.620973+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'6200", 'ondisk_log_start': "44'6200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18731008, 'num_objects': 4573, 'num_object_clones': 0, 'num_object_copies': 9146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 875, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4573, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14186, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2959, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3698}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:08.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:08.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'15946", 'reported_seq': 16359, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:05.631700+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:00:05.631700+0000', 'last_peered': '2024-09-10T16:00:05.631700+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:00:05.631700+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'5900", 'ondisk_log_start': "44'5900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18472960, 'num_objects': 4510, 'num_object_clones': 0, 'num_object_copies': 9020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4510, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13815, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 824}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:08.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:08.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'16294", 'reported_seq': 17137, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:05.477162+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:00:05.477162+0000', 'last_peered': '2024-09-10T16:00:05.477162+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:00:05.477162+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'6200", 'ondisk_log_start': "44'6200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18677760, 'num_objects': 4560, 'num_object_clones': 0, 'num_object_copies': 9120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4560, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13486, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:08.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:08.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'16250", 'reported_seq': 17001, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:05.513082+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:00:05.513082+0000', 'last_peered': '2024-09-10T16:00:05.513082+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:00:05.513082+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'6200", 'ondisk_log_start': "44'6200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18300928, 'num_objects': 4468, 'num_object_clones': 0, 'num_object_copies': 8936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4468, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12668, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 842}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:08.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:08.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'16140", 'reported_seq': 16967, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:05.497144+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:00:05.497144+0000', 'last_peered': '2024-09-10T16:00:05.497144+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:00:05.497144+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18948096, 'num_objects': 4626, 'num_object_clones': 0, 'num_object_copies': 9252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12746, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1090}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:08.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:08.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'16516", 'reported_seq': 17278, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:05.464818+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:00:05.464818+0000', 'last_peered': '2024-09-10T16:00:05.464818+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:00:05.464818+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'6500", 'ondisk_log_start': "44'6500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18768355, 'num_objects': 4583, 'num_object_clones': 0, 'num_object_copies': 9166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13335, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 889}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:08.337 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:09.022 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:09.022 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33195/144922 objects degraded (22.905%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":534,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":72461,"data_bytes":296784398,"bytes_used":1919721472,"bytes_avail":287990571008,"bytes_total":289910292480,"degraded_objects":33195,"degraded_total":144922,"degraded_ratio":0.2290542498723451,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":403},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T15:59:36.565732+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:00:09.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:09.053 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:09.633 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:09.635 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:09.664 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-10T16:00:08.582057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":296784398,"num_objects":72461,"num_object_clones":0,"num_object_copies":144922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220593,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13455,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160673,"ondisk_log_size":160673,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1874728,"kb_used_data":769132,"kb_used_omap":12,"kb_used_meta":1105523,"kb_avail":281240792,"statfs":{"total":289910292480,"available":287990571008,"internally_reserved":0,"allocated":787591168,"data_stored":786695722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1132055557},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,3,10,17,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19832832,"num_objects":-4842,"num_object_clones":0,"num_object_copies":-9684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4842,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007017"},"pg_stats":[{"pgid":"2.f","version":"48'16263","reported_seq":17066,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.479806+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:05.479806+0000","last_peered":"2024-09-10T16:00:05.479806+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:05.479806+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18460672,"num_objects":4507,"num_object_clones":0,"num_object_copies":9014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16263,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16523","reported_seq":16636,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.610064+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:05.610064+0000","last_peered":"2024-09-10T16:00:05.610064+0000","last_clean":"2024-09-10T16:00:05.610064+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:05.610064+0000","last_undegraded":"2024-09-10T16:00:05.610064+0000","last_fullsized":"2024-09-10T16:00:05.610064+0000","mapping_epoch":36,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18640896,"num_objects":4551,"num_object_clones":0,"num_object_copies":9102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14859,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16079","reported_seq":16555,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.659709+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:05.659709+0000","last_peered":"2024-09-10T16:00:05.659709+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:05.659709+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18636824,"num_objects":4551,"num_object_clones":0,"num_object_copies":9102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14347,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16113","reported_seq":16198,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.635698+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:05.635698+0000","last_peered":"2024-09-10T16:00:05.635698+0000","last_clean":"2024-09-10T16:00:05.635698+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:05.635698+0000","last_undegraded":"2024-09-10T16:00:05.635698+0000","last_fullsized":"2024-09-10T16:00:05.635698+0000","mapping_epoch":36,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16113,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'15901","reported_seq":16033,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.534684+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:05.534684+0000","last_peered":"2024-09-10T16:00:05.534684+0000","last_clean":"2024-09-10T16:00:05.534684+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:05.534684+0000","last_undegraded":"2024-09-10T16:00:05.534684+0000","last_fullsized":"2024-09-10T16:00:05.534684+0000","mapping_epoch":35,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18145280,"num_objects":4430,"num_object_clones":0,"num_object_copies":8860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12972,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16302","reported_seq":24195,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.964775+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:05.964775+0000","last_peered":"2024-09-10T16:00:05.964775+0000","last_clean":"2024-09-10T16:00:05.964775+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:05.964775+0000","last_undegraded":"2024-09-10T16:00:05.964775+0000","last_fullsized":"2024-09-10T16:00:05.964775+0000","mapping_epoch":47,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18698240,"num_objects":4565,"num_object_clones":0,"num_object_copies":9130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13020,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'15858","reported_seq":16621,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.532437+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:05.532437+0000","last_peered":"2024-09-10T16:00:05.532437+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:05.532437+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18137088,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12508,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":904}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16333","reported_seq":16470,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.461873+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:05.461873+0000","last_peered":"2024-09-10T16:00:05.461873+0000","last_clean":"2024-09-10T16:00:05.461873+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:05.461873+0000","last_undegraded":"2024-09-10T16:00:05.461873+0000","last_fullsized":"2024-09-10T16:00:05.461873+0000","mapping_epoch":35,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13092,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16287","reported_seq":24420,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:05.944631+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:05.944631+0000","last_peered":"2024-09-10T16:00:05.944631+0000","last_clean":"2024-09-10T16:00:05.944631+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:05.944631+0000","last_undegraded":"2024-09-10T16:00:05.944631+0000","last_fullsized":"2024-09-10T16:00:05.944631+0000","mapping_epoch":45,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18153472,"num_objects":4432,"num_object_clones":0,"num_object_copies":8864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14264,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16053","reported_seq":16873,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.540471+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:05.540471+0000","last_peered":"2024-09-10T16:00:05.540471+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:05.540471+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12917,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1006}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16212","reported_seq":22261,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.620973+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:05.620973+0000","last_peered":"2024-09-10T16:00:05.620973+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:05.620973+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18731008,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14186,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2959,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3698}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'15946","reported_seq":16359,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.631700+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:05.631700+0000","last_peered":"2024-09-10T16:00:05.631700+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:05.631700+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18472960,"num_objects":4510,"num_object_clones":0,"num_object_copies":9020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13815,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":824}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16294","reported_seq":17137,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.477162+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:05.477162+0000","last_peered":"2024-09-10T16:00:05.477162+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:05.477162+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18677760,"num_objects":4560,"num_object_clones":0,"num_object_copies":9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13486,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16250","reported_seq":17001,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.513082+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:05.513082+0000","last_peered":"2024-09-10T16:00:05.513082+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:05.513082+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18300928,"num_objects":4468,"num_object_clones":0,"num_object_copies":8936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12668,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16140","reported_seq":16967,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.497144+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:05.497144+0000","last_peered":"2024-09-10T16:00:05.497144+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:05.497144+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18948096,"num_objects":4626,"num_object_clones":0,"num_object_copies":9252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12746,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16516","reported_seq":17278,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:05.464818+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:05.464818+0000","last_peered":"2024-09-10T16:00:05.464818+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:05.464818+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18768355,"num_objects":4583,"num_object_clones":0,"num_object_copies":9166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13335,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":889}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":296784379,"num_objects":72459,"num_object_clones":0,"num_object_copies":144918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220591,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13453,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":784142336,"data_stored":784065545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626518,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571120,"kb_used_data":218780,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93800720,"statfs":{"total":96636764160,"available":96051937280,"internally_reserved":0,"allocated":224030720,"data_stored":223692342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":360771246},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149799,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708844,"kb_used_data":280148,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93662996,"statfs":{"total":96636764160,"available":95910907904,"internally_reserved":0,"allocated":286871552,"data_stored":286595054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":438955693},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594764,"kb_used_data":270204,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93777076,"statfs":{"total":96636764160,"available":96027725824,"internally_reserved":0,"allocated":276688896,"data_stored":276408326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":286138368,"data_stored":286134755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":222052352,"data_stored":221986859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":275951616,"data_stored":275943931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:12.667 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:13.231 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:13.232 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:13.255 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-10T16:00:12.588188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":286781966,"num_objects":70019,"num_object_clones":0,"num_object_copies":140038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70019,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223035,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13580,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160715,"ondisk_log_size":160715,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871924,"kb_used_data":749680,"kb_used_omap":12,"kb_used_meta":1122163,"kb_avail":281243596,"statfs":{"total":289910292480,"available":287993442304,"internally_reserved":0,"allocated":767672320,"data_stored":766776874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1149094917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,4,14,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24322048,"num_objects":-5938,"num_object_clones":0,"num_object_copies":-11876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5938,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011752"},"pg_stats":[{"pgid":"2.f","version":"48'16404","reported_seq":17208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.517976+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:10.517976+0000","last_peered":"2024-09-10T16:00:10.517976+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:10.517976+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17883136,"num_objects":4366,"num_object_clones":0,"num_object_copies":8732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16404,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":754}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16698","reported_seq":16809,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.673487+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:10.673487+0000","last_peered":"2024-09-10T16:00:10.673487+0000","last_clean":"2024-09-10T16:00:10.673487+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:10.673487+0000","last_undegraded":"2024-09-10T16:00:10.673487+0000","last_fullsized":"2024-09-10T16:00:10.673487+0000","mapping_epoch":36,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17924096,"num_objects":4376,"num_object_clones":0,"num_object_copies":8752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15034,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16223","reported_seq":16699,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.642687+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:10.642687+0000","last_peered":"2024-09-10T16:00:10.642687+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:10.642687+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18047000,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14491,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":778}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16251","reported_seq":16336,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.624074+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:10.624074+0000","last_peered":"2024-09-10T16:00:10.624074+0000","last_clean":"2024-09-10T16:00:10.624074+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:10.624074+0000","last_undegraded":"2024-09-10T16:00:10.624074+0000","last_fullsized":"2024-09-10T16:00:10.624074+0000","mapping_epoch":36,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16251,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16049","reported_seq":16179,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.565402+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:10.565402+0000","last_peered":"2024-09-10T16:00:10.565402+0000","last_clean":"2024-09-10T16:00:10.565402+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:10.565402+0000","last_undegraded":"2024-09-10T16:00:10.565402+0000","last_fullsized":"2024-09-10T16:00:10.565402+0000","mapping_epoch":35,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17539072,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13120,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16477","reported_seq":24370,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.971712+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:10.971712+0000","last_peered":"2024-09-10T16:00:10.971712+0000","last_clean":"2024-09-10T16:00:10.971712+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:10.971712+0000","last_undegraded":"2024-09-10T16:00:10.971712+0000","last_fullsized":"2024-09-10T16:00:10.971712+0000","mapping_epoch":47,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17981440,"num_objects":4390,"num_object_clones":0,"num_object_copies":8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13195,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16009","reported_seq":16775,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.505022+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:10.505022+0000","last_peered":"2024-09-10T16:00:10.505022+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:10.505022+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17518592,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12659,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":753}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16481","reported_seq":16618,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.525153+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:10.525153+0000","last_peered":"2024-09-10T16:00:10.525153+0000","last_clean":"2024-09-10T16:00:10.525153+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:10.525153+0000","last_undegraded":"2024-09-10T16:00:10.525153+0000","last_fullsized":"2024-09-10T16:00:10.525153+0000","mapping_epoch":35,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18157568,"num_objects":4433,"num_object_clones":0,"num_object_copies":8866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13240,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16442","reported_seq":24575,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.946410+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:10.946410+0000","last_peered":"2024-09-10T16:00:10.946410+0000","last_clean":"2024-09-10T16:00:10.946410+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:10.946410+0000","last_undegraded":"2024-09-10T16:00:10.946410+0000","last_fullsized":"2024-09-10T16:00:10.946410+0000","mapping_epoch":45,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17518592,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14419,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16201","reported_seq":17020,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.537624+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:10.537624+0000","last_peered":"2024-09-10T16:00:10.537624+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:10.537624+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17928192,"num_objects":4377,"num_object_clones":0,"num_object_copies":8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13065,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":858}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16366","reported_seq":22665,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.664243+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:10.664243+0000","last_peered":"2024-09-10T16:00:10.664243+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:10.664243+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18100224,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14340,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3084,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3669}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16104","reported_seq":16521,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.602087+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:10.602087+0000","last_peered":"2024-09-10T16:00:10.602087+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:10.602087+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13973,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":666}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16431","reported_seq":17274,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.485234+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:10.485234+0000","last_peered":"2024-09-10T16:00:10.485234+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:10.485234+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18116608,"num_objects":4423,"num_object_clones":0,"num_object_copies":8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13623,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16382","reported_seq":17133,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.484305+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:10.484305+0000","last_peered":"2024-09-10T16:00:10.484305+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:10.484305+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12800,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16319","reported_seq":17146,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.511573+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:10.511573+0000","last_peered":"2024-09-10T16:00:10.511573+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:10.511573+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18214912,"num_objects":4447,"num_object_clones":0,"num_object_copies":8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12925,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16675","reported_seq":17436,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.553107+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:10.553107+0000","last_peered":"2024-09-10T16:00:10.553107+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:10.553107+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18117091,"num_objects":4424,"num_object_clones":0,"num_object_copies":8848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13494,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":286781947,"num_objects":70017,"num_object_clones":0,"num_object_copies":140034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70017,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223033,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13578,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":764116992,"data_stored":764040201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160712,"ondisk_log_size":160712,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626519,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572044,"kb_used_data":211388,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93799796,"statfs":{"total":96636764160,"available":96050991104,"internally_reserved":0,"allocated":216461312,"data_stored":216122934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369290926},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149801,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705788,"kb_used_data":272936,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93666052,"statfs":{"total":96636764160,"available":95914037248,"internally_reserved":0,"allocated":279486464,"data_stored":279209966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":443215533},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594092,"kb_used_data":265356,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93777748,"statfs":{"total":96636764160,"available":96028413952,"internally_reserved":0,"allocated":271724544,"data_stored":271443974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278646784,"data_stored":278643171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":214536192,"data_stored":214470699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270934016,"data_stored":270926331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:13.256 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:13.799 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:13.804 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:13.831 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-10T16:00:12.588188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":286781966,"num_objects":70019,"num_object_clones":0,"num_object_copies":140038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70019,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223035,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13580,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160715,"ondisk_log_size":160715,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871924,"kb_used_data":749680,"kb_used_omap":12,"kb_used_meta":1122163,"kb_avail":281243596,"statfs":{"total":289910292480,"available":287993442304,"internally_reserved":0,"allocated":767672320,"data_stored":766776874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1149094917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,4,14,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24322048,"num_objects":-5938,"num_object_clones":0,"num_object_copies":-11876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5938,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011752"},"pg_stats":[{"pgid":"2.f","version":"48'16404","reported_seq":17208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.517976+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:10.517976+0000","last_peered":"2024-09-10T16:00:10.517976+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:10.517976+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17883136,"num_objects":4366,"num_object_clones":0,"num_object_copies":8732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16404,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":754}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16698","reported_seq":16809,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.673487+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:10.673487+0000","last_peered":"2024-09-10T16:00:10.673487+0000","last_clean":"2024-09-10T16:00:10.673487+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:10.673487+0000","last_undegraded":"2024-09-10T16:00:10.673487+0000","last_fullsized":"2024-09-10T16:00:10.673487+0000","mapping_epoch":36,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17924096,"num_objects":4376,"num_object_clones":0,"num_object_copies":8752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15034,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16223","reported_seq":16699,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.642687+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:10.642687+0000","last_peered":"2024-09-10T16:00:10.642687+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:10.642687+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18047000,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14491,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":778}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16251","reported_seq":16336,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.624074+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:10.624074+0000","last_peered":"2024-09-10T16:00:10.624074+0000","last_clean":"2024-09-10T16:00:10.624074+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:10.624074+0000","last_undegraded":"2024-09-10T16:00:10.624074+0000","last_fullsized":"2024-09-10T16:00:10.624074+0000","mapping_epoch":36,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16251,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16049","reported_seq":16179,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.565402+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:10.565402+0000","last_peered":"2024-09-10T16:00:10.565402+0000","last_clean":"2024-09-10T16:00:10.565402+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:10.565402+0000","last_undegraded":"2024-09-10T16:00:10.565402+0000","last_fullsized":"2024-09-10T16:00:10.565402+0000","mapping_epoch":35,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17539072,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13120,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16477","reported_seq":24370,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.971712+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:10.971712+0000","last_peered":"2024-09-10T16:00:10.971712+0000","last_clean":"2024-09-10T16:00:10.971712+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:10.971712+0000","last_undegraded":"2024-09-10T16:00:10.971712+0000","last_fullsized":"2024-09-10T16:00:10.971712+0000","mapping_epoch":47,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17981440,"num_objects":4390,"num_object_clones":0,"num_object_copies":8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13195,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16009","reported_seq":16775,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.505022+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:10.505022+0000","last_peered":"2024-09-10T16:00:10.505022+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:10.505022+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17518592,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12659,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":753}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16481","reported_seq":16618,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.525153+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:10.525153+0000","last_peered":"2024-09-10T16:00:10.525153+0000","last_clean":"2024-09-10T16:00:10.525153+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:10.525153+0000","last_undegraded":"2024-09-10T16:00:10.525153+0000","last_fullsized":"2024-09-10T16:00:10.525153+0000","mapping_epoch":35,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18157568,"num_objects":4433,"num_object_clones":0,"num_object_copies":8866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13240,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16442","reported_seq":24575,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.946410+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:10.946410+0000","last_peered":"2024-09-10T16:00:10.946410+0000","last_clean":"2024-09-10T16:00:10.946410+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:10.946410+0000","last_undegraded":"2024-09-10T16:00:10.946410+0000","last_fullsized":"2024-09-10T16:00:10.946410+0000","mapping_epoch":45,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17518592,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14419,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16201","reported_seq":17020,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.537624+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:10.537624+0000","last_peered":"2024-09-10T16:00:10.537624+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:10.537624+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17928192,"num_objects":4377,"num_object_clones":0,"num_object_copies":8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13065,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":858}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16366","reported_seq":22665,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.664243+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:10.664243+0000","last_peered":"2024-09-10T16:00:10.664243+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:10.664243+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18100224,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14340,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3084,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3669}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16104","reported_seq":16521,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.602087+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:10.602087+0000","last_peered":"2024-09-10T16:00:10.602087+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:10.602087+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13973,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":666}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16431","reported_seq":17274,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.485234+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:10.485234+0000","last_peered":"2024-09-10T16:00:10.485234+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:10.485234+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18116608,"num_objects":4423,"num_object_clones":0,"num_object_copies":8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13623,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16382","reported_seq":17133,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.484305+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:10.484305+0000","last_peered":"2024-09-10T16:00:10.484305+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:10.484305+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12800,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16319","reported_seq":17146,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.511573+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:10.511573+0000","last_peered":"2024-09-10T16:00:10.511573+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:10.511573+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18214912,"num_objects":4447,"num_object_clones":0,"num_object_copies":8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12925,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16675","reported_seq":17436,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.553107+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:10.553107+0000","last_peered":"2024-09-10T16:00:10.553107+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:10.553107+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18117091,"num_objects":4424,"num_object_clones":0,"num_object_copies":8848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13494,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":286781947,"num_objects":70017,"num_object_clones":0,"num_object_copies":140034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70017,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223033,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13578,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":764116992,"data_stored":764040201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160712,"ondisk_log_size":160712,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626519,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572044,"kb_used_data":211388,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93799796,"statfs":{"total":96636764160,"available":96050991104,"internally_reserved":0,"allocated":216461312,"data_stored":216122934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369290926},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149801,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705788,"kb_used_data":272936,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93666052,"statfs":{"total":96636764160,"available":95914037248,"internally_reserved":0,"allocated":279486464,"data_stored":279209966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":443215533},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594092,"kb_used_data":265356,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93777748,"statfs":{"total":96636764160,"available":96028413952,"internally_reserved":0,"allocated":271724544,"data_stored":271443974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278646784,"data_stored":278643171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":214536192,"data_stored":214470699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270934016,"data_stored":270926331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:13.832 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:13.832 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'16404", 'reported_seq': 17208, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:10.517976+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T16:00:10.517976+0000', 'last_peered': '2024-09-10T16:00:10.517976+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:00:10.517976+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'6400", 'ondisk_log_start': "44'6400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17883136, 'num_objects': 4366, 'num_object_clones': 0, 'num_object_copies': 8732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4366, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16404, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 754}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:13.832 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:13.832 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'16223", 'reported_seq': 16699, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:10.642687+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:00:10.642687+0000', 'last_peered': '2024-09-10T16:00:10.642687+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:00:10.642687+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'6200", 'ondisk_log_start': "44'6200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18047000, 'num_objects': 4407, 'num_object_clones': 0, 'num_object_copies': 8814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4407, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14491, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 778}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:13.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:13.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'16009", 'reported_seq': 16775, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:10.505022+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:00:10.505022+0000', 'last_peered': '2024-09-10T16:00:10.505022+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:00:10.505022+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'6000", 'ondisk_log_start': "44'6000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17518592, 'num_objects': 4277, 'num_object_clones': 0, 'num_object_copies': 8554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4277, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12659, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 753}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:13.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:13.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'16201", 'reported_seq': 17020, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:10.537624+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:00:10.537624+0000', 'last_peered': '2024-09-10T16:00:10.537624+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:00:10.537624+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'6200", 'ondisk_log_start': "44'6200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17928192, 'num_objects': 4377, 'num_object_clones': 0, 'num_object_copies': 8754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13065, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 858}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:13.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:13.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'16366", 'reported_seq': 22665, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:10.664243+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T16:00:10.664243+0000', 'last_peered': '2024-09-10T16:00:10.664243+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T16:00:10.664243+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'6300", 'ondisk_log_start': "44'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18100224, 'num_objects': 4419, 'num_object_clones': 0, 'num_object_copies': 8838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 750, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4419, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14340, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3084, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3669}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:13.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:13.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'16104", 'reported_seq': 16521, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:10.602087+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:00:10.602087+0000', 'last_peered': '2024-09-10T16:00:10.602087+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:00:10.602087+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17825792, 'num_objects': 4352, 'num_object_clones': 0, 'num_object_copies': 8704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4352, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13973, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 666}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:13.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:13.833 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'16431", 'reported_seq': 17274, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:10.485234+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:00:10.485234+0000', 'last_peered': '2024-09-10T16:00:10.485234+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:00:10.485234+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'6400", 'ondisk_log_start': "44'6400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18116608, 'num_objects': 4423, 'num_object_clones': 0, 'num_object_copies': 8846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4423, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13623, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 945}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:13.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:13.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'16382", 'reported_seq': 17133, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:10.484305+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:00:10.484305+0000', 'last_peered': '2024-09-10T16:00:10.484305+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:00:10.484305+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'6300", 'ondisk_log_start': "44'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17760256, 'num_objects': 4336, 'num_object_clones': 0, 'num_object_copies': 8672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4336, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12800, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 710}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:13.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:13.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'16319", 'reported_seq': 17146, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:10.511573+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:00:10.511573+0000', 'last_peered': '2024-09-10T16:00:10.511573+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:00:10.511573+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'6300", 'ondisk_log_start': "44'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18214912, 'num_objects': 4447, 'num_object_clones': 0, 'num_object_copies': 8894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4447, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12925, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 911}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:13.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:13.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'16675", 'reported_seq': 17436, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:10.553107+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:00:10.553107+0000', 'last_peered': '2024-09-10T16:00:10.553107+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:00:10.553107+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'6600", 'ondisk_log_start': "44'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18117091, 'num_objects': 4424, 'num_object_clones': 0, 'num_object_copies': 8848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4424, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13494, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 730}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:13.834 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:14.566 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:14.567 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33070/140038 objects degraded (23.615%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":540,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":70019,"data_bytes":286781966,"bytes_used":1916850176,"bytes_avail":287993442304,"bytes_total":289910292480,"degraded_objects":33070,"degraded_total":140038,"degraded_ratio":0.23615018780616689,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":386,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":494},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T15:59:36.565732+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:00:14.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:14.596 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:15.822 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:15.825 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:15.843 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-10T16:00:14.589050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":286781966,"num_objects":70019,"num_object_clones":0,"num_object_copies":140038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70019,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223035,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13580,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160715,"ondisk_log_size":160715,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871924,"kb_used_data":749680,"kb_used_omap":12,"kb_used_meta":1122163,"kb_avail":281243596,"statfs":{"total":289910292480,"available":287993442304,"internally_reserved":0,"allocated":767672320,"data_stored":766776874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1149094917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,4,14,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19845120,"num_objects":-4845,"num_object_clones":0,"num_object_copies":-9690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4845,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011777"},"pg_stats":[{"pgid":"2.f","version":"48'16404","reported_seq":17208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.517976+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:10.517976+0000","last_peered":"2024-09-10T16:00:10.517976+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:10.517976+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17883136,"num_objects":4366,"num_object_clones":0,"num_object_copies":8732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16404,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":754}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16698","reported_seq":16809,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.673487+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:10.673487+0000","last_peered":"2024-09-10T16:00:10.673487+0000","last_clean":"2024-09-10T16:00:10.673487+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:10.673487+0000","last_undegraded":"2024-09-10T16:00:10.673487+0000","last_fullsized":"2024-09-10T16:00:10.673487+0000","mapping_epoch":36,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17924096,"num_objects":4376,"num_object_clones":0,"num_object_copies":8752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15034,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16223","reported_seq":16699,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.642687+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:10.642687+0000","last_peered":"2024-09-10T16:00:10.642687+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:10.642687+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18047000,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14491,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":778}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16251","reported_seq":16336,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.624074+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:10.624074+0000","last_peered":"2024-09-10T16:00:10.624074+0000","last_clean":"2024-09-10T16:00:10.624074+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:10.624074+0000","last_undegraded":"2024-09-10T16:00:10.624074+0000","last_fullsized":"2024-09-10T16:00:10.624074+0000","mapping_epoch":36,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16251,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16049","reported_seq":16179,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.565402+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:10.565402+0000","last_peered":"2024-09-10T16:00:10.565402+0000","last_clean":"2024-09-10T16:00:10.565402+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:10.565402+0000","last_undegraded":"2024-09-10T16:00:10.565402+0000","last_fullsized":"2024-09-10T16:00:10.565402+0000","mapping_epoch":35,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17539072,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13120,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16477","reported_seq":24370,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.971712+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:10.971712+0000","last_peered":"2024-09-10T16:00:10.971712+0000","last_clean":"2024-09-10T16:00:10.971712+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:10.971712+0000","last_undegraded":"2024-09-10T16:00:10.971712+0000","last_fullsized":"2024-09-10T16:00:10.971712+0000","mapping_epoch":47,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17981440,"num_objects":4390,"num_object_clones":0,"num_object_copies":8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13195,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16009","reported_seq":16775,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.505022+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:10.505022+0000","last_peered":"2024-09-10T16:00:10.505022+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:10.505022+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17518592,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12659,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":753}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16481","reported_seq":16618,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.525153+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:10.525153+0000","last_peered":"2024-09-10T16:00:10.525153+0000","last_clean":"2024-09-10T16:00:10.525153+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:10.525153+0000","last_undegraded":"2024-09-10T16:00:10.525153+0000","last_fullsized":"2024-09-10T16:00:10.525153+0000","mapping_epoch":35,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18157568,"num_objects":4433,"num_object_clones":0,"num_object_copies":8866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13240,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16442","reported_seq":24575,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:10.946410+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:10.946410+0000","last_peered":"2024-09-10T16:00:10.946410+0000","last_clean":"2024-09-10T16:00:10.946410+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:10.946410+0000","last_undegraded":"2024-09-10T16:00:10.946410+0000","last_fullsized":"2024-09-10T16:00:10.946410+0000","mapping_epoch":45,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17518592,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14419,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16201","reported_seq":17020,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.537624+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:10.537624+0000","last_peered":"2024-09-10T16:00:10.537624+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:10.537624+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17928192,"num_objects":4377,"num_object_clones":0,"num_object_copies":8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13065,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":858}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16366","reported_seq":22665,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.664243+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:10.664243+0000","last_peered":"2024-09-10T16:00:10.664243+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:10.664243+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18100224,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14340,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3084,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3669}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16104","reported_seq":16521,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.602087+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:10.602087+0000","last_peered":"2024-09-10T16:00:10.602087+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:10.602087+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13973,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":666}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16431","reported_seq":17274,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.485234+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:10.485234+0000","last_peered":"2024-09-10T16:00:10.485234+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:10.485234+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18116608,"num_objects":4423,"num_object_clones":0,"num_object_copies":8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13623,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16382","reported_seq":17133,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.484305+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:10.484305+0000","last_peered":"2024-09-10T16:00:10.484305+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:10.484305+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12800,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16319","reported_seq":17146,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.511573+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:10.511573+0000","last_peered":"2024-09-10T16:00:10.511573+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:10.511573+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18214912,"num_objects":4447,"num_object_clones":0,"num_object_copies":8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12925,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16675","reported_seq":17436,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:10.553107+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:10.553107+0000","last_peered":"2024-09-10T16:00:10.553107+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:10.553107+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18117091,"num_objects":4424,"num_object_clones":0,"num_object_copies":8848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13494,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":286781947,"num_objects":70017,"num_object_clones":0,"num_object_copies":140034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70017,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223033,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13578,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":764116992,"data_stored":764040201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160712,"ondisk_log_size":160712,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626519,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572044,"kb_used_data":211388,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93799796,"statfs":{"total":96636764160,"available":96050991104,"internally_reserved":0,"allocated":216461312,"data_stored":216122934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":369290926},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149801,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705788,"kb_used_data":272936,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93666052,"statfs":{"total":96636764160,"available":95914037248,"internally_reserved":0,"allocated":279486464,"data_stored":279209966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":443215533},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594092,"kb_used_data":265356,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93777748,"statfs":{"total":96636764160,"available":96028413952,"internally_reserved":0,"allocated":271724544,"data_stored":271443974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278646784,"data_stored":278643171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":214536192,"data_stored":214470699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270934016,"data_stored":270926331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:18.845 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:19.357 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:19.358 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:19.369 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-10T16:00:18.590342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":276771342,"num_objects":67575,"num_object_clones":0,"num_object_copies":135150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67575,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225479,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13691,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160859,"ondisk_log_size":160859,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1850916,"kb_used_data":728676,"kb_used_omap":12,"kb_used_meta":1122163,"kb_avail":281264604,"statfs":{"total":289910292480,"available":288014954496,"internally_reserved":0,"allocated":746164224,"data_stored":745268778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1149094917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,12,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20013056,"num_objects":-4886,"num_object_clones":0,"num_object_copies":-9772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4886,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009196"},"pg_stats":[{"pgid":"2.f","version":"48'16556","reported_seq":17360,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.480980+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:15.480980+0000","last_peered":"2024-09-10T16:00:15.480980+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:15.480980+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17260544,"num_objects":4214,"num_object_clones":0,"num_object_copies":8428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16556,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":602}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16858","reported_seq":16971,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.612138+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:15.612138+0000","last_peered":"2024-09-10T16:00:15.612138+0000","last_clean":"2024-09-10T16:00:15.612138+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:15.612138+0000","last_undegraded":"2024-09-10T16:00:15.612138+0000","last_fullsized":"2024-09-10T16:00:15.612138+0000","mapping_epoch":36,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17268736,"num_objects":4216,"num_object_clones":0,"num_object_copies":8432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15194,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16385","reported_seq":16860,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.686906+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:15.686906+0000","last_peered":"2024-09-10T16:00:15.686906+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:15.686906+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17383448,"num_objects":4245,"num_object_clones":0,"num_object_copies":8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14653,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":616}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16411","reported_seq":16496,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.645879+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:15.645879+0000","last_peered":"2024-09-10T16:00:15.645879+0000","last_clean":"2024-09-10T16:00:15.645879+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:15.645879+0000","last_undegraded":"2024-09-10T16:00:15.645879+0000","last_fullsized":"2024-09-10T16:00:15.645879+0000","mapping_epoch":36,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494016,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16411,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16191","reported_seq":16322,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.571308+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:15.571308+0000","last_peered":"2024-09-10T16:00:15.571308+0000","last_clean":"2024-09-10T16:00:15.571308+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:15.571308+0000","last_undegraded":"2024-09-10T16:00:15.571308+0000","last_fullsized":"2024-09-10T16:00:15.571308+0000","mapping_epoch":35,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16957440,"num_objects":4140,"num_object_clones":0,"num_object_copies":8280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13262,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16617","reported_seq":24510,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.959229+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:15.959229+0000","last_peered":"2024-09-10T16:00:15.959229+0000","last_clean":"2024-09-10T16:00:15.959229+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:15.959229+0000","last_undegraded":"2024-09-10T16:00:15.959229+0000","last_fullsized":"2024-09-10T16:00:15.959229+0000","mapping_epoch":47,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17408000,"num_objects":4250,"num_object_clones":0,"num_object_copies":8500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13335,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16175","reported_seq":16941,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.562344+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:15.562344+0000","last_peered":"2024-09-10T16:00:15.562344+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:15.562344+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16838656,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12825,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":587}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16647","reported_seq":16784,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.546825+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:15.546825+0000","last_peered":"2024-09-10T16:00:15.546825+0000","last_clean":"2024-09-10T16:00:15.546825+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:15.546825+0000","last_undegraded":"2024-09-10T16:00:15.546825+0000","last_fullsized":"2024-09-10T16:00:15.546825+0000","mapping_epoch":35,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17477632,"num_objects":4267,"num_object_clones":0,"num_object_copies":8534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13406,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16599","reported_seq":24732,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:16.025897+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:16.025897+0000","last_peered":"2024-09-10T16:00:16.025897+0000","last_clean":"2024-09-10T16:00:16.025897+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:16.025897+0000","last_undegraded":"2024-09-10T16:00:16.025897+0000","last_fullsized":"2024-09-10T16:00:16.025897+0000","mapping_epoch":45,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14576,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16348","reported_seq":17168,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.443028+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:15.443028+0000","last_peered":"2024-09-10T16:00:15.443028+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:15.443028+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17326080,"num_objects":4230,"num_object_clones":0,"num_object_copies":8460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13212,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16513","reported_seq":23034,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.691979+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:15.691979+0000","last_peered":"2024-09-10T16:00:15.691979+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:15.691979+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17498112,"num_objects":4272,"num_object_clones":0,"num_object_copies":8544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14487,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3195,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3633}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16258","reported_seq":16676,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.637894+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:15.637894+0000","last_peered":"2024-09-10T16:00:15.637894+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:15.637894+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17195008,"num_objects":4198,"num_object_clones":0,"num_object_copies":8396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14127,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":512}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16596","reported_seq":17439,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.607765+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:15.607765+0000","last_peered":"2024-09-10T16:00:15.607765+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:15.607765+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17440768,"num_objects":4258,"num_object_clones":0,"num_object_copies":8516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13788,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":780}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16525","reported_seq":17273,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.554482+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:15.554482+0000","last_peered":"2024-09-10T16:00:15.554482+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:15.554482+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17174528,"num_objects":4193,"num_object_clones":0,"num_object_copies":8386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12943,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16473","reported_seq":17298,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.606000+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:15.606000+0000","last_peered":"2024-09-10T16:00:15.606000+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:15.606000+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17584128,"num_objects":4293,"num_object_clones":0,"num_object_copies":8586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13079,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16804","reported_seq":17566,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.567797+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:15.567797+0000","last_peered":"2024-09-10T16:00:15.567797+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:15.567797+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17588707,"num_objects":4295,"num_object_clones":0,"num_object_copies":8590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13623,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":601}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":276771323,"num_objects":67573,"num_object_clones":0,"num_object_copies":135146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67573,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225477,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13689,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":743989248,"data_stored":743912457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626520,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571584,"kb_used_data":202604,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93800256,"statfs":{"total":96636764160,"available":96051462144,"internally_reserved":0,"allocated":207466496,"data_stored":207128118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":377810606},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,3,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149802,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":702652,"kb_used_data":265644,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93669188,"statfs":{"total":96636764160,"available":95917248512,"internally_reserved":0,"allocated":272019456,"data_stored":271742958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":447475373},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576680,"kb_used_data":260428,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93795160,"statfs":{"total":96636764160,"available":96046243840,"internally_reserved":0,"allocated":266678272,"data_stored":266397702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":271106048,"data_stored":271102435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207032320,"data_stored":206966827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":265850880,"data_stored":265843195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:19.371 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:19.897 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:19.897 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:19.912 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-10T16:00:18.590342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":276771342,"num_objects":67575,"num_object_clones":0,"num_object_copies":135150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67575,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225479,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13691,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160859,"ondisk_log_size":160859,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1850916,"kb_used_data":728676,"kb_used_omap":12,"kb_used_meta":1122163,"kb_avail":281264604,"statfs":{"total":289910292480,"available":288014954496,"internally_reserved":0,"allocated":746164224,"data_stored":745268778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1149094917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,12,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20013056,"num_objects":-4886,"num_object_clones":0,"num_object_copies":-9772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4886,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009196"},"pg_stats":[{"pgid":"2.f","version":"48'16556","reported_seq":17360,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.480980+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:15.480980+0000","last_peered":"2024-09-10T16:00:15.480980+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:15.480980+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17260544,"num_objects":4214,"num_object_clones":0,"num_object_copies":8428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16556,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":602}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16858","reported_seq":16971,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.612138+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:15.612138+0000","last_peered":"2024-09-10T16:00:15.612138+0000","last_clean":"2024-09-10T16:00:15.612138+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:15.612138+0000","last_undegraded":"2024-09-10T16:00:15.612138+0000","last_fullsized":"2024-09-10T16:00:15.612138+0000","mapping_epoch":36,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17268736,"num_objects":4216,"num_object_clones":0,"num_object_copies":8432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15194,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16385","reported_seq":16860,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.686906+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:15.686906+0000","last_peered":"2024-09-10T16:00:15.686906+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:15.686906+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17383448,"num_objects":4245,"num_object_clones":0,"num_object_copies":8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14653,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":616}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16411","reported_seq":16496,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.645879+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:15.645879+0000","last_peered":"2024-09-10T16:00:15.645879+0000","last_clean":"2024-09-10T16:00:15.645879+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:15.645879+0000","last_undegraded":"2024-09-10T16:00:15.645879+0000","last_fullsized":"2024-09-10T16:00:15.645879+0000","mapping_epoch":36,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494016,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16411,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16191","reported_seq":16322,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.571308+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:15.571308+0000","last_peered":"2024-09-10T16:00:15.571308+0000","last_clean":"2024-09-10T16:00:15.571308+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:15.571308+0000","last_undegraded":"2024-09-10T16:00:15.571308+0000","last_fullsized":"2024-09-10T16:00:15.571308+0000","mapping_epoch":35,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16957440,"num_objects":4140,"num_object_clones":0,"num_object_copies":8280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13262,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16617","reported_seq":24510,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.959229+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:15.959229+0000","last_peered":"2024-09-10T16:00:15.959229+0000","last_clean":"2024-09-10T16:00:15.959229+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:15.959229+0000","last_undegraded":"2024-09-10T16:00:15.959229+0000","last_fullsized":"2024-09-10T16:00:15.959229+0000","mapping_epoch":47,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17408000,"num_objects":4250,"num_object_clones":0,"num_object_copies":8500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13335,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16175","reported_seq":16941,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.562344+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:15.562344+0000","last_peered":"2024-09-10T16:00:15.562344+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:15.562344+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16838656,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12825,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":587}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16647","reported_seq":16784,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.546825+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:15.546825+0000","last_peered":"2024-09-10T16:00:15.546825+0000","last_clean":"2024-09-10T16:00:15.546825+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:15.546825+0000","last_undegraded":"2024-09-10T16:00:15.546825+0000","last_fullsized":"2024-09-10T16:00:15.546825+0000","mapping_epoch":35,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17477632,"num_objects":4267,"num_object_clones":0,"num_object_copies":8534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13406,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16599","reported_seq":24732,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:16.025897+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:16.025897+0000","last_peered":"2024-09-10T16:00:16.025897+0000","last_clean":"2024-09-10T16:00:16.025897+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:16.025897+0000","last_undegraded":"2024-09-10T16:00:16.025897+0000","last_fullsized":"2024-09-10T16:00:16.025897+0000","mapping_epoch":45,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14576,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16348","reported_seq":17168,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.443028+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:15.443028+0000","last_peered":"2024-09-10T16:00:15.443028+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:15.443028+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17326080,"num_objects":4230,"num_object_clones":0,"num_object_copies":8460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13212,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16513","reported_seq":23034,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.691979+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:15.691979+0000","last_peered":"2024-09-10T16:00:15.691979+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:15.691979+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17498112,"num_objects":4272,"num_object_clones":0,"num_object_copies":8544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14487,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3195,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3633}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16258","reported_seq":16676,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.637894+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:15.637894+0000","last_peered":"2024-09-10T16:00:15.637894+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:15.637894+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17195008,"num_objects":4198,"num_object_clones":0,"num_object_copies":8396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14127,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":512}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16596","reported_seq":17439,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.607765+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:15.607765+0000","last_peered":"2024-09-10T16:00:15.607765+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:15.607765+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17440768,"num_objects":4258,"num_object_clones":0,"num_object_copies":8516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13788,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":780}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16525","reported_seq":17273,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.554482+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:15.554482+0000","last_peered":"2024-09-10T16:00:15.554482+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:15.554482+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17174528,"num_objects":4193,"num_object_clones":0,"num_object_copies":8386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12943,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16473","reported_seq":17298,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.606000+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:15.606000+0000","last_peered":"2024-09-10T16:00:15.606000+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:15.606000+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17584128,"num_objects":4293,"num_object_clones":0,"num_object_copies":8586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13079,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16804","reported_seq":17566,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.567797+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:15.567797+0000","last_peered":"2024-09-10T16:00:15.567797+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:15.567797+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17588707,"num_objects":4295,"num_object_clones":0,"num_object_copies":8590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13623,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":601}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":276771323,"num_objects":67573,"num_object_clones":0,"num_object_copies":135146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67573,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225477,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13689,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":743989248,"data_stored":743912457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626520,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571584,"kb_used_data":202604,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93800256,"statfs":{"total":96636764160,"available":96051462144,"internally_reserved":0,"allocated":207466496,"data_stored":207128118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":377810606},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,3,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149802,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":702652,"kb_used_data":265644,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93669188,"statfs":{"total":96636764160,"available":95917248512,"internally_reserved":0,"allocated":272019456,"data_stored":271742958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":447475373},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576680,"kb_used_data":260428,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93795160,"statfs":{"total":96636764160,"available":96046243840,"internally_reserved":0,"allocated":266678272,"data_stored":266397702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":271106048,"data_stored":271102435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207032320,"data_stored":206966827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":265850880,"data_stored":265843195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:19.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:19.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'16556", 'reported_seq': 17360, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:15.480980+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T16:00:15.480980+0000', 'last_peered': '2024-09-10T16:00:15.480980+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:00:15.480980+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'6500", 'ondisk_log_start': "44'6500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17260544, 'num_objects': 4214, 'num_object_clones': 0, 'num_object_copies': 8428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4214, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16556, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 602}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:19.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:19.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'16385", 'reported_seq': 16860, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:15.686906+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:00:15.686906+0000', 'last_peered': '2024-09-10T16:00:15.686906+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:00:15.686906+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'6300", 'ondisk_log_start': "44'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17383448, 'num_objects': 4245, 'num_object_clones': 0, 'num_object_copies': 8490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4245, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14653, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 616}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:19.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:19.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'16175", 'reported_seq': 16941, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:15.562344+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:00:15.562344+0000', 'last_peered': '2024-09-10T16:00:15.562344+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:00:15.562344+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16838656, 'num_objects': 4111, 'num_object_clones': 0, 'num_object_copies': 8222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12825, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 587}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:19.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:19.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'16348", 'reported_seq': 17168, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:15.443028+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:00:15.443028+0000', 'last_peered': '2024-09-10T16:00:15.443028+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:00:15.443028+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'6300", 'ondisk_log_start': "44'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17326080, 'num_objects': 4230, 'num_object_clones': 0, 'num_object_copies': 8460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4230, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13212, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 711}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:19.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:19.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'16513", 'reported_seq': 23034, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:15.691979+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T16:00:15.691979+0000', 'last_peered': '2024-09-10T16:00:15.691979+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T16:00:15.691979+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'6500", 'ondisk_log_start': "44'6500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17498112, 'num_objects': 4272, 'num_object_clones': 0, 'num_object_copies': 8544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 639, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4272, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14487, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3195, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3633}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:19.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:19.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'16258", 'reported_seq': 16676, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:15.637894+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:00:15.637894+0000', 'last_peered': '2024-09-10T16:00:15.637894+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:00:15.637894+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'6200", 'ondisk_log_start': "44'6200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17195008, 'num_objects': 4198, 'num_object_clones': 0, 'num_object_copies': 8396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4198, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14127, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 512}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:19.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:19.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'16596", 'reported_seq': 17439, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:15.607765+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:00:15.607765+0000', 'last_peered': '2024-09-10T16:00:15.607765+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:00:15.607765+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'6500", 'ondisk_log_start': "44'6500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17440768, 'num_objects': 4258, 'num_object_clones': 0, 'num_object_copies': 8516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4258, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13788, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 780}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:19.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:19.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'16525", 'reported_seq': 17273, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:15.554482+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:00:15.554482+0000', 'last_peered': '2024-09-10T16:00:15.554482+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:00:15.554482+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'6500", 'ondisk_log_start': "44'6500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17174528, 'num_objects': 4193, 'num_object_clones': 0, 'num_object_copies': 8386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4193, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12943, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:19.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:19.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'16473", 'reported_seq': 17298, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:15.606000+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:00:15.606000+0000', 'last_peered': '2024-09-10T16:00:15.606000+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:00:15.606000+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'6400", 'ondisk_log_start': "44'6400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17584128, 'num_objects': 4293, 'num_object_clones': 0, 'num_object_copies': 8586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4293, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13079, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 757}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:19.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:19.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'16804", 'reported_seq': 17566, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:15.567797+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:00:15.567797+0000', 'last_peered': '2024-09-10T16:00:15.567797+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:00:15.567797+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'6800", 'ondisk_log_start': "44'6800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17588707, 'num_objects': 4295, 'num_object_clones': 0, 'num_object_copies': 8590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13623, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 601}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:19.915 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:20.581 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:20.581 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32959/135150 objects degraded (24.387%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":546,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":67575,"data_bytes":276771342,"bytes_used":1895337984,"bytes_avail":288014954496,"bytes_total":289910292480,"degraded_objects":32959,"degraded_total":135150,"degraded_ratio":0.24386977432482426,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":236,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":406},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T15:59:36.565732+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:00:20.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:20.604 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:21.193 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:21.195 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:21.224 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-10T16:00:20.590923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":276771342,"num_objects":67575,"num_object_clones":0,"num_object_copies":135150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67575,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225479,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13691,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160859,"ondisk_log_size":160859,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1850916,"kb_used_data":728676,"kb_used_omap":12,"kb_used_meta":1122163,"kb_avail":281264604,"statfs":{"total":289910292480,"available":288014954496,"internally_reserved":0,"allocated":746164224,"data_stored":745268778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1149094917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,12,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20013056,"num_objects":-4886,"num_object_clones":0,"num_object_copies":-9772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4886,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008866"},"pg_stats":[{"pgid":"2.f","version":"48'16556","reported_seq":17360,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.480980+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:15.480980+0000","last_peered":"2024-09-10T16:00:15.480980+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:15.480980+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17260544,"num_objects":4214,"num_object_clones":0,"num_object_copies":8428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16556,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":602}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'16858","reported_seq":16971,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.612138+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:15.612138+0000","last_peered":"2024-09-10T16:00:15.612138+0000","last_clean":"2024-09-10T16:00:15.612138+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:15.612138+0000","last_undegraded":"2024-09-10T16:00:15.612138+0000","last_fullsized":"2024-09-10T16:00:15.612138+0000","mapping_epoch":36,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17268736,"num_objects":4216,"num_object_clones":0,"num_object_copies":8432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15194,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16385","reported_seq":16860,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.686906+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:15.686906+0000","last_peered":"2024-09-10T16:00:15.686906+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:15.686906+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17383448,"num_objects":4245,"num_object_clones":0,"num_object_copies":8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14653,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":616}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16411","reported_seq":16496,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.645879+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:15.645879+0000","last_peered":"2024-09-10T16:00:15.645879+0000","last_clean":"2024-09-10T16:00:15.645879+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:15.645879+0000","last_undegraded":"2024-09-10T16:00:15.645879+0000","last_fullsized":"2024-09-10T16:00:15.645879+0000","mapping_epoch":36,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494016,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16411,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16191","reported_seq":16322,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.571308+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:15.571308+0000","last_peered":"2024-09-10T16:00:15.571308+0000","last_clean":"2024-09-10T16:00:15.571308+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:15.571308+0000","last_undegraded":"2024-09-10T16:00:15.571308+0000","last_fullsized":"2024-09-10T16:00:15.571308+0000","mapping_epoch":35,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16957440,"num_objects":4140,"num_object_clones":0,"num_object_copies":8280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13262,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16617","reported_seq":24510,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.959229+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:15.959229+0000","last_peered":"2024-09-10T16:00:15.959229+0000","last_clean":"2024-09-10T16:00:15.959229+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:15.959229+0000","last_undegraded":"2024-09-10T16:00:15.959229+0000","last_fullsized":"2024-09-10T16:00:15.959229+0000","mapping_epoch":47,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17408000,"num_objects":4250,"num_object_clones":0,"num_object_copies":8500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13335,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16175","reported_seq":16941,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.562344+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:15.562344+0000","last_peered":"2024-09-10T16:00:15.562344+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:15.562344+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16838656,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12825,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":587}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16647","reported_seq":16784,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:15.546825+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:15.546825+0000","last_peered":"2024-09-10T16:00:15.546825+0000","last_clean":"2024-09-10T16:00:15.546825+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:15.546825+0000","last_undegraded":"2024-09-10T16:00:15.546825+0000","last_fullsized":"2024-09-10T16:00:15.546825+0000","mapping_epoch":35,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17477632,"num_objects":4267,"num_object_clones":0,"num_object_copies":8534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13406,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16599","reported_seq":24732,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:16.025897+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:16.025897+0000","last_peered":"2024-09-10T16:00:16.025897+0000","last_clean":"2024-09-10T16:00:16.025897+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:16.025897+0000","last_undegraded":"2024-09-10T16:00:16.025897+0000","last_fullsized":"2024-09-10T16:00:16.025897+0000","mapping_epoch":45,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14576,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16348","reported_seq":17168,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.443028+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:15.443028+0000","last_peered":"2024-09-10T16:00:15.443028+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:15.443028+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17326080,"num_objects":4230,"num_object_clones":0,"num_object_copies":8460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13212,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16513","reported_seq":23034,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.691979+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:15.691979+0000","last_peered":"2024-09-10T16:00:15.691979+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:15.691979+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17498112,"num_objects":4272,"num_object_clones":0,"num_object_copies":8544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14487,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3195,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3633}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16258","reported_seq":16676,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.637894+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:15.637894+0000","last_peered":"2024-09-10T16:00:15.637894+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:15.637894+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17195008,"num_objects":4198,"num_object_clones":0,"num_object_copies":8396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14127,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":512}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16596","reported_seq":17439,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.607765+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:15.607765+0000","last_peered":"2024-09-10T16:00:15.607765+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:15.607765+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17440768,"num_objects":4258,"num_object_clones":0,"num_object_copies":8516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13788,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":780}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16525","reported_seq":17273,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.554482+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:15.554482+0000","last_peered":"2024-09-10T16:00:15.554482+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:15.554482+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17174528,"num_objects":4193,"num_object_clones":0,"num_object_copies":8386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12943,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16473","reported_seq":17298,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.606000+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:15.606000+0000","last_peered":"2024-09-10T16:00:15.606000+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:15.606000+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17584128,"num_objects":4293,"num_object_clones":0,"num_object_copies":8586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13079,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16804","reported_seq":17566,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:15.567797+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:15.567797+0000","last_peered":"2024-09-10T16:00:15.567797+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:15.567797+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17588707,"num_objects":4295,"num_object_clones":0,"num_object_copies":8590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13623,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":601}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":276771323,"num_objects":67573,"num_object_clones":0,"num_object_copies":135146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32959,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67573,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225477,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13689,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":743989248,"data_stored":743912457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626520,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571584,"kb_used_data":202604,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93800256,"statfs":{"total":96636764160,"available":96051462144,"internally_reserved":0,"allocated":207466496,"data_stored":207128118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":377810606},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,3,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149802,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":702652,"kb_used_data":265644,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93669188,"statfs":{"total":96636764160,"available":95917248512,"internally_reserved":0,"allocated":272019456,"data_stored":271742958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":447475373},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576680,"kb_used_data":260428,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93795160,"statfs":{"total":96636764160,"available":96046243840,"internally_reserved":0,"allocated":266678272,"data_stored":266397702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":271106048,"data_stored":271102435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207032320,"data_stored":206966827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":265850880,"data_stored":265843195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:24.226 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:24.781 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:24.783 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:24.802 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-10T16:00:24.591842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":266969614,"num_objects":65182,"num_object_clones":0,"num_object_copies":130364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227872,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13842,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815352,"kb_used_data":709756,"kb_used_omap":12,"kb_used_meta":1105523,"kb_avail":281300168,"statfs":{"total":289910292480,"available":288051372032,"internally_reserved":0,"allocated":726790144,"data_stored":725894698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1132055557},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,4,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19812352,"num_objects":-4837,"num_object_clones":0,"num_object_copies":-9674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4837,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003654"},"pg_stats":[{"pgid":"2.f","version":"48'16694","reported_seq":17498,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.587459+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:20.587459+0000","last_peered":"2024-09-10T16:00:20.587459+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:20.587459+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16695296,"num_objects":4076,"num_object_clones":0,"num_object_copies":8152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16694,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17009","reported_seq":17122,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:20.631596+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:20.631596+0000","last_peered":"2024-09-10T16:00:20.631596+0000","last_clean":"2024-09-10T16:00:20.631596+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:20.631596+0000","last_undegraded":"2024-09-10T16:00:20.631596+0000","last_fullsized":"2024-09-10T16:00:20.631596+0000","mapping_epoch":36,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16650240,"num_objects":4065,"num_object_clones":0,"num_object_copies":8130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15345,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16522","reported_seq":16997,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.636911+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:20.636911+0000","last_peered":"2024-09-10T16:00:20.636911+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:20.636911+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16822296,"num_objects":4108,"num_object_clones":0,"num_object_copies":8216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4108,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14790,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":479}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16559","reported_seq":16644,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:20.678760+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:20.678760+0000","last_peered":"2024-09-10T16:00:20.678760+0000","last_clean":"2024-09-10T16:00:20.678760+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:20.678760+0000","last_undegraded":"2024-09-10T16:00:20.678760+0000","last_fullsized":"2024-09-10T16:00:20.678760+0000","mapping_epoch":36,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16887808,"num_objects":4123,"num_object_clones":0,"num_object_copies":8246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16559,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16343","reported_seq":16475,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:20.555594+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:20.555594+0000","last_peered":"2024-09-10T16:00:20.555594+0000","last_clean":"2024-09-10T16:00:20.555594+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:20.555594+0000","last_undegraded":"2024-09-10T16:00:20.555594+0000","last_fullsized":"2024-09-10T16:00:20.555594+0000","mapping_epoch":35,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16334848,"num_objects":3988,"num_object_clones":0,"num_object_copies":7976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13414,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16780","reported_seq":24673,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:21.039095+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:21.039095+0000","last_peered":"2024-09-10T16:00:21.039095+0000","last_clean":"2024-09-10T16:00:21.039095+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:21.039095+0000","last_undegraded":"2024-09-10T16:00:21.039095+0000","last_fullsized":"2024-09-10T16:00:21.039095+0000","mapping_epoch":47,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16740352,"num_objects":4087,"num_object_clones":0,"num_object_copies":8174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13498,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16339","reported_seq":17105,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.584168+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:20.584168+0000","last_peered":"2024-09-10T16:00:20.584168+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:20.584168+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16166912,"num_objects":3947,"num_object_clones":0,"num_object_copies":7894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12989,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":423}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16794","reported_seq":16931,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:20.624412+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:20.624412+0000","last_peered":"2024-09-10T16:00:20.624412+0000","last_clean":"2024-09-10T16:00:20.624412+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:20.624412+0000","last_undegraded":"2024-09-10T16:00:20.624412+0000","last_fullsized":"2024-09-10T16:00:20.624412+0000","mapping_epoch":35,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13553,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16731","reported_seq":24864,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:20.986162+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:20.986162+0000","last_peered":"2024-09-10T16:00:20.986162+0000","last_clean":"2024-09-10T16:00:20.986162+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:20.986162+0000","last_undegraded":"2024-09-10T16:00:20.986162+0000","last_fullsized":"2024-09-10T16:00:20.986162+0000","mapping_epoch":45,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16334848,"num_objects":3988,"num_object_clones":0,"num_object_copies":7976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14708,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16523","reported_seq":17343,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.518045+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:20.518045+0000","last_peered":"2024-09-10T16:00:20.518045+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:20.518045+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16609280,"num_objects":4055,"num_object_clones":0,"num_object_copies":8110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13387,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":536}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16656","reported_seq":23477,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.675737+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:20.675737+0000","last_peered":"2024-09-10T16:00:20.675737+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:20.675737+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16912384,"num_objects":4129,"num_object_clones":0,"num_object_copies":8258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14630,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3641}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16398","reported_seq":16816,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.633165+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:20.633165+0000","last_peered":"2024-09-10T16:00:20.633165+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:20.633165+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16621568,"num_objects":4058,"num_object_clones":0,"num_object_copies":8116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14267,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":372}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16741","reported_seq":17581,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.574214+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:20.574214+0000","last_peered":"2024-09-10T16:00:20.574214+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:20.574214+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16846848,"num_objects":4113,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13933,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16657","reported_seq":17408,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.605795+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:20.605795+0000","last_peered":"2024-09-10T16:00:20.605795+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:20.605795+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16633856,"num_objects":4061,"num_object_clones":0,"num_object_copies":8122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13075,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16638","reported_seq":17465,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.575510+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:20.575510+0000","last_peered":"2024-09-10T16:00:20.575510+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:20.575510+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16908288,"num_objects":4128,"num_object_clones":0,"num_object_copies":8256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13244,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16965","reported_seq":17727,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.613836+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:20.613836+0000","last_peered":"2024-09-10T16:00:20.613836+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:20.613836+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16929251,"num_objects":4134,"num_object_clones":0,"num_object_copies":8268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13784,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":440}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":266969595,"num_objects":65180,"num_object_clones":0,"num_object_copies":130360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227870,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13840,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724480000,"data_stored":724403209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626521,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552152,"kb_used_data":195652,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93819688,"statfs":{"total":96636764160,"available":96071360512,"internally_reserved":0,"allocated":200347648,"data_stored":200009270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365031086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149803,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687056,"kb_used_data":258360,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93684784,"statfs":{"total":96636764160,"available":95933218816,"internally_reserved":0,"allocated":264560640,"data_stored":264284142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":438955693},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,8,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576144,"kb_used_data":255744,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93795696,"statfs":{"total":96636764160,"available":96046792704,"internally_reserved":0,"allocated":261881856,"data_stored":261601286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":263577600,"data_stored":263573987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199774208,"data_stored":199708715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":261128192,"data_stored":261120507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:24.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:25.346 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:25.347 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:25.376 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-10T16:00:24.591842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":266969614,"num_objects":65182,"num_object_clones":0,"num_object_copies":130364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227872,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13842,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815352,"kb_used_data":709756,"kb_used_omap":12,"kb_used_meta":1105523,"kb_avail":281300168,"statfs":{"total":289910292480,"available":288051372032,"internally_reserved":0,"allocated":726790144,"data_stored":725894698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1132055557},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,4,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19812352,"num_objects":-4837,"num_object_clones":0,"num_object_copies":-9674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4837,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003654"},"pg_stats":[{"pgid":"2.f","version":"48'16694","reported_seq":17498,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.587459+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:20.587459+0000","last_peered":"2024-09-10T16:00:20.587459+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:20.587459+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16695296,"num_objects":4076,"num_object_clones":0,"num_object_copies":8152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16694,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17009","reported_seq":17122,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:20.631596+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:20.631596+0000","last_peered":"2024-09-10T16:00:20.631596+0000","last_clean":"2024-09-10T16:00:20.631596+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:20.631596+0000","last_undegraded":"2024-09-10T16:00:20.631596+0000","last_fullsized":"2024-09-10T16:00:20.631596+0000","mapping_epoch":36,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16650240,"num_objects":4065,"num_object_clones":0,"num_object_copies":8130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15345,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16522","reported_seq":16997,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.636911+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:20.636911+0000","last_peered":"2024-09-10T16:00:20.636911+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:20.636911+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16822296,"num_objects":4108,"num_object_clones":0,"num_object_copies":8216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4108,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14790,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":479}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16559","reported_seq":16644,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:20.678760+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:20.678760+0000","last_peered":"2024-09-10T16:00:20.678760+0000","last_clean":"2024-09-10T16:00:20.678760+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:20.678760+0000","last_undegraded":"2024-09-10T16:00:20.678760+0000","last_fullsized":"2024-09-10T16:00:20.678760+0000","mapping_epoch":36,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16887808,"num_objects":4123,"num_object_clones":0,"num_object_copies":8246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16559,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16343","reported_seq":16475,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:20.555594+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:20.555594+0000","last_peered":"2024-09-10T16:00:20.555594+0000","last_clean":"2024-09-10T16:00:20.555594+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:20.555594+0000","last_undegraded":"2024-09-10T16:00:20.555594+0000","last_fullsized":"2024-09-10T16:00:20.555594+0000","mapping_epoch":35,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16334848,"num_objects":3988,"num_object_clones":0,"num_object_copies":7976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13414,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16780","reported_seq":24673,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:21.039095+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:21.039095+0000","last_peered":"2024-09-10T16:00:21.039095+0000","last_clean":"2024-09-10T16:00:21.039095+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:21.039095+0000","last_undegraded":"2024-09-10T16:00:21.039095+0000","last_fullsized":"2024-09-10T16:00:21.039095+0000","mapping_epoch":47,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16740352,"num_objects":4087,"num_object_clones":0,"num_object_copies":8174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13498,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16339","reported_seq":17105,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.584168+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:20.584168+0000","last_peered":"2024-09-10T16:00:20.584168+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:20.584168+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16166912,"num_objects":3947,"num_object_clones":0,"num_object_copies":7894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12989,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":423}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16794","reported_seq":16931,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:20.624412+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:20.624412+0000","last_peered":"2024-09-10T16:00:20.624412+0000","last_clean":"2024-09-10T16:00:20.624412+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:20.624412+0000","last_undegraded":"2024-09-10T16:00:20.624412+0000","last_fullsized":"2024-09-10T16:00:20.624412+0000","mapping_epoch":35,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13553,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16731","reported_seq":24864,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:20.986162+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:20.986162+0000","last_peered":"2024-09-10T16:00:20.986162+0000","last_clean":"2024-09-10T16:00:20.986162+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:20.986162+0000","last_undegraded":"2024-09-10T16:00:20.986162+0000","last_fullsized":"2024-09-10T16:00:20.986162+0000","mapping_epoch":45,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16334848,"num_objects":3988,"num_object_clones":0,"num_object_copies":7976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14708,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16523","reported_seq":17343,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.518045+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:20.518045+0000","last_peered":"2024-09-10T16:00:20.518045+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:20.518045+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16609280,"num_objects":4055,"num_object_clones":0,"num_object_copies":8110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13387,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":536}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16656","reported_seq":23477,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.675737+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:20.675737+0000","last_peered":"2024-09-10T16:00:20.675737+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:20.675737+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16912384,"num_objects":4129,"num_object_clones":0,"num_object_copies":8258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14630,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3641}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16398","reported_seq":16816,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.633165+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:20.633165+0000","last_peered":"2024-09-10T16:00:20.633165+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:20.633165+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16621568,"num_objects":4058,"num_object_clones":0,"num_object_copies":8116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14267,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":372}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16741","reported_seq":17581,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.574214+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:20.574214+0000","last_peered":"2024-09-10T16:00:20.574214+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:20.574214+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16846848,"num_objects":4113,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13933,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16657","reported_seq":17408,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.605795+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:20.605795+0000","last_peered":"2024-09-10T16:00:20.605795+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:20.605795+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16633856,"num_objects":4061,"num_object_clones":0,"num_object_copies":8122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13075,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16638","reported_seq":17465,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.575510+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:20.575510+0000","last_peered":"2024-09-10T16:00:20.575510+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:20.575510+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16908288,"num_objects":4128,"num_object_clones":0,"num_object_copies":8256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13244,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'16965","reported_seq":17727,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:20.613836+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:20.613836+0000","last_peered":"2024-09-10T16:00:20.613836+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:20.613836+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16929251,"num_objects":4134,"num_object_clones":0,"num_object_copies":8268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13784,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":440}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":266969595,"num_objects":65180,"num_object_clones":0,"num_object_copies":130360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227870,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13840,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724480000,"data_stored":724403209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626521,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552152,"kb_used_data":195652,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93819688,"statfs":{"total":96636764160,"available":96071360512,"internally_reserved":0,"allocated":200347648,"data_stored":200009270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365031086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149803,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687056,"kb_used_data":258360,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93684784,"statfs":{"total":96636764160,"available":95933218816,"internally_reserved":0,"allocated":264560640,"data_stored":264284142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":438955693},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,8,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576144,"kb_used_data":255744,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93795696,"statfs":{"total":96636764160,"available":96046792704,"internally_reserved":0,"allocated":261881856,"data_stored":261601286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":263577600,"data_stored":263573987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199774208,"data_stored":199708715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":261128192,"data_stored":261120507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:25.378 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:25.378 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'16694", 'reported_seq': 17498, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:20.587459+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T16:00:20.587459+0000', 'last_peered': '2024-09-10T16:00:20.587459+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:00:20.587459+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'6600", 'ondisk_log_start': "44'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16695296, 'num_objects': 4076, 'num_object_clones': 0, 'num_object_copies': 8152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4076, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16694, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 464}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:25.378 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:25.378 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'16522", 'reported_seq': 16997, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:20.636911+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:00:20.636911+0000', 'last_peered': '2024-09-10T16:00:20.636911+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:00:20.636911+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'6500", 'ondisk_log_start': "44'6500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16822296, 'num_objects': 4108, 'num_object_clones': 0, 'num_object_copies': 8216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4108, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14790, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 479}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:25.378 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:25.378 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'16339", 'reported_seq': 17105, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:20.584168+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:00:20.584168+0000', 'last_peered': '2024-09-10T16:00:20.584168+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:00:20.584168+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'6300", 'ondisk_log_start': "44'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16166912, 'num_objects': 3947, 'num_object_clones': 0, 'num_object_copies': 7894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3947, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12989, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 423}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:25.378 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:25.378 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'16523", 'reported_seq': 17343, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:20.518045+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:00:20.518045+0000', 'last_peered': '2024-09-10T16:00:20.518045+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:00:20.518045+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'6500", 'ondisk_log_start': "44'6500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16609280, 'num_objects': 4055, 'num_object_clones': 0, 'num_object_copies': 8110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13387, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 536}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:25.379 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:25.379 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'16656", 'reported_seq': 23477, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:20.675737+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T16:00:20.675737+0000', 'last_peered': '2024-09-10T16:00:20.675737+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T16:00:20.675737+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'6600", 'ondisk_log_start': "44'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16912384, 'num_objects': 4129, 'num_object_clones': 0, 'num_object_copies': 8258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 488, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4129, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14630, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3346, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3641}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:25.379 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:25.379 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'16398", 'reported_seq': 16816, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:20.633165+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:00:20.633165+0000', 'last_peered': '2024-09-10T16:00:20.633165+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:00:20.633165+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'6300", 'ondisk_log_start': "44'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16621568, 'num_objects': 4058, 'num_object_clones': 0, 'num_object_copies': 8116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4058, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14267, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 372}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:25.379 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:25.379 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'16741", 'reported_seq': 17581, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:20.574214+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:00:20.574214+0000', 'last_peered': '2024-09-10T16:00:20.574214+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:00:20.574214+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'6700", 'ondisk_log_start': "44'6700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16846848, 'num_objects': 4113, 'num_object_clones': 0, 'num_object_copies': 8226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13933, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 635}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:25.379 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:25.379 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'16657", 'reported_seq': 17408, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:20.605795+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:00:20.605795+0000', 'last_peered': '2024-09-10T16:00:20.605795+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:00:20.605795+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'6600", 'ondisk_log_start': "44'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16633856, 'num_objects': 4061, 'num_object_clones': 0, 'num_object_copies': 8122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4061, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13075, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:25.379 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:25.379 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'16638", 'reported_seq': 17465, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:20.575510+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:00:20.575510+0000', 'last_peered': '2024-09-10T16:00:20.575510+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:00:20.575510+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'6600", 'ondisk_log_start': "44'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16908288, 'num_objects': 4128, 'num_object_clones': 0, 'num_object_copies': 8256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4128, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13244, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 592}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:25.380 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:25.380 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'16965", 'reported_seq': 17727, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:20.613836+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:00:20.613836+0000', 'last_peered': '2024-09-10T16:00:20.613836+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:00:20.613836+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'6900", 'ondisk_log_start': "44'6900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16929251, 'num_objects': 4134, 'num_object_clones': 0, 'num_object_copies': 8268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4134, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13784, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 440}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:25.380 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:26.097 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:26.097 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32808/130364 objects degraded (25.166%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":552,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":65182,"data_bytes":266969614,"bytes_used":1858920448,"bytes_avail":288051372032,"bytes_total":289910292480,"degraded_objects":32808,"degraded_total":130364,"degraded_ratio":0.2516645699733055,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":262,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":402},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T15:59:36.565732+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:00:26.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:26.138 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:26.690 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:26.691 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:26.716 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-10T16:00:26.592354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257311246,"num_objects":62824,"num_object_clones":0,"num_object_copies":125648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32620,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62824,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230230,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14030,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161010,"ondisk_log_size":161010,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1831380,"kb_used_data":691084,"kb_used_omap":12,"kb_used_meta":1140211,"kb_avail":281284140,"statfs":{"total":289910292480,"available":288034959360,"internally_reserved":0,"allocated":707670016,"data_stored":706774570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1167576069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,6,7,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29470720,"num_objects":-7195,"num_object_clones":0,"num_object_copies":-14390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7195,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003304"},"pg_stats":[{"pgid":"2.f","version":"48'16849","reported_seq":17653,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.647659+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:25.647659+0000","last_peered":"2024-09-10T16:00:25.647659+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:25.647659+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16849,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":309}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17174","reported_seq":17286,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.649220+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:25.649220+0000","last_peered":"2024-09-10T16:00:25.649220+0000","last_clean":"2024-09-10T16:00:25.649220+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:25.649220+0000","last_undegraded":"2024-09-10T16:00:25.649220+0000","last_fullsized":"2024-09-10T16:00:25.649220+0000","mapping_epoch":36,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15974400,"num_objects":3900,"num_object_clones":0,"num_object_copies":7800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15510,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16686","reported_seq":17162,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.685867+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:25.685867+0000","last_peered":"2024-09-10T16:00:25.685867+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:25.685867+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16150552,"num_objects":3944,"num_object_clones":0,"num_object_copies":7888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14954,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":315}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16698","reported_seq":16782,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.699300+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:25.699300+0000","last_peered":"2024-09-10T16:00:25.699300+0000","last_clean":"2024-09-10T16:00:25.699300+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:25.699300+0000","last_undegraded":"2024-09-10T16:00:25.699300+0000","last_fullsized":"2024-09-10T16:00:25.699300+0000","mapping_epoch":36,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16318464,"num_objects":3984,"num_object_clones":0,"num_object_copies":7968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16698,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16470","reported_seq":16602,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.518977+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:25.518977+0000","last_peered":"2024-09-10T16:00:25.518977+0000","last_clean":"2024-09-10T16:00:25.518977+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:25.518977+0000","last_undegraded":"2024-09-10T16:00:25.518977+0000","last_fullsized":"2024-09-10T16:00:25.518977+0000","mapping_epoch":35,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15814656,"num_objects":3861,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13541,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16937","reported_seq":24830,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:26.102245+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:26.102245+0000","last_peered":"2024-09-10T16:00:26.102245+0000","last_clean":"2024-09-10T16:00:26.102245+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:26.102245+0000","last_undegraded":"2024-09-10T16:00:26.102245+0000","last_fullsized":"2024-09-10T16:00:26.102245+0000","mapping_epoch":47,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16097280,"num_objects":3930,"num_object_clones":0,"num_object_copies":7860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13655,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16485","reported_seq":17251,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.596159+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:25.596159+0000","last_peered":"2024-09-10T16:00:25.596159+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:25.596159+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15568896,"num_objects":3801,"num_object_clones":0,"num_object_copies":7602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13135,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16944","reported_seq":17081,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.634584+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:25.634584+0000","last_peered":"2024-09-10T16:00:25.634584+0000","last_clean":"2024-09-10T16:00:25.634584+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:25.634584+0000","last_undegraded":"2024-09-10T16:00:25.634584+0000","last_fullsized":"2024-09-10T16:00:25.634584+0000","mapping_epoch":35,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16261120,"num_objects":3970,"num_object_clones":0,"num_object_copies":7940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13703,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16879","reported_seq":25012,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:26.049968+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:26.049968+0000","last_peered":"2024-09-10T16:00:26.049968+0000","last_clean":"2024-09-10T16:00:26.049968+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:26.049968+0000","last_undegraded":"2024-09-10T16:00:26.049968+0000","last_fullsized":"2024-09-10T16:00:26.049968+0000","mapping_epoch":45,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15728640,"num_objects":3840,"num_object_clones":0,"num_object_copies":7680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14856,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16663","reported_seq":17483,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.478508+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:25.478508+0000","last_peered":"2024-09-10T16:00:25.478508+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:25.478508+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16035840,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13527,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":396}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16784","reported_seq":23983,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.723140+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:25.723140+0000","last_peered":"2024-09-10T16:00:25.723140+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:25.723140+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16388096,"num_objects":4001,"num_object_clones":0,"num_object_copies":8002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14758,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3534,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3701}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16531","reported_seq":16948,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.677731+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:25.677731+0000","last_peered":"2024-09-10T16:00:25.677731+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:25.677731+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16076800,"num_objects":3925,"num_object_clones":0,"num_object_copies":7850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14400,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":239}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16892","reported_seq":17735,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.615200+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:25.615200+0000","last_peered":"2024-09-10T16:00:25.615200+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:25.615200+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16228352,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14084,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":484}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16815","reported_seq":17566,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.650181+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:25.650181+0000","last_peered":"2024-09-10T16:00:25.650181+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:25.650181+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15986688,"num_objects":3903,"num_object_clones":0,"num_object_copies":7806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13233,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16781","reported_seq":17608,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.488302+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:25.488302+0000","last_peered":"2024-09-10T16:00:25.488302+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:25.488302+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16322560,"num_objects":3985,"num_object_clones":0,"num_object_copies":7970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13387,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":449}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'17119","reported_seq":17881,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.630555+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:25.630555+0000","last_peered":"2024-09-10T16:00:25.630555+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:25.630555+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16298467,"num_objects":3980,"num_object_clones":0,"num_object_copies":7960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13938,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":286}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":257311227,"num_objects":62822,"num_object_clones":0,"num_object_copies":125644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32620,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230228,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14028,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":705216512,"data_stored":705139721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161007,"ondisk_log_size":161007,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626522,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553424,"kb_used_data":188600,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93818416,"statfs":{"total":96636764160,"available":96070057984,"internally_reserved":0,"allocated":193126400,"data_stored":192788022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":373550766},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149804,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684204,"kb_used_data":251344,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93687636,"statfs":{"total":96636764160,"available":95936139264,"internally_reserved":0,"allocated":257376256,"data_stored":257099758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":443215533},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593752,"kb_used_data":251140,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93778088,"statfs":{"total":96636764160,"available":96028762112,"internally_reserved":0,"allocated":257167360,"data_stored":256886790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":256294912,"data_stored":256291299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":192512000,"data_stored":192446507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":256409600,"data_stored":256401915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:29.718 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:30.261 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:30.261 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:30.288 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-10T16:00:28.594022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257311246,"num_objects":62824,"num_object_clones":0,"num_object_copies":125648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32620,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62824,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230230,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14030,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161010,"ondisk_log_size":161010,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1831380,"kb_used_data":691084,"kb_used_omap":12,"kb_used_meta":1140211,"kb_avail":281284140,"statfs":{"total":289910292480,"available":288034959360,"internally_reserved":0,"allocated":707670016,"data_stored":706774570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1167576069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,6,7,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19460096,"num_objects":-4751,"num_object_clones":0,"num_object_copies":-9502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4751,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004382"},"pg_stats":[{"pgid":"2.f","version":"48'16849","reported_seq":17653,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.647659+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:25.647659+0000","last_peered":"2024-09-10T16:00:25.647659+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:25.647659+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16849,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":309}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17174","reported_seq":17286,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.649220+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:25.649220+0000","last_peered":"2024-09-10T16:00:25.649220+0000","last_clean":"2024-09-10T16:00:25.649220+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:25.649220+0000","last_undegraded":"2024-09-10T16:00:25.649220+0000","last_fullsized":"2024-09-10T16:00:25.649220+0000","mapping_epoch":36,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15974400,"num_objects":3900,"num_object_clones":0,"num_object_copies":7800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15510,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16686","reported_seq":17162,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.685867+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:25.685867+0000","last_peered":"2024-09-10T16:00:25.685867+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:25.685867+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16150552,"num_objects":3944,"num_object_clones":0,"num_object_copies":7888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14954,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":315}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16698","reported_seq":16782,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.699300+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:25.699300+0000","last_peered":"2024-09-10T16:00:25.699300+0000","last_clean":"2024-09-10T16:00:25.699300+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:25.699300+0000","last_undegraded":"2024-09-10T16:00:25.699300+0000","last_fullsized":"2024-09-10T16:00:25.699300+0000","mapping_epoch":36,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16318464,"num_objects":3984,"num_object_clones":0,"num_object_copies":7968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16698,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16470","reported_seq":16602,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.518977+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:25.518977+0000","last_peered":"2024-09-10T16:00:25.518977+0000","last_clean":"2024-09-10T16:00:25.518977+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:25.518977+0000","last_undegraded":"2024-09-10T16:00:25.518977+0000","last_fullsized":"2024-09-10T16:00:25.518977+0000","mapping_epoch":35,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15814656,"num_objects":3861,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13541,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16937","reported_seq":24830,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:26.102245+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:26.102245+0000","last_peered":"2024-09-10T16:00:26.102245+0000","last_clean":"2024-09-10T16:00:26.102245+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:26.102245+0000","last_undegraded":"2024-09-10T16:00:26.102245+0000","last_fullsized":"2024-09-10T16:00:26.102245+0000","mapping_epoch":47,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16097280,"num_objects":3930,"num_object_clones":0,"num_object_copies":7860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13655,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16485","reported_seq":17251,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.596159+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:25.596159+0000","last_peered":"2024-09-10T16:00:25.596159+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:25.596159+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15568896,"num_objects":3801,"num_object_clones":0,"num_object_copies":7602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13135,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16944","reported_seq":17081,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.634584+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:25.634584+0000","last_peered":"2024-09-10T16:00:25.634584+0000","last_clean":"2024-09-10T16:00:25.634584+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:25.634584+0000","last_undegraded":"2024-09-10T16:00:25.634584+0000","last_fullsized":"2024-09-10T16:00:25.634584+0000","mapping_epoch":35,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16261120,"num_objects":3970,"num_object_clones":0,"num_object_copies":7940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13703,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16879","reported_seq":25012,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:26.049968+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:26.049968+0000","last_peered":"2024-09-10T16:00:26.049968+0000","last_clean":"2024-09-10T16:00:26.049968+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:26.049968+0000","last_undegraded":"2024-09-10T16:00:26.049968+0000","last_fullsized":"2024-09-10T16:00:26.049968+0000","mapping_epoch":45,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15728640,"num_objects":3840,"num_object_clones":0,"num_object_copies":7680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14856,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16663","reported_seq":17483,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.478508+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:25.478508+0000","last_peered":"2024-09-10T16:00:25.478508+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:25.478508+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16035840,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13527,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":396}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16784","reported_seq":23983,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.723140+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:25.723140+0000","last_peered":"2024-09-10T16:00:25.723140+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:25.723140+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16388096,"num_objects":4001,"num_object_clones":0,"num_object_copies":8002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14758,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3534,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3701}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16531","reported_seq":16948,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.677731+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:25.677731+0000","last_peered":"2024-09-10T16:00:25.677731+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:25.677731+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16076800,"num_objects":3925,"num_object_clones":0,"num_object_copies":7850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14400,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":239}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16892","reported_seq":17735,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.615200+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:25.615200+0000","last_peered":"2024-09-10T16:00:25.615200+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:25.615200+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16228352,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14084,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":484}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16815","reported_seq":17566,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.650181+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:25.650181+0000","last_peered":"2024-09-10T16:00:25.650181+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:25.650181+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15986688,"num_objects":3903,"num_object_clones":0,"num_object_copies":7806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13233,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16781","reported_seq":17608,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.488302+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:25.488302+0000","last_peered":"2024-09-10T16:00:25.488302+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:25.488302+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16322560,"num_objects":3985,"num_object_clones":0,"num_object_copies":7970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13387,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":449}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'17119","reported_seq":17881,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.630555+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:25.630555+0000","last_peered":"2024-09-10T16:00:25.630555+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:25.630555+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16298467,"num_objects":3980,"num_object_clones":0,"num_object_copies":7960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13938,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":286}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":257311227,"num_objects":62822,"num_object_clones":0,"num_object_copies":125644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32620,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230228,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14028,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":705216512,"data_stored":705139721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161007,"ondisk_log_size":161007,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626522,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553424,"kb_used_data":188600,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93818416,"statfs":{"total":96636764160,"available":96070057984,"internally_reserved":0,"allocated":193126400,"data_stored":192788022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":373550766},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149804,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684204,"kb_used_data":251344,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93687636,"statfs":{"total":96636764160,"available":95936139264,"internally_reserved":0,"allocated":257376256,"data_stored":257099758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":443215533},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593752,"kb_used_data":251140,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93778088,"statfs":{"total":96636764160,"available":96028762112,"internally_reserved":0,"allocated":257167360,"data_stored":256886790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":256294912,"data_stored":256291299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":192512000,"data_stored":192446507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":256409600,"data_stored":256401915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:30.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:30.844 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:30.844 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:30.876 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-10T16:00:30.594742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257311246,"num_objects":62824,"num_object_clones":0,"num_object_copies":125648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32620,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62824,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230230,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14030,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161010,"ondisk_log_size":161010,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1831380,"kb_used_data":691084,"kb_used_omap":12,"kb_used_meta":1140211,"kb_avail":281284140,"statfs":{"total":289910292480,"available":288034959360,"internally_reserved":0,"allocated":707670016,"data_stored":706774570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1167576069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,6,7,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19460096,"num_objects":-4751,"num_object_clones":0,"num_object_copies":-9502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4751,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004400"},"pg_stats":[{"pgid":"2.f","version":"48'16849","reported_seq":17653,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.647659+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:25.647659+0000","last_peered":"2024-09-10T16:00:25.647659+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:25.647659+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16849,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":309}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17174","reported_seq":17286,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.649220+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:25.649220+0000","last_peered":"2024-09-10T16:00:25.649220+0000","last_clean":"2024-09-10T16:00:25.649220+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:25.649220+0000","last_undegraded":"2024-09-10T16:00:25.649220+0000","last_fullsized":"2024-09-10T16:00:25.649220+0000","mapping_epoch":36,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15974400,"num_objects":3900,"num_object_clones":0,"num_object_copies":7800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15510,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16686","reported_seq":17162,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.685867+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:25.685867+0000","last_peered":"2024-09-10T16:00:25.685867+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:25.685867+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16150552,"num_objects":3944,"num_object_clones":0,"num_object_copies":7888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14954,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":315}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16698","reported_seq":16782,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.699300+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:25.699300+0000","last_peered":"2024-09-10T16:00:25.699300+0000","last_clean":"2024-09-10T16:00:25.699300+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:25.699300+0000","last_undegraded":"2024-09-10T16:00:25.699300+0000","last_fullsized":"2024-09-10T16:00:25.699300+0000","mapping_epoch":36,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16318464,"num_objects":3984,"num_object_clones":0,"num_object_copies":7968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16698,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16470","reported_seq":16602,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.518977+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:25.518977+0000","last_peered":"2024-09-10T16:00:25.518977+0000","last_clean":"2024-09-10T16:00:25.518977+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:25.518977+0000","last_undegraded":"2024-09-10T16:00:25.518977+0000","last_fullsized":"2024-09-10T16:00:25.518977+0000","mapping_epoch":35,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15814656,"num_objects":3861,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13541,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16937","reported_seq":24830,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:26.102245+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:26.102245+0000","last_peered":"2024-09-10T16:00:26.102245+0000","last_clean":"2024-09-10T16:00:26.102245+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:26.102245+0000","last_undegraded":"2024-09-10T16:00:26.102245+0000","last_fullsized":"2024-09-10T16:00:26.102245+0000","mapping_epoch":47,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16097280,"num_objects":3930,"num_object_clones":0,"num_object_copies":7860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13655,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16485","reported_seq":17251,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.596159+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:25.596159+0000","last_peered":"2024-09-10T16:00:25.596159+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:25.596159+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15568896,"num_objects":3801,"num_object_clones":0,"num_object_copies":7602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13135,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16944","reported_seq":17081,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.634584+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:25.634584+0000","last_peered":"2024-09-10T16:00:25.634584+0000","last_clean":"2024-09-10T16:00:25.634584+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:25.634584+0000","last_undegraded":"2024-09-10T16:00:25.634584+0000","last_fullsized":"2024-09-10T16:00:25.634584+0000","mapping_epoch":35,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16261120,"num_objects":3970,"num_object_clones":0,"num_object_copies":7940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13703,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16879","reported_seq":25012,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:26.049968+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:26.049968+0000","last_peered":"2024-09-10T16:00:26.049968+0000","last_clean":"2024-09-10T16:00:26.049968+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:26.049968+0000","last_undegraded":"2024-09-10T16:00:26.049968+0000","last_fullsized":"2024-09-10T16:00:26.049968+0000","mapping_epoch":45,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15728640,"num_objects":3840,"num_object_clones":0,"num_object_copies":7680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14856,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16663","reported_seq":17483,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.478508+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:25.478508+0000","last_peered":"2024-09-10T16:00:25.478508+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:25.478508+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16035840,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13527,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":396}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16784","reported_seq":23983,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.723140+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:25.723140+0000","last_peered":"2024-09-10T16:00:25.723140+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:25.723140+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16388096,"num_objects":4001,"num_object_clones":0,"num_object_copies":8002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14758,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3534,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3701}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16531","reported_seq":16948,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.677731+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:25.677731+0000","last_peered":"2024-09-10T16:00:25.677731+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:25.677731+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16076800,"num_objects":3925,"num_object_clones":0,"num_object_copies":7850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14400,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":239}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16892","reported_seq":17735,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.615200+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:25.615200+0000","last_peered":"2024-09-10T16:00:25.615200+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:25.615200+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16228352,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14084,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":484}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16815","reported_seq":17566,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.650181+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:25.650181+0000","last_peered":"2024-09-10T16:00:25.650181+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:25.650181+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15986688,"num_objects":3903,"num_object_clones":0,"num_object_copies":7806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13233,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16781","reported_seq":17608,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.488302+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:25.488302+0000","last_peered":"2024-09-10T16:00:25.488302+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:25.488302+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16322560,"num_objects":3985,"num_object_clones":0,"num_object_copies":7970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13387,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":449}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'17119","reported_seq":17881,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.630555+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:25.630555+0000","last_peered":"2024-09-10T16:00:25.630555+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:25.630555+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16298467,"num_objects":3980,"num_object_clones":0,"num_object_copies":7960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13938,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":286}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":257311227,"num_objects":62822,"num_object_clones":0,"num_object_copies":125644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32620,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230228,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14028,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":705216512,"data_stored":705139721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161007,"ondisk_log_size":161007,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626522,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553424,"kb_used_data":188600,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93818416,"statfs":{"total":96636764160,"available":96070057984,"internally_reserved":0,"allocated":193126400,"data_stored":192788022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":373550766},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149804,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684204,"kb_used_data":251344,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93687636,"statfs":{"total":96636764160,"available":95936139264,"internally_reserved":0,"allocated":257376256,"data_stored":257099758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":443215533},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593752,"kb_used_data":251140,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93778088,"statfs":{"total":96636764160,"available":96028762112,"internally_reserved":0,"allocated":257167360,"data_stored":256886790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":256294912,"data_stored":256291299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":192512000,"data_stored":192446507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":256409600,"data_stored":256401915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:30.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:30.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'16849", 'reported_seq': 17653, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:25.647659+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T16:00:25.647659+0000', 'last_peered': '2024-09-10T16:00:25.647659+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:00:25.647659+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'6800", 'ondisk_log_start': "44'6800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16060416, 'num_objects': 3921, 'num_object_clones': 0, 'num_object_copies': 7842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3921, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16849, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 309}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:30.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:30.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'16686", 'reported_seq': 17162, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:25.685867+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:00:25.685867+0000', 'last_peered': '2024-09-10T16:00:25.685867+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:00:25.685867+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'6600", 'ondisk_log_start': "44'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16150552, 'num_objects': 3944, 'num_object_clones': 0, 'num_object_copies': 7888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3944, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14954, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 315}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:30.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:30.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'16485", 'reported_seq': 17251, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:25.596159+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:00:25.596159+0000', 'last_peered': '2024-09-10T16:00:25.596159+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:00:25.596159+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'6400", 'ondisk_log_start': "44'6400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15568896, 'num_objects': 3801, 'num_object_clones': 0, 'num_object_copies': 7602, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3801, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13135, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 277}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:30.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:30.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'16663", 'reported_seq': 17483, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:25.478508+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:00:25.478508+0000', 'last_peered': '2024-09-10T16:00:25.478508+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:00:25.478508+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'6600", 'ondisk_log_start': "44'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16035840, 'num_objects': 3915, 'num_object_clones': 0, 'num_object_copies': 7830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3915, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13527, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 396}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:30.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:30.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'16784", 'reported_seq': 23983, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:25.723140+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T16:00:25.723140+0000', 'last_peered': '2024-09-10T16:00:25.723140+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T16:00:25.723140+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'6700", 'ondisk_log_start': "44'6700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16388096, 'num_objects': 4001, 'num_object_clones': 0, 'num_object_copies': 8002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 300, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4001, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14758, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3534, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3701}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:30.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:30.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'16531", 'reported_seq': 16948, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:25.677731+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:00:25.677731+0000', 'last_peered': '2024-09-10T16:00:25.677731+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:00:25.677731+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'6500", 'ondisk_log_start': "44'6500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16076800, 'num_objects': 3925, 'num_object_clones': 0, 'num_object_copies': 7850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3925, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14400, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 239}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:30.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:30.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'16892", 'reported_seq': 17735, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:25.615200+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:00:25.615200+0000', 'last_peered': '2024-09-10T16:00:25.615200+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:00:25.615200+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'6800", 'ondisk_log_start': "44'6800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16228352, 'num_objects': 3962, 'num_object_clones': 0, 'num_object_copies': 7924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3962, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14084, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 484}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:30.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:30.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'16815", 'reported_seq': 17566, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:25.650181+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:00:25.650181+0000', 'last_peered': '2024-09-10T16:00:25.650181+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:00:25.650181+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'6800", 'ondisk_log_start': "44'6800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15986688, 'num_objects': 3903, 'num_object_clones': 0, 'num_object_copies': 7806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13233, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 277}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:30.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:30.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'16781", 'reported_seq': 17608, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:25.488302+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:00:25.488302+0000', 'last_peered': '2024-09-10T16:00:25.488302+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:00:25.488302+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'6700", 'ondisk_log_start': "44'6700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16322560, 'num_objects': 3985, 'num_object_clones': 0, 'num_object_copies': 7970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3985, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13387, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 449}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:30.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:30.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'17119", 'reported_seq': 17881, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:25.630555+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:00:25.630555+0000', 'last_peered': '2024-09-10T16:00:25.630555+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:00:25.630555+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'7100", 'ondisk_log_start': "44'7100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16298467, 'num_objects': 3980, 'num_object_clones': 0, 'num_object_copies': 7960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3980, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13938, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 286}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:30.880 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:31.567 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:31.567 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32620/125648 objects degraded (25.961%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":557,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":62824,"data_bytes":257311246,"bytes_used":1875333120,"bytes_avail":288034959360,"bytes_total":289910292480,"degraded_objects":32620,"degraded_total":125648,"degraded_ratio":0.2596141601935566,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":395},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T15:59:36.565732+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:00:31.597 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:31.598 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:32.137 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:32.137 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:32.158 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-10T16:00:30.594742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257311246,"num_objects":62824,"num_object_clones":0,"num_object_copies":125648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32620,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62824,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230230,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14030,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161010,"ondisk_log_size":161010,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1831380,"kb_used_data":691084,"kb_used_omap":12,"kb_used_meta":1140211,"kb_avail":281284140,"statfs":{"total":289910292480,"available":288034959360,"internally_reserved":0,"allocated":707670016,"data_stored":706774570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1167576069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,6,7,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19460096,"num_objects":-4751,"num_object_clones":0,"num_object_copies":-9502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4751,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004400"},"pg_stats":[{"pgid":"2.f","version":"48'16849","reported_seq":17653,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.647659+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:25.647659+0000","last_peered":"2024-09-10T16:00:25.647659+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:25.647659+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16849,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":309}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17174","reported_seq":17286,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.649220+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:25.649220+0000","last_peered":"2024-09-10T16:00:25.649220+0000","last_clean":"2024-09-10T16:00:25.649220+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:25.649220+0000","last_undegraded":"2024-09-10T16:00:25.649220+0000","last_fullsized":"2024-09-10T16:00:25.649220+0000","mapping_epoch":36,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15974400,"num_objects":3900,"num_object_clones":0,"num_object_copies":7800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15510,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16686","reported_seq":17162,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.685867+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:25.685867+0000","last_peered":"2024-09-10T16:00:25.685867+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:25.685867+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16150552,"num_objects":3944,"num_object_clones":0,"num_object_copies":7888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14954,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":315}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16698","reported_seq":16782,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.699300+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:25.699300+0000","last_peered":"2024-09-10T16:00:25.699300+0000","last_clean":"2024-09-10T16:00:25.699300+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:25.699300+0000","last_undegraded":"2024-09-10T16:00:25.699300+0000","last_fullsized":"2024-09-10T16:00:25.699300+0000","mapping_epoch":36,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16318464,"num_objects":3984,"num_object_clones":0,"num_object_copies":7968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16698,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16470","reported_seq":16602,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.518977+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:25.518977+0000","last_peered":"2024-09-10T16:00:25.518977+0000","last_clean":"2024-09-10T16:00:25.518977+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:25.518977+0000","last_undegraded":"2024-09-10T16:00:25.518977+0000","last_fullsized":"2024-09-10T16:00:25.518977+0000","mapping_epoch":35,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15814656,"num_objects":3861,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13541,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'16937","reported_seq":24830,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:26.102245+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:26.102245+0000","last_peered":"2024-09-10T16:00:26.102245+0000","last_clean":"2024-09-10T16:00:26.102245+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:26.102245+0000","last_undegraded":"2024-09-10T16:00:26.102245+0000","last_fullsized":"2024-09-10T16:00:26.102245+0000","mapping_epoch":47,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16097280,"num_objects":3930,"num_object_clones":0,"num_object_copies":7860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13655,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16485","reported_seq":17251,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.596159+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:25.596159+0000","last_peered":"2024-09-10T16:00:25.596159+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:25.596159+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15568896,"num_objects":3801,"num_object_clones":0,"num_object_copies":7602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13135,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'16944","reported_seq":17081,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:25.634584+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:25.634584+0000","last_peered":"2024-09-10T16:00:25.634584+0000","last_clean":"2024-09-10T16:00:25.634584+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:25.634584+0000","last_undegraded":"2024-09-10T16:00:25.634584+0000","last_fullsized":"2024-09-10T16:00:25.634584+0000","mapping_epoch":35,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16261120,"num_objects":3970,"num_object_clones":0,"num_object_copies":7940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13703,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'16879","reported_seq":25012,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:26.049968+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:26.049968+0000","last_peered":"2024-09-10T16:00:26.049968+0000","last_clean":"2024-09-10T16:00:26.049968+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:26.049968+0000","last_undegraded":"2024-09-10T16:00:26.049968+0000","last_fullsized":"2024-09-10T16:00:26.049968+0000","mapping_epoch":45,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15728640,"num_objects":3840,"num_object_clones":0,"num_object_copies":7680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14856,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16663","reported_seq":17483,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.478508+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:25.478508+0000","last_peered":"2024-09-10T16:00:25.478508+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:25.478508+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16035840,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13527,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":396}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":106,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.753192+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:12.753192+0000","last_peered":"2024-09-10T15:58:12.753192+0000","last_clean":"2024-09-10T15:58:12.753192+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:12.753192+0000","last_undegraded":"2024-09-10T15:58:12.753192+0000","last_fullsized":"2024-09-10T15:58:12.753192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16784","reported_seq":23983,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.723140+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:25.723140+0000","last_peered":"2024-09-10T16:00:25.723140+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:25.723140+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16388096,"num_objects":4001,"num_object_clones":0,"num_object_copies":8002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14758,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3534,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3701}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16531","reported_seq":16948,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.677731+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:25.677731+0000","last_peered":"2024-09-10T16:00:25.677731+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:25.677731+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6500","ondisk_log_start":"44'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16076800,"num_objects":3925,"num_object_clones":0,"num_object_copies":7850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14400,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":239}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'16892","reported_seq":17735,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.615200+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:25.615200+0000","last_peered":"2024-09-10T16:00:25.615200+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:25.615200+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16228352,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14084,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":484}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16815","reported_seq":17566,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.650181+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:25.650181+0000","last_peered":"2024-09-10T16:00:25.650181+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:25.650181+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15986688,"num_objects":3903,"num_object_clones":0,"num_object_copies":7806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13233,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16781","reported_seq":17608,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.488302+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:25.488302+0000","last_peered":"2024-09-10T16:00:25.488302+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:25.488302+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16322560,"num_objects":3985,"num_object_clones":0,"num_object_copies":7970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13387,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":449}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'17119","reported_seq":17881,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:25.630555+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:25.630555+0000","last_peered":"2024-09-10T16:00:25.630555+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:25.630555+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16298467,"num_objects":3980,"num_object_clones":0,"num_object_copies":7960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13938,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":286}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":257311227,"num_objects":62822,"num_object_clones":0,"num_object_copies":125644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32620,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230228,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14028,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":705216512,"data_stored":705139721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161007,"ondisk_log_size":161007,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626522,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553424,"kb_used_data":188600,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93818416,"statfs":{"total":96636764160,"available":96070057984,"internally_reserved":0,"allocated":193126400,"data_stored":192788022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":373550766},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149804,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684204,"kb_used_data":251344,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93687636,"statfs":{"total":96636764160,"available":95936139264,"internally_reserved":0,"allocated":257376256,"data_stored":257099758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":443215533},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593752,"kb_used_data":251140,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93778088,"statfs":{"total":96636764160,"available":96028762112,"internally_reserved":0,"allocated":257167360,"data_stored":256886790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":256294912,"data_stored":256291299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":192512000,"data_stored":192446507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":256409600,"data_stored":256401915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:35.161 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:35.756 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:35.757 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:35.782 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-10T16:00:34.596026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":247128590,"num_objects":60338,"num_object_clones":0,"num_object_copies":120676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60338,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232716,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14169,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1825440,"kb_used_data":671264,"kb_used_omap":12,"kb_used_meta":1154099,"kb_avail":281290080,"statfs":{"total":289910292480,"available":288041041920,"internally_reserved":0,"allocated":687374336,"data_stored":686478890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1181797381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,7,2,13,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19841024,"num_objects":-4844,"num_object_clones":0,"num_object_copies":-9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4844,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004666"},"pg_stats":[{"pgid":"2.f","version":"48'17012","reported_seq":17812,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.660361+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:30.660361+0000","last_peered":"2024-09-10T16:00:30.660361+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:30.660361+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15392768,"num_objects":3758,"num_object_clones":0,"num_object_copies":7516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17012,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17305","reported_seq":17418,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:30.674082+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:30.674082+0000","last_peered":"2024-09-10T16:00:30.674082+0000","last_clean":"2024-09-10T16:00:30.674082+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:30.674082+0000","last_undegraded":"2024-09-10T16:00:30.674082+0000","last_fullsized":"2024-09-10T16:00:30.674082+0000","mapping_epoch":36,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15437824,"num_objects":3769,"num_object_clones":0,"num_object_copies":7538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15641,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16837","reported_seq":17312,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.590906+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:30.590906+0000","last_peered":"2024-09-10T16:00:30.590906+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:30.590906+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15532056,"num_objects":3793,"num_object_clones":0,"num_object_copies":7586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3793,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15105,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":164}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16849","reported_seq":16934,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:30.655575+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:30.655575+0000","last_peered":"2024-09-10T16:00:30.655575+0000","last_clean":"2024-09-10T16:00:30.655575+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:30.655575+0000","last_undegraded":"2024-09-10T16:00:30.655575+0000","last_fullsized":"2024-09-10T16:00:30.655575+0000","mapping_epoch":36,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15699968,"num_objects":3833,"num_object_clones":0,"num_object_copies":7666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16849,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16636","reported_seq":16768,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:30.616151+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:30.616151+0000","last_peered":"2024-09-10T16:00:30.616151+0000","last_clean":"2024-09-10T16:00:30.616151+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:30.616151+0000","last_undegraded":"2024-09-10T16:00:30.616151+0000","last_fullsized":"2024-09-10T16:00:30.616151+0000","mapping_epoch":35,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15134720,"num_objects":3695,"num_object_clones":0,"num_object_copies":7390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13707,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'17100","reported_seq":24994,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:31.152966+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:31.152966+0000","last_peered":"2024-09-10T16:00:31.152966+0000","last_clean":"2024-09-10T16:00:31.152966+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:31.152966+0000","last_undegraded":"2024-09-10T16:00:31.152966+0000","last_fullsized":"2024-09-10T16:00:31.152966+0000","mapping_epoch":47,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15429632,"num_objects":3767,"num_object_clones":0,"num_object_copies":7534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13818,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16630","reported_seq":17396,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.555300+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:30.555300+0000","last_peered":"2024-09-10T16:00:30.555300+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:30.555300+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14974976,"num_objects":3656,"num_object_clones":0,"num_object_copies":7312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13280,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":132}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'17097","reported_seq":17234,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:30.602118+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:30.602118+0000","last_peered":"2024-09-10T16:00:30.602118+0000","last_clean":"2024-09-10T16:00:30.602118+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:30.602118+0000","last_undegraded":"2024-09-10T16:00:30.602118+0000","last_fullsized":"2024-09-10T16:00:30.602118+0000","mapping_epoch":35,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15634432,"num_objects":3817,"num_object_clones":0,"num_object_copies":7634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13856,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'17020","reported_seq":25153,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:31.020070+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:31.020070+0000","last_peered":"2024-09-10T16:00:31.020070+0000","last_clean":"2024-09-10T16:00:31.020070+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:31.020070+0000","last_undegraded":"2024-09-10T16:00:31.020070+0000","last_fullsized":"2024-09-10T16:00:31.020070+0000","mapping_epoch":45,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15151104,"num_objects":3699,"num_object_clones":0,"num_object_copies":7398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14997,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16834","reported_seq":17655,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.653801+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:30.653801+0000","last_peered":"2024-09-10T16:00:30.653801+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:30.653801+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15335424,"num_objects":3744,"num_object_clones":0,"num_object_copies":7488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13698,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":225}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":107,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.992521+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:24.992521+0000","last_peered":"2024-09-10T15:58:24.992521+0000","last_clean":"2024-09-10T15:58:24.992521+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:24.992521+0000","last_undegraded":"2024-09-10T15:58:24.992521+0000","last_fullsized":"2024-09-10T15:58:24.992521+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16945","reported_seq":24422,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.745196+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:30.745196+0000","last_peered":"2024-09-10T16:00:30.745196+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:30.745196+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15728640,"num_objects":3840,"num_object_clones":0,"num_object_copies":7680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14919,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3673,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3679}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16682","reported_seq":17100,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.692367+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:30.692367+0000","last_peered":"2024-09-10T16:00:30.692367+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:30.692367+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15458304,"num_objects":3774,"num_object_clones":0,"num_object_copies":7548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14551,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":88}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'17023","reported_seq":17866,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.635333+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:30.635333+0000","last_peered":"2024-09-10T16:00:30.635333+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:30.635333+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15691776,"num_objects":3831,"num_object_clones":0,"num_object_copies":7662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":353}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16993","reported_seq":17744,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.634046+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:30.634046+0000","last_peered":"2024-09-10T16:00:30.634046+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:30.634046+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15257600,"num_objects":3725,"num_object_clones":0,"num_object_copies":7450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13411,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16962","reported_seq":17789,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.647189+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:30.647189+0000","last_peered":"2024-09-10T16:00:30.647189+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:30.647189+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15581184,"num_objects":3804,"num_object_clones":0,"num_object_copies":7608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13568,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'17268","reported_seq":18030,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.632547+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:30.632547+0000","last_peered":"2024-09-10T16:00:30.632547+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:30.632547+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15688163,"num_objects":3831,"num_object_clones":0,"num_object_copies":7662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14087,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":247128571,"num_objects":60336,"num_object_clones":0,"num_object_copies":120672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60336,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232714,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14167,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":684806144,"data_stored":684729353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626523,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551652,"kb_used_data":181260,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93820188,"statfs":{"total":96636764160,"available":96071872512,"internally_reserved":0,"allocated":185610240,"data_stored":185271862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":379252398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149805,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680748,"kb_used_data":243732,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93691092,"statfs":{"total":96636764160,"available":95939678208,"internally_reserved":0,"allocated":249581568,"data_stored":249305070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":447475373},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593040,"kb_used_data":246272,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93778800,"statfs":{"total":96636764160,"available":96029491200,"internally_reserved":0,"allocated":252182528,"data_stored":251901958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":248602624,"data_stored":248599011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":184741888,"data_stored":184676395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":251461632,"data_stored":251453947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:35.783 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:36.314 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:36.317 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:36.343 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-10T16:00:34.596026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":247128590,"num_objects":60338,"num_object_clones":0,"num_object_copies":120676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60338,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232716,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14169,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1825440,"kb_used_data":671264,"kb_used_omap":12,"kb_used_meta":1154099,"kb_avail":281290080,"statfs":{"total":289910292480,"available":288041041920,"internally_reserved":0,"allocated":687374336,"data_stored":686478890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1181797381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,7,2,13,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19841024,"num_objects":-4844,"num_object_clones":0,"num_object_copies":-9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4844,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004666"},"pg_stats":[{"pgid":"2.f","version":"48'17012","reported_seq":17812,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.660361+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:30.660361+0000","last_peered":"2024-09-10T16:00:30.660361+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:30.660361+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15392768,"num_objects":3758,"num_object_clones":0,"num_object_copies":7516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17012,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17305","reported_seq":17418,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:30.674082+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:30.674082+0000","last_peered":"2024-09-10T16:00:30.674082+0000","last_clean":"2024-09-10T16:00:30.674082+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:30.674082+0000","last_undegraded":"2024-09-10T16:00:30.674082+0000","last_fullsized":"2024-09-10T16:00:30.674082+0000","mapping_epoch":36,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15437824,"num_objects":3769,"num_object_clones":0,"num_object_copies":7538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15641,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16837","reported_seq":17312,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.590906+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:30.590906+0000","last_peered":"2024-09-10T16:00:30.590906+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:30.590906+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15532056,"num_objects":3793,"num_object_clones":0,"num_object_copies":7586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3793,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15105,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":164}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'16849","reported_seq":16934,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:30.655575+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:30.655575+0000","last_peered":"2024-09-10T16:00:30.655575+0000","last_clean":"2024-09-10T16:00:30.655575+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:30.655575+0000","last_undegraded":"2024-09-10T16:00:30.655575+0000","last_fullsized":"2024-09-10T16:00:30.655575+0000","mapping_epoch":36,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15699968,"num_objects":3833,"num_object_clones":0,"num_object_copies":7666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16849,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16636","reported_seq":16768,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:30.616151+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:30.616151+0000","last_peered":"2024-09-10T16:00:30.616151+0000","last_clean":"2024-09-10T16:00:30.616151+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:30.616151+0000","last_undegraded":"2024-09-10T16:00:30.616151+0000","last_fullsized":"2024-09-10T16:00:30.616151+0000","mapping_epoch":35,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15134720,"num_objects":3695,"num_object_clones":0,"num_object_copies":7390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13707,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'17100","reported_seq":24994,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:31.152966+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:31.152966+0000","last_peered":"2024-09-10T16:00:31.152966+0000","last_clean":"2024-09-10T16:00:31.152966+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:31.152966+0000","last_undegraded":"2024-09-10T16:00:31.152966+0000","last_fullsized":"2024-09-10T16:00:31.152966+0000","mapping_epoch":47,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15429632,"num_objects":3767,"num_object_clones":0,"num_object_copies":7534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13818,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16630","reported_seq":17396,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.555300+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:30.555300+0000","last_peered":"2024-09-10T16:00:30.555300+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:30.555300+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14974976,"num_objects":3656,"num_object_clones":0,"num_object_copies":7312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13280,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":132}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'17097","reported_seq":17234,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:30.602118+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:30.602118+0000","last_peered":"2024-09-10T16:00:30.602118+0000","last_clean":"2024-09-10T16:00:30.602118+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:30.602118+0000","last_undegraded":"2024-09-10T16:00:30.602118+0000","last_fullsized":"2024-09-10T16:00:30.602118+0000","mapping_epoch":35,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15634432,"num_objects":3817,"num_object_clones":0,"num_object_copies":7634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13856,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'17020","reported_seq":25153,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:31.020070+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:31.020070+0000","last_peered":"2024-09-10T16:00:31.020070+0000","last_clean":"2024-09-10T16:00:31.020070+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:31.020070+0000","last_undegraded":"2024-09-10T16:00:31.020070+0000","last_fullsized":"2024-09-10T16:00:31.020070+0000","mapping_epoch":45,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15151104,"num_objects":3699,"num_object_clones":0,"num_object_copies":7398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14997,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16834","reported_seq":17655,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.653801+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:30.653801+0000","last_peered":"2024-09-10T16:00:30.653801+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:30.653801+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15335424,"num_objects":3744,"num_object_clones":0,"num_object_copies":7488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13698,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":225}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":107,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.992521+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:24.992521+0000","last_peered":"2024-09-10T15:58:24.992521+0000","last_clean":"2024-09-10T15:58:24.992521+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:24.992521+0000","last_undegraded":"2024-09-10T15:58:24.992521+0000","last_fullsized":"2024-09-10T15:58:24.992521+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'16945","reported_seq":24422,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.745196+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:30.745196+0000","last_peered":"2024-09-10T16:00:30.745196+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:30.745196+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15728640,"num_objects":3840,"num_object_clones":0,"num_object_copies":7680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14919,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3673,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3679}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16682","reported_seq":17100,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.692367+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:30.692367+0000","last_peered":"2024-09-10T16:00:30.692367+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:30.692367+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6600","ondisk_log_start":"44'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15458304,"num_objects":3774,"num_object_clones":0,"num_object_copies":7548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14551,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":88}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'17023","reported_seq":17866,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.635333+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:30.635333+0000","last_peered":"2024-09-10T16:00:30.635333+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:30.635333+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15691776,"num_objects":3831,"num_object_clones":0,"num_object_copies":7662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":353}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16993","reported_seq":17744,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.634046+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:30.634046+0000","last_peered":"2024-09-10T16:00:30.634046+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:30.634046+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15257600,"num_objects":3725,"num_object_clones":0,"num_object_copies":7450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13411,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'16962","reported_seq":17789,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.647189+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:30.647189+0000","last_peered":"2024-09-10T16:00:30.647189+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:30.647189+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15581184,"num_objects":3804,"num_object_clones":0,"num_object_copies":7608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13568,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'17268","reported_seq":18030,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:30.632547+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:30.632547+0000","last_peered":"2024-09-10T16:00:30.632547+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:30.632547+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15688163,"num_objects":3831,"num_object_clones":0,"num_object_copies":7662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14087,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":247128571,"num_objects":60336,"num_object_clones":0,"num_object_copies":120672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60336,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232714,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14167,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":684806144,"data_stored":684729353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626523,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551652,"kb_used_data":181260,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93820188,"statfs":{"total":96636764160,"available":96071872512,"internally_reserved":0,"allocated":185610240,"data_stored":185271862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":379252398},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149805,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680748,"kb_used_data":243732,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93691092,"statfs":{"total":96636764160,"available":95939678208,"internally_reserved":0,"allocated":249581568,"data_stored":249305070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":447475373},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593040,"kb_used_data":246272,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93778800,"statfs":{"total":96636764160,"available":96029491200,"internally_reserved":0,"allocated":252182528,"data_stored":251901958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":248602624,"data_stored":248599011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":184741888,"data_stored":184676395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":251461632,"data_stored":251453947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:36.344 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:36.344 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'17012", 'reported_seq': 17812, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:30.660361+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T16:00:30.660361+0000', 'last_peered': '2024-09-10T16:00:30.660361+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:00:30.660361+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'7000", 'ondisk_log_start': "44'7000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15392768, 'num_objects': 3758, 'num_object_clones': 0, 'num_object_copies': 7516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3758, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17012, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 146}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:36.344 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:36.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'16837", 'reported_seq': 17312, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:30.590906+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:00:30.590906+0000', 'last_peered': '2024-09-10T16:00:30.590906+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:00:30.590906+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'6800", 'ondisk_log_start': "44'6800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15532056, 'num_objects': 3793, 'num_object_clones': 0, 'num_object_copies': 7586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3793, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15105, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:36.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:36.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'16630", 'reported_seq': 17396, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:30.555300+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:00:30.555300+0000', 'last_peered': '2024-09-10T16:00:30.555300+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:00:30.555300+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'6600", 'ondisk_log_start': "44'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14974976, 'num_objects': 3656, 'num_object_clones': 0, 'num_object_copies': 7312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3656, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13280, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 132}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:36.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:36.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'16834", 'reported_seq': 17655, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:30.653801+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:00:30.653801+0000', 'last_peered': '2024-09-10T16:00:30.653801+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:00:30.653801+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'6800", 'ondisk_log_start': "44'6800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15335424, 'num_objects': 3744, 'num_object_clones': 0, 'num_object_copies': 7488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3744, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13698, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 225}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:36.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:36.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'16945", 'reported_seq': 24422, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:30.745196+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T16:00:30.745196+0000', 'last_peered': '2024-09-10T16:00:30.745196+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T16:00:30.745196+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'6900", 'ondisk_log_start': "44'6900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15728640, 'num_objects': 3840, 'num_object_clones': 0, 'num_object_copies': 7680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3840, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14919, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3673, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3679}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:36.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:36.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'16682", 'reported_seq': 17100, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:30.692367+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:00:30.692367+0000', 'last_peered': '2024-09-10T16:00:30.692367+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:00:30.692367+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'6600", 'ondisk_log_start': "44'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15458304, 'num_objects': 3774, 'num_object_clones': 0, 'num_object_copies': 7548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3774, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14551, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 88}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:36.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:36.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'17023", 'reported_seq': 17866, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:30.635333+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:00:30.635333+0000', 'last_peered': '2024-09-10T16:00:30.635333+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:00:30.635333+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'7000", 'ondisk_log_start': "44'7000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15691776, 'num_objects': 3831, 'num_object_clones': 0, 'num_object_copies': 7662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3831, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14215, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 353}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:36.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:36.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'16993", 'reported_seq': 17744, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:30.634046+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:00:30.634046+0000', 'last_peered': '2024-09-10T16:00:30.634046+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:00:30.634046+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'6900", 'ondisk_log_start': "44'6900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15257600, 'num_objects': 3725, 'num_object_clones': 0, 'num_object_copies': 7450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3725, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13411, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 99}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:36.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:36.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'16962", 'reported_seq': 17789, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:30.647189+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:00:30.647189+0000', 'last_peered': '2024-09-10T16:00:30.647189+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:00:30.647189+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'6900", 'ondisk_log_start': "44'6900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15581184, 'num_objects': 3804, 'num_object_clones': 0, 'num_object_copies': 7608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13568, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:36.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:36.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'17268", 'reported_seq': 18030, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:30.632547+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:00:30.632547+0000', 'last_peered': '2024-09-10T16:00:30.632547+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:00:30.632547+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'7200", 'ondisk_log_start': "44'7200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15688163, 'num_objects': 3831, 'num_object_clones': 0, 'num_object_copies': 7662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3831, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14087, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 137}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:36.348 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:37.050 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:37.050 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32360/115792 objects degraded (27.947%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":562,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":57896,"data_bytes":237126158,"bytes_used":1836417024,"bytes_avail":288073875456,"bytes_total":289910292480,"degraded_objects":32360,"degraded_total":115792,"degraded_ratio":0.27946662981898579,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":448,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":606},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T15:59:36.565732+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:00:37.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:37.076 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:37.655 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:37.657 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:37.682 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-10T16:00:36.596654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":237126158,"num_objects":57896,"num_object_clones":0,"num_object_copies":115792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32360,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57896,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235158,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14290,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160838,"ondisk_log_size":160838,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1793376,"kb_used_data":651684,"kb_used_omap":12,"kb_used_meta":1141619,"kb_avail":281322144,"statfs":{"total":289910292480,"available":288073875456,"internally_reserved":0,"allocated":667324416,"data_stored":666428970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1169017861},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,8,7,17],"upper_bound":64},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29843456,"num_objects":-7286,"num_object_clones":0,"num_object_copies":-14572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":448,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004812"},"pg_stats":[{"pgid":"2.f","version":"48'17139","reported_seq":17943,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.698800+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:35.698800+0000","last_peered":"2024-09-10T16:00:35.698800+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:35.698800+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14872576,"num_objects":3631,"num_object_clones":0,"num_object_copies":7262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17139,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":19}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17450","reported_seq":17563,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:35.713746+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:35.713746+0000","last_peered":"2024-09-10T16:00:35.713746+0000","last_clean":"2024-09-10T16:00:35.713746+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:35.713746+0000","last_undegraded":"2024-09-10T16:00:35.713746+0000","last_fullsized":"2024-09-10T16:00:35.713746+0000","mapping_epoch":36,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14843904,"num_objects":3624,"num_object_clones":0,"num_object_copies":7248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15786,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16992","reported_seq":17466,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.727513+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:35.727513+0000","last_peered":"2024-09-10T16:00:35.727513+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:35.727513+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14897176,"num_objects":3638,"num_object_clones":0,"num_object_copies":7276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3638,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15260,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'17025","reported_seq":17110,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:35.760984+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:35.760984+0000","last_peered":"2024-09-10T16:00:35.760984+0000","last_clean":"2024-09-10T16:00:35.760984+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:35.760984+0000","last_undegraded":"2024-09-10T16:00:35.760984+0000","last_fullsized":"2024-09-10T16:00:35.760984+0000","mapping_epoch":36,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14979072,"num_objects":3657,"num_object_clones":0,"num_object_copies":7314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17025,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16797","reported_seq":16929,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:35.627014+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:35.627014+0000","last_peered":"2024-09-10T16:00:35.627014+0000","last_clean":"2024-09-10T16:00:35.627014+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:35.627014+0000","last_undegraded":"2024-09-10T16:00:35.627014+0000","last_fullsized":"2024-09-10T16:00:35.627014+0000","mapping_epoch":35,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14475264,"num_objects":3534,"num_object_clones":0,"num_object_copies":7068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13868,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'17256","reported_seq":25150,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:36.154967+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:36.154967+0000","last_peered":"2024-09-10T16:00:36.154967+0000","last_clean":"2024-09-10T16:00:36.154967+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:36.154967+0000","last_undegraded":"2024-09-10T16:00:36.154967+0000","last_fullsized":"2024-09-10T16:00:36.154967+0000","mapping_epoch":47,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14790656,"num_objects":3611,"num_object_clones":0,"num_object_copies":7222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13974,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16766","reported_seq":17532,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.619964+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:35.619964+0000","last_peered":"2024-09-10T16:00:35.619964+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:35.619964+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14417920,"num_objects":3520,"num_object_clones":0,"num_object_copies":7040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13416,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'17259","reported_seq":17396,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:35.634088+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:35.634088+0000","last_peered":"2024-09-10T16:00:35.634088+0000","last_clean":"2024-09-10T16:00:35.634088+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:35.634088+0000","last_undegraded":"2024-09-10T16:00:35.634088+0000","last_fullsized":"2024-09-10T16:00:35.634088+0000","mapping_epoch":35,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14970880,"num_objects":3655,"num_object_clones":0,"num_object_copies":7310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14018,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'17152","reported_seq":25285,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:36.174344+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:36.174344+0000","last_peered":"2024-09-10T16:00:36.174344+0000","last_clean":"2024-09-10T16:00:36.174344+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:36.174344+0000","last_undegraded":"2024-09-10T16:00:36.174344+0000","last_fullsized":"2024-09-10T16:00:36.174344+0000","mapping_epoch":45,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14610432,"num_objects":3567,"num_object_clones":0,"num_object_copies":7134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15129,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16992","reported_seq":17811,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.648717+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:35.648717+0000","last_peered":"2024-09-10T16:00:35.648717+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:35.648717+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14688256,"num_objects":3586,"num_object_clones":0,"num_object_copies":7172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13856,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":67}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":107,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.992521+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:24.992521+0000","last_peered":"2024-09-10T15:58:24.992521+0000","last_clean":"2024-09-10T15:58:24.992521+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:24.992521+0000","last_undegraded":"2024-09-10T15:58:24.992521+0000","last_fullsized":"2024-09-10T15:58:24.992521+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'17115","reported_seq":24834,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.747377+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:35.747377+0000","last_peered":"2024-09-10T16:00:35.747377+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:35.747377+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15032320,"num_objects":3670,"num_object_clones":0,"num_object_copies":7340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15089,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3794,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3630}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16816","reported_seq":17234,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.701047+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:35.701047+0000","last_peered":"2024-09-10T16:00:35.701047+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:35.701047+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14909440,"num_objects":3640,"num_object_clones":0,"num_object_copies":7280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14685,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-46}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'17188","reported_seq":18031,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.663128+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:35.663128+0000","last_peered":"2024-09-10T16:00:35.663128+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:35.663128+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15015936,"num_objects":3666,"num_object_clones":0,"num_object_copies":7332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14380,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":188}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'17152","reported_seq":17903,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.676546+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:35.676546+0000","last_peered":"2024-09-10T16:00:35.676546+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:35.676546+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14606336,"num_objects":3566,"num_object_clones":0,"num_object_copies":7132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13570,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-60}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'17108","reported_seq":17935,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.616599+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:35.616599+0000","last_peered":"2024-09-10T16:00:35.616599+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:35.616599+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14983168,"num_objects":3658,"num_object_clones":0,"num_object_copies":7316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13714,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":122}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'17428","reported_seq":18188,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.681406+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:35.681406+0000","last_peered":"2024-09-10T16:00:35.681406+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:35.681406+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15032803,"num_objects":3671,"num_object_clones":0,"num_object_copies":7342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14247,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-23}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":237126139,"num_objects":57894,"num_object_clones":0,"num_object_copies":115788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32360,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57894,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235156,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14288,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":664780800,"data_stored":664704009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160835,"ondisk_log_size":160835,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626524,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552696,"kb_used_data":173984,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93819144,"statfs":{"total":96636764160,"available":96070803456,"internally_reserved":0,"allocated":178159616,"data_stored":177821238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":387772078},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,2,9],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149806,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":665056,"kb_used_data":236364,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93706784,"statfs":{"total":96636764160,"available":95955746816,"internally_reserved":0,"allocated":242036736,"data_stored":241760238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":438955693},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575624,"kb_used_data":241336,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93796216,"statfs":{"total":96636764160,"available":96047325184,"internally_reserved":0,"allocated":247128064,"data_stored":246847494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":241012736,"data_stored":241009123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":177360896,"data_stored":177295403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":246407168,"data_stored":246399483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:40.685 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:41.222 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:41.225 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:41.246 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-10T16:00:40.598274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":237126158,"num_objects":57896,"num_object_clones":0,"num_object_copies":115792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32360,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57896,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235158,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14290,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160838,"ondisk_log_size":160838,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1793376,"kb_used_data":651684,"kb_used_omap":12,"kb_used_meta":1141619,"kb_avail":281322144,"statfs":{"total":289910292480,"available":288073875456,"internally_reserved":0,"allocated":667324416,"data_stored":666428970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1169017861},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,8,7,17],"upper_bound":64},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10002432,"num_objects":-2442,"num_object_clones":0,"num_object_copies":-4884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2442,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002905"},"pg_stats":[{"pgid":"2.f","version":"48'17139","reported_seq":17943,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.698800+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:35.698800+0000","last_peered":"2024-09-10T16:00:35.698800+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:35.698800+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14872576,"num_objects":3631,"num_object_clones":0,"num_object_copies":7262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17139,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":19}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17450","reported_seq":17563,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:35.713746+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:35.713746+0000","last_peered":"2024-09-10T16:00:35.713746+0000","last_clean":"2024-09-10T16:00:35.713746+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:35.713746+0000","last_undegraded":"2024-09-10T16:00:35.713746+0000","last_fullsized":"2024-09-10T16:00:35.713746+0000","mapping_epoch":36,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14843904,"num_objects":3624,"num_object_clones":0,"num_object_copies":7248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15786,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16992","reported_seq":17466,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.727513+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:35.727513+0000","last_peered":"2024-09-10T16:00:35.727513+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:35.727513+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14897176,"num_objects":3638,"num_object_clones":0,"num_object_copies":7276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3638,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15260,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'17025","reported_seq":17110,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:35.760984+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:35.760984+0000","last_peered":"2024-09-10T16:00:35.760984+0000","last_clean":"2024-09-10T16:00:35.760984+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:35.760984+0000","last_undegraded":"2024-09-10T16:00:35.760984+0000","last_fullsized":"2024-09-10T16:00:35.760984+0000","mapping_epoch":36,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14979072,"num_objects":3657,"num_object_clones":0,"num_object_copies":7314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17025,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16797","reported_seq":16929,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:35.627014+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:35.627014+0000","last_peered":"2024-09-10T16:00:35.627014+0000","last_clean":"2024-09-10T16:00:35.627014+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:35.627014+0000","last_undegraded":"2024-09-10T16:00:35.627014+0000","last_fullsized":"2024-09-10T16:00:35.627014+0000","mapping_epoch":35,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14475264,"num_objects":3534,"num_object_clones":0,"num_object_copies":7068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13868,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'17256","reported_seq":25150,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:36.154967+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:36.154967+0000","last_peered":"2024-09-10T16:00:36.154967+0000","last_clean":"2024-09-10T16:00:36.154967+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:36.154967+0000","last_undegraded":"2024-09-10T16:00:36.154967+0000","last_fullsized":"2024-09-10T16:00:36.154967+0000","mapping_epoch":47,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14790656,"num_objects":3611,"num_object_clones":0,"num_object_copies":7222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13974,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16766","reported_seq":17532,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.619964+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:35.619964+0000","last_peered":"2024-09-10T16:00:35.619964+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:35.619964+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14417920,"num_objects":3520,"num_object_clones":0,"num_object_copies":7040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13416,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'17259","reported_seq":17396,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:35.634088+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:35.634088+0000","last_peered":"2024-09-10T16:00:35.634088+0000","last_clean":"2024-09-10T16:00:35.634088+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:35.634088+0000","last_undegraded":"2024-09-10T16:00:35.634088+0000","last_fullsized":"2024-09-10T16:00:35.634088+0000","mapping_epoch":35,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14970880,"num_objects":3655,"num_object_clones":0,"num_object_copies":7310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14018,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'17152","reported_seq":25285,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:36.174344+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:36.174344+0000","last_peered":"2024-09-10T16:00:36.174344+0000","last_clean":"2024-09-10T16:00:36.174344+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:36.174344+0000","last_undegraded":"2024-09-10T16:00:36.174344+0000","last_fullsized":"2024-09-10T16:00:36.174344+0000","mapping_epoch":45,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14610432,"num_objects":3567,"num_object_clones":0,"num_object_copies":7134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15129,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16992","reported_seq":17811,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.648717+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:35.648717+0000","last_peered":"2024-09-10T16:00:35.648717+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:35.648717+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14688256,"num_objects":3586,"num_object_clones":0,"num_object_copies":7172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13856,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":67}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":107,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.992521+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:24.992521+0000","last_peered":"2024-09-10T15:58:24.992521+0000","last_clean":"2024-09-10T15:58:24.992521+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:24.992521+0000","last_undegraded":"2024-09-10T15:58:24.992521+0000","last_fullsized":"2024-09-10T15:58:24.992521+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'17115","reported_seq":24834,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.747377+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:35.747377+0000","last_peered":"2024-09-10T16:00:35.747377+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:35.747377+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15032320,"num_objects":3670,"num_object_clones":0,"num_object_copies":7340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15089,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3794,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3630}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16816","reported_seq":17234,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.701047+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:35.701047+0000","last_peered":"2024-09-10T16:00:35.701047+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:35.701047+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14909440,"num_objects":3640,"num_object_clones":0,"num_object_copies":7280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14685,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-46}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'17188","reported_seq":18031,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.663128+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:35.663128+0000","last_peered":"2024-09-10T16:00:35.663128+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:35.663128+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15015936,"num_objects":3666,"num_object_clones":0,"num_object_copies":7332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14380,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":188}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'17152","reported_seq":17903,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.676546+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:35.676546+0000","last_peered":"2024-09-10T16:00:35.676546+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:35.676546+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14606336,"num_objects":3566,"num_object_clones":0,"num_object_copies":7132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13570,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-60}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'17108","reported_seq":17935,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.616599+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:35.616599+0000","last_peered":"2024-09-10T16:00:35.616599+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:35.616599+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14983168,"num_objects":3658,"num_object_clones":0,"num_object_copies":7316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13714,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":122}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'17428","reported_seq":18188,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.681406+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:35.681406+0000","last_peered":"2024-09-10T16:00:35.681406+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:35.681406+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15032803,"num_objects":3671,"num_object_clones":0,"num_object_copies":7342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14247,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-23}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":237126139,"num_objects":57894,"num_object_clones":0,"num_object_copies":115788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32360,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57894,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235156,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14288,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":664780800,"data_stored":664704009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160835,"ondisk_log_size":160835,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626524,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552696,"kb_used_data":173984,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93819144,"statfs":{"total":96636764160,"available":96070803456,"internally_reserved":0,"allocated":178159616,"data_stored":177821238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":387772078},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,2,9],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149806,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":665056,"kb_used_data":236364,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93706784,"statfs":{"total":96636764160,"available":95955746816,"internally_reserved":0,"allocated":242036736,"data_stored":241760238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":438955693},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575624,"kb_used_data":241336,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93796216,"statfs":{"total":96636764160,"available":96047325184,"internally_reserved":0,"allocated":247128064,"data_stored":246847494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":241012736,"data_stored":241009123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":177360896,"data_stored":177295403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":246407168,"data_stored":246399483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:41.248 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:41.793 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:41.794 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:41.820 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-10T16:00:40.598274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":237126158,"num_objects":57896,"num_object_clones":0,"num_object_copies":115792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32360,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57896,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235158,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14290,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160838,"ondisk_log_size":160838,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1793376,"kb_used_data":651684,"kb_used_omap":12,"kb_used_meta":1141619,"kb_avail":281322144,"statfs":{"total":289910292480,"available":288073875456,"internally_reserved":0,"allocated":667324416,"data_stored":666428970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1169017861},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,8,7,17],"upper_bound":64},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10002432,"num_objects":-2442,"num_object_clones":0,"num_object_copies":-4884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2442,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002905"},"pg_stats":[{"pgid":"2.f","version":"48'17139","reported_seq":17943,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.698800+0000","last_change":"2024-09-10T15:53:19.926488+0000","last_active":"2024-09-10T16:00:35.698800+0000","last_peered":"2024-09-10T16:00:35.698800+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:35.698800+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14872576,"num_objects":3631,"num_object_clones":0,"num_object_copies":7262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17139,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":19}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'17450","reported_seq":17563,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:35.713746+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:35.713746+0000","last_peered":"2024-09-10T16:00:35.713746+0000","last_clean":"2024-09-10T16:00:35.713746+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:35.713746+0000","last_undegraded":"2024-09-10T16:00:35.713746+0000","last_fullsized":"2024-09-10T16:00:35.713746+0000","mapping_epoch":36,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14843904,"num_objects":3624,"num_object_clones":0,"num_object_copies":7248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15786,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'16992","reported_seq":17466,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.727513+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:35.727513+0000","last_peered":"2024-09-10T16:00:35.727513+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:35.727513+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14897176,"num_objects":3638,"num_object_clones":0,"num_object_copies":7276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3638,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15260,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"48'17025","reported_seq":17110,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:35.760984+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:35.760984+0000","last_peered":"2024-09-10T16:00:35.760984+0000","last_clean":"2024-09-10T16:00:35.760984+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:35.760984+0000","last_undegraded":"2024-09-10T16:00:35.760984+0000","last_fullsized":"2024-09-10T16:00:35.760984+0000","mapping_epoch":36,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14979072,"num_objects":3657,"num_object_clones":0,"num_object_copies":7314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17025,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'16797","reported_seq":16929,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:35.627014+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:35.627014+0000","last_peered":"2024-09-10T16:00:35.627014+0000","last_clean":"2024-09-10T16:00:35.627014+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:35.627014+0000","last_undegraded":"2024-09-10T16:00:35.627014+0000","last_fullsized":"2024-09-10T16:00:35.627014+0000","mapping_epoch":35,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14475264,"num_objects":3534,"num_object_clones":0,"num_object_copies":7068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13868,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'17256","reported_seq":25150,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:36.154967+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:36.154967+0000","last_peered":"2024-09-10T16:00:36.154967+0000","last_clean":"2024-09-10T16:00:36.154967+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:36.154967+0000","last_undegraded":"2024-09-10T16:00:36.154967+0000","last_fullsized":"2024-09-10T16:00:36.154967+0000","mapping_epoch":47,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14790656,"num_objects":3611,"num_object_clones":0,"num_object_copies":7222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13974,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"48'16766","reported_seq":17532,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.619964+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:35.619964+0000","last_peered":"2024-09-10T16:00:35.619964+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:35.619964+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6700","ondisk_log_start":"44'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14417920,"num_objects":3520,"num_object_clones":0,"num_object_copies":7040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13416,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'17259","reported_seq":17396,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:35.634088+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:35.634088+0000","last_peered":"2024-09-10T16:00:35.634088+0000","last_clean":"2024-09-10T16:00:35.634088+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:35.634088+0000","last_undegraded":"2024-09-10T16:00:35.634088+0000","last_fullsized":"2024-09-10T16:00:35.634088+0000","mapping_epoch":35,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14970880,"num_objects":3655,"num_object_clones":0,"num_object_copies":7310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14018,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746346+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T15:58:12.746346+0000","last_peered":"2024-09-10T15:58:12.746346+0000","last_clean":"2024-09-10T15:58:12.746346+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T15:58:12.746346+0000","last_undegraded":"2024-09-10T15:58:12.746346+0000","last_fullsized":"2024-09-10T15:58:12.746346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'17152","reported_seq":25285,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T16:00:36.174344+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:36.174344+0000","last_peered":"2024-09-10T16:00:36.174344+0000","last_clean":"2024-09-10T16:00:36.174344+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:36.174344+0000","last_undegraded":"2024-09-10T16:00:36.174344+0000","last_fullsized":"2024-09-10T16:00:36.174344+0000","mapping_epoch":45,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14610432,"num_objects":3567,"num_object_clones":0,"num_object_copies":7134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15129,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.748014+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T15:58:12.748014+0000","last_peered":"2024-09-10T15:58:12.748014+0000","last_clean":"2024-09-10T15:58:12.748014+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T15:58:12.748014+0000","last_undegraded":"2024-09-10T15:58:12.748014+0000","last_fullsized":"2024-09-10T15:58:12.748014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'16992","reported_seq":17811,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.648717+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:35.648717+0000","last_peered":"2024-09-10T16:00:35.648717+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:35.648717+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14688256,"num_objects":3586,"num_object_clones":0,"num_object_copies":7172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13856,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":67}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":107,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:24.992521+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T15:58:24.992521+0000","last_peered":"2024-09-10T15:58:24.992521+0000","last_clean":"2024-09-10T15:58:24.992521+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T15:58:24.992521+0000","last_undegraded":"2024-09-10T15:58:24.992521+0000","last_fullsized":"2024-09-10T15:58:24.992521+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'17115","reported_seq":24834,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.747377+0000","last_change":"2024-09-10T15:58:10.939004+0000","last_active":"2024-09-10T16:00:35.747377+0000","last_peered":"2024-09-10T16:00:35.747377+0000","last_clean":"2024-09-10T15:53:17.326494+0000","last_became_active":"2024-09-10T15:53:19.928986+0000","last_became_peered":"2024-09-10T15:53:19.928986+0000","last_unstale":"2024-09-10T16:00:35.747377+0000","last_undegraded":"2024-09-10T15:53:19.505406+0000","last_fullsized":"2024-09-10T15:53:19.469209+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15032320,"num_objects":3670,"num_object_clones":0,"num_object_copies":7340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15089,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3794,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3630}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746399+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T15:58:12.746399+0000","last_peered":"2024-09-10T15:58:12.746399+0000","last_clean":"2024-09-10T15:58:12.746399+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T15:58:12.746399+0000","last_undegraded":"2024-09-10T15:58:12.746399+0000","last_fullsized":"2024-09-10T15:58:12.746399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'16816","reported_seq":17234,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.701047+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:35.701047+0000","last_peered":"2024-09-10T16:00:35.701047+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:35.701047+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14909440,"num_objects":3640,"num_object_clones":0,"num_object_copies":7280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14685,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-46}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'17188","reported_seq":18031,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.663128+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:35.663128+0000","last_peered":"2024-09-10T16:00:35.663128+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:35.663128+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15015936,"num_objects":3666,"num_object_clones":0,"num_object_copies":7332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14380,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":188}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747229+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T15:58:12.747229+0000","last_peered":"2024-09-10T15:58:12.747229+0000","last_clean":"2024-09-10T15:58:12.747229+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T15:58:12.747229+0000","last_undegraded":"2024-09-10T15:58:12.747229+0000","last_fullsized":"2024-09-10T15:58:12.747229+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'17152","reported_seq":17903,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.676546+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:35.676546+0000","last_peered":"2024-09-10T16:00:35.676546+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:35.676546+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14606336,"num_objects":3566,"num_object_clones":0,"num_object_copies":7132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13570,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-60}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.747711+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T15:58:12.747711+0000","last_peered":"2024-09-10T15:58:12.747711+0000","last_clean":"2024-09-10T15:58:12.747711+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T15:58:12.747711+0000","last_undegraded":"2024-09-10T15:58:12.747711+0000","last_fullsized":"2024-09-10T15:58:12.747711+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'17108","reported_seq":17935,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.616599+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:35.616599+0000","last_peered":"2024-09-10T16:00:35.616599+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:35.616599+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14983168,"num_objects":3658,"num_object_clones":0,"num_object_copies":7316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13714,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":122}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746590+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T15:58:12.746590+0000","last_peered":"2024-09-10T15:58:12.746590+0000","last_clean":"2024-09-10T15:58:12.746590+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T15:58:12.746590+0000","last_undegraded":"2024-09-10T15:58:12.746590+0000","last_fullsized":"2024-09-10T15:58:12.746590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-10T15:58:12.746581+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T15:58:12.746581+0000","last_peered":"2024-09-10T15:58:12.746581+0000","last_clean":"2024-09-10T15:58:12.746581+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T15:58:12.746581+0000","last_undegraded":"2024-09-10T15:58:12.746581+0000","last_fullsized":"2024-09-10T15:58:12.746581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'17428","reported_seq":18188,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:35.681406+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:35.681406+0000","last_peered":"2024-09-10T16:00:35.681406+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:35.681406+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15032803,"num_objects":3671,"num_object_clones":0,"num_object_copies":7342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14247,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-23}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":237126139,"num_objects":57894,"num_object_clones":0,"num_object_copies":115788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32360,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57894,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235156,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14288,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":664780800,"data_stored":664704009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160835,"ondisk_log_size":160835,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626524,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552696,"kb_used_data":173984,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93819144,"statfs":{"total":96636764160,"available":96070803456,"internally_reserved":0,"allocated":178159616,"data_stored":177821238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":387772078},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,2,9],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149806,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":665056,"kb_used_data":236364,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93706784,"statfs":{"total":96636764160,"available":95955746816,"internally_reserved":0,"allocated":242036736,"data_stored":241760238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":438955693},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575624,"kb_used_data":241336,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93796216,"statfs":{"total":96636764160,"available":96047325184,"internally_reserved":0,"allocated":247128064,"data_stored":246847494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":241012736,"data_stored":241009123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":177360896,"data_stored":177295403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":246407168,"data_stored":246399483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:41.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:41.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'17139", 'reported_seq': 17943, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:35.698800+0000', 'last_change': '2024-09-10T15:53:19.926488+0000', 'last_active': '2024-09-10T16:00:35.698800+0000', 'last_peered': '2024-09-10T16:00:35.698800+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:00:35.698800+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'7100", 'ondisk_log_start': "44'7100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14872576, 'num_objects': 3631, 'num_object_clones': 0, 'num_object_copies': 7262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3612, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3631, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17139, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 19}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:41.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:41.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "48'16992", 'reported_seq': 17466, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:35.727513+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:00:35.727513+0000', 'last_peered': '2024-09-10T16:00:35.727513+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:00:35.727513+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'6900", 'ondisk_log_start': "44'6900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14897176, 'num_objects': 3638, 'num_object_clones': 0, 'num_object_copies': 7276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3638, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15260, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 9}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:41.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:41.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'16766", 'reported_seq': 17532, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:35.619964+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:00:35.619964+0000', 'last_peered': '2024-09-10T16:00:35.619964+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:00:35.619964+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'6700", 'ondisk_log_start': "44'6700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14417920, 'num_objects': 3520, 'num_object_clones': 0, 'num_object_copies': 7040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3520, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13416, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:41.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:41.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'16992", 'reported_seq': 17811, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:35.648717+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:00:35.648717+0000', 'last_peered': '2024-09-10T16:00:35.648717+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:00:35.648717+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'6900", 'ondisk_log_start': "44'6900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14688256, 'num_objects': 3586, 'num_object_clones': 0, 'num_object_copies': 7172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3586, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13856, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 67}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:41.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-10T16:00:41.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'17115", 'reported_seq': 24834, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:35.747377+0000', 'last_change': '2024-09-10T15:58:10.939004+0000', 'last_active': '2024-09-10T16:00:35.747377+0000', 'last_peered': '2024-09-10T16:00:35.747377+0000', 'last_clean': '2024-09-10T15:53:17.326494+0000', 'last_became_active': '2024-09-10T15:53:19.928986+0000', 'last_became_peered': '2024-09-10T15:53:19.928986+0000', 'last_unstale': '2024-09-10T16:00:35.747377+0000', 'last_undegraded': '2024-09-10T15:53:19.505406+0000', 'last_fullsized': '2024-09-10T15:53:19.469209+0000', 'mapping_epoch': 43, 'log_start': "44'7100", 'ondisk_log_start': "44'7100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2062", 'last_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:53.665808+0000', 'objects_scrubbed': 2021, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 746, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15032320, 'num_objects': 3670, 'num_object_clones': 0, 'num_object_copies': 7340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 40, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3670, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15089, 'num_write_kb': 33476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3794, 'num_bytes_recovered': 5206016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3630}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:41.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:41.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'16816", 'reported_seq': 17234, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:35.701047+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:00:35.701047+0000', 'last_peered': '2024-09-10T16:00:35.701047+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:00:35.701047+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'6800", 'ondisk_log_start': "44'6800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14909440, 'num_objects': 3640, 'num_object_clones': 0, 'num_object_copies': 7280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3640, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14685, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -46}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:41.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:41.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'17188", 'reported_seq': 18031, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:35.663128+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:00:35.663128+0000', 'last_peered': '2024-09-10T16:00:35.663128+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:00:35.663128+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'7100", 'ondisk_log_start': "44'7100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15015936, 'num_objects': 3666, 'num_object_clones': 0, 'num_object_copies': 7332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3666, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14380, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 188}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:41.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:41.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'17152", 'reported_seq': 17903, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:35.676546+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:00:35.676546+0000', 'last_peered': '2024-09-10T16:00:35.676546+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:00:35.676546+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'7100", 'ondisk_log_start': "44'7100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14606336, 'num_objects': 3566, 'num_object_clones': 0, 'num_object_copies': 7132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13570, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -60}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:41.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:41.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'17108", 'reported_seq': 17935, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:35.616599+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:00:35.616599+0000', 'last_peered': '2024-09-10T16:00:35.616599+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:00:35.616599+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'7100", 'ondisk_log_start': "44'7100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14983168, 'num_objects': 3658, 'num_object_clones': 0, 'num_object_copies': 7316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3658, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13714, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 122}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:41.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:41.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "48'17428", 'reported_seq': 18188, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:35.681406+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:00:35.681406+0000', 'last_peered': '2024-09-10T16:00:35.681406+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:00:35.681406+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'7400", 'ondisk_log_start': "44'7400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15032803, 'num_objects': 3671, 'num_object_clones': 0, 'num_object_copies': 7342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3671, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14247, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -23}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:41.823 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:42.518 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:42.518 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32360/115792 objects degraded (27.947%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":568,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":57896,"data_bytes":237126158,"bytes_used":1836417024,"bytes_avail":288073875456,"bytes_total":289910292480,"degraded_objects":32360,"degraded_total":115792,"degraded_ratio":0.27946662981898579,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":121,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":305},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T15:59:36.565732+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:00:42.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:42.540 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:43.729 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:43.732 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:43.752 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-10T16:00:42.599094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":227922446,"num_objects":55649,"num_object_clones":0,"num_object_copies":111298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":237405,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14447,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160685,"ondisk_log_size":160685,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1788180,"kb_used_data":634012,"kb_used_omap":12,"kb_used_meta":1154099,"kb_avail":281327340,"statfs":{"total":289910292480,"available":288079196160,"internally_reserved":0,"allocated":649228288,"data_stored":648301357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1181797381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,11,9,3,7],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19206144,"num_objects":-4689,"num_object_clones":0,"num_object_copies":-9378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4689,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003068"},"pg_stats":[{"pgid":"2.f","version":"50'17280","reported_seq":18322,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:40.739142+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:00:40.739142+0000","last_peered":"2024-09-10T16:00:40.739142+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:40.739142+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14295040,"num_objects":3490,"num_object_clones":0,"num_object_copies":6980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17280,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":478,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'17609","reported_seq":17726,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:40.708200+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:40.708200+0000","last_peered":"2024-09-10T16:00:40.708200+0000","last_clean":"2024-09-10T16:00:40.708200+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:40.708200+0000","last_undegraded":"2024-09-10T16:00:40.708200+0000","last_fullsized":"2024-09-10T16:00:40.708200+0000","mapping_epoch":36,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14192640,"num_objects":3465,"num_object_clones":0,"num_object_copies":6930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15945,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17147","reported_seq":17626,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:40.740089+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:40.740089+0000","last_peered":"2024-09-10T16:00:40.740089+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:40.740089+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14262296,"num_objects":3483,"num_object_clones":0,"num_object_copies":6966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15415,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-146}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17160","reported_seq":17249,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:40.635148+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:40.635148+0000","last_peered":"2024-09-10T16:00:40.635148+0000","last_clean":"2024-09-10T16:00:40.635148+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:40.635148+0000","last_undegraded":"2024-09-10T16:00:40.635148+0000","last_fullsized":"2024-09-10T16:00:40.635148+0000","mapping_epoch":36,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14426112,"num_objects":3522,"num_object_clones":0,"num_object_copies":7044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17160,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'16942","reported_seq":17078,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:40.699013+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:40.699013+0000","last_peered":"2024-09-10T16:00:40.699013+0000","last_clean":"2024-09-10T16:00:40.699013+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:40.699013+0000","last_undegraded":"2024-09-10T16:00:40.699013+0000","last_fullsized":"2024-09-10T16:00:40.699013+0000","mapping_epoch":35,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13881344,"num_objects":3389,"num_object_clones":0,"num_object_copies":6778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14013,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'17416","reported_seq":25314,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:41.191997+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:41.191997+0000","last_peered":"2024-09-10T16:00:41.191997+0000","last_clean":"2024-09-10T16:00:41.191997+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:41.191997+0000","last_undegraded":"2024-09-10T16:00:41.191997+0000","last_fullsized":"2024-09-10T16:00:41.191997+0000","mapping_epoch":47,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14135296,"num_objects":3451,"num_object_clones":0,"num_object_copies":6902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14134,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'16886","reported_seq":17656,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:40.736570+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:40.736570+0000","last_peered":"2024-09-10T16:00:40.736570+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:40.736570+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'6800","ondisk_log_start":"44'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13926400,"num_objects":3400,"num_object_clones":0,"num_object_copies":6800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13536,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17405","reported_seq":17546,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:40.720526+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:40.720526+0000","last_peered":"2024-09-10T16:00:40.720526+0000","last_clean":"2024-09-10T16:00:40.720526+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:40.720526+0000","last_undegraded":"2024-09-10T16:00:40.720526+0000","last_fullsized":"2024-09-10T16:00:40.720526+0000","mapping_epoch":35,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14372864,"num_objects":3509,"num_object_clones":0,"num_object_copies":7018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14164,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17307","reported_seq":25444,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:41.201299+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:41.201299+0000","last_peered":"2024-09-10T16:00:41.201299+0000","last_clean":"2024-09-10T16:00:41.201299+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:41.201299+0000","last_undegraded":"2024-09-10T16:00:41.201299+0000","last_fullsized":"2024-09-10T16:00:41.201299+0000","mapping_epoch":45,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13975552,"num_objects":3412,"num_object_clones":0,"num_object_copies":6824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15284,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947149+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947149+0000","last_peered":"2024-09-10T16:00:38.947149+0000","last_clean":"2024-09-10T16:00:38.947149+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947149+0000","last_undegraded":"2024-09-10T16:00:38.947149+0000","last_fullsized":"2024-09-10T16:00:38.947149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17125","reported_seq":17950,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:40.638997+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:40.638997+0000","last_peered":"2024-09-10T16:00:40.638997+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:40.638997+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14143488,"num_objects":3453,"num_object_clones":0,"num_object_copies":6906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13989,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-66}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17251","reported_seq":25060,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:40.766867+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:00:40.766867+0000","last_peered":"2024-09-10T16:00:40.766867+0000","last_clean":"2024-09-10T16:00:40.766867+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:00:40.766867+0000","last_undegraded":"2024-09-10T16:00:40.766867+0000","last_fullsized":"2024-09-10T16:00:40.766867+0000","mapping_epoch":49,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14475264,"num_objects":3534,"num_object_clones":0,"num_object_copies":7068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15225,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'16941","reported_seq":17363,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:40.708246+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:40.708246+0000","last_peered":"2024-09-10T16:00:40.708246+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:40.708246+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'6900","ondisk_log_start":"44'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14397440,"num_objects":3515,"num_object_clones":0,"num_object_copies":7030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14810,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-171}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17319","reported_seq":18166,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:40.665910+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:40.665910+0000","last_peered":"2024-09-10T16:00:40.665910+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:40.665910+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14479360,"num_objects":3535,"num_object_clones":0,"num_object_copies":7070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14511,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":57}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17269","reported_seq":18024,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:40.740318+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:40.740318+0000","last_peered":"2024-09-10T16:00:40.740318+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:40.740318+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14127104,"num_objects":3449,"num_object_clones":0,"num_object_copies":6898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13687,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-177}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.948956+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.948956+0000","last_peered":"2024-09-10T16:00:38.948956+0000","last_clean":"2024-09-10T16:00:38.948956+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.948956+0000","last_undegraded":"2024-09-10T16:00:38.948956+0000","last_fullsized":"2024-09-10T16:00:38.948956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17261","reported_seq":18091,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:40.740754+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:40.740754+0000","last_peered":"2024-09-10T16:00:40.740754+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:40.740754+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14356480,"num_objects":3505,"num_object_clones":0,"num_object_copies":7010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13867,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-31}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'17564","reported_seq":18330,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:40.699613+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:40.699613+0000","last_peered":"2024-09-10T16:00:40.699613+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:40.699613+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14475747,"num_objects":3535,"num_object_clones":0,"num_object_copies":7070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14383,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":227922427,"num_objects":55647,"num_object_clones":0,"num_object_copies":111294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55647,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":237403,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14445,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":646328320,"data_stored":646251529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160682,"ondisk_log_size":160682,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626525,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550324,"kb_used_data":167456,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93821516,"statfs":{"total":96636764160,"available":96073232384,"internally_reserved":0,"allocated":171474944,"data_stored":171126071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":392031918},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149807,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662692,"kb_used_data":229840,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93709148,"statfs":{"total":96636764160,"available":95958167552,"internally_reserved":0,"allocated":235356160,"data_stored":235069167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":443215533},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953578,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575164,"kb_used_data":236716,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93796676,"statfs":{"total":96636764160,"available":96047796224,"internally_reserved":0,"allocated":242397184,"data_stored":242106119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1,3,7],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":234123264,"data_stored":234119651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":170516480,"data_stored":170450987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":241688576,"data_stored":241680891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:46.754 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:47.319 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:47.320 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:47.356 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-10T16:00:46.600774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":217940494,"num_objects":53212,"num_object_clones":0,"num_object_copies":106424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53212,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239842,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14519,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1786736,"kb_used_data":614524,"kb_used_omap":12,"kb_used_meta":1172147,"kb_avail":281328784,"statfs":{"total":289910292480,"available":288080674816,"internally_reserved":0,"allocated":629272576,"data_stored":628345645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1200278533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,6,4,9,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19185664,"num_objects":-4684,"num_object_clones":0,"num_object_copies":-9368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-229,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4684,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.658590"},"pg_stats":[{"pgid":"2.f","version":"50'17431","reported_seq":18618,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.769807+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:00:45.769807+0000","last_peered":"2024-09-10T16:00:45.769807+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:45.769807+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13676544,"num_objects":3339,"num_object_clones":0,"num_object_copies":6678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17431,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-84}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'17761","reported_seq":17877,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.777790+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:45.777790+0000","last_peered":"2024-09-10T16:00:45.777790+0000","last_clean":"2024-09-10T16:00:45.777790+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:45.777790+0000","last_undegraded":"2024-09-10T16:00:45.777790+0000","last_fullsized":"2024-09-10T16:00:45.777790+0000","mapping_epoch":36,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13570048,"num_objects":3313,"num_object_clones":0,"num_object_copies":6626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16097,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17304","reported_seq":17784,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.756136+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:45.756136+0000","last_peered":"2024-09-10T16:00:45.756136+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:45.756136+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13619224,"num_objects":3326,"num_object_clones":0,"num_object_copies":6652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15572,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-303}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17313","reported_seq":17402,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.770173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:45.770173+0000","last_peered":"2024-09-10T16:00:45.770173+0000","last_clean":"2024-09-10T16:00:45.770173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:45.770173+0000","last_undegraded":"2024-09-10T16:00:45.770173+0000","last_fullsized":"2024-09-10T16:00:45.770173+0000","mapping_epoch":36,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13799424,"num_objects":3369,"num_object_clones":0,"num_object_copies":6738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17313,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17074","reported_seq":17210,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.734058+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:45.734058+0000","last_peered":"2024-09-10T16:00:45.734058+0000","last_clean":"2024-09-10T16:00:45.734058+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:45.734058+0000","last_undegraded":"2024-09-10T16:00:45.734058+0000","last_fullsized":"2024-09-10T16:00:45.734058+0000","mapping_epoch":35,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13340672,"num_objects":3257,"num_object_clones":0,"num_object_copies":6514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14145,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'17571","reported_seq":25469,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:46.233950+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:46.233950+0000","last_peered":"2024-09-10T16:00:46.233950+0000","last_clean":"2024-09-10T16:00:46.233950+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:46.233950+0000","last_undegraded":"2024-09-10T16:00:46.233950+0000","last_fullsized":"2024-09-10T16:00:46.233950+0000","mapping_epoch":47,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":3296,"num_object_clones":0,"num_object_copies":6592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14289,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17043","reported_seq":17813,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.774019+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:45.774019+0000","last_peered":"2024-09-10T16:00:45.774019+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:45.774019+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13283328,"num_objects":3243,"num_object_clones":0,"num_object_copies":6486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-281}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17553","reported_seq":17694,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.726946+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:45.726946+0000","last_peered":"2024-09-10T16:00:45.726946+0000","last_clean":"2024-09-10T16:00:45.726946+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:45.726946+0000","last_undegraded":"2024-09-10T16:00:45.726946+0000","last_fullsized":"2024-09-10T16:00:45.726946+0000","mapping_epoch":35,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13766656,"num_objects":3361,"num_object_clones":0,"num_object_copies":6722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14312,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17456","reported_seq":25593,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:46.206920+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:46.206920+0000","last_peered":"2024-09-10T16:00:46.206920+0000","last_clean":"2024-09-10T16:00:46.206920+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:46.206920+0000","last_undegraded":"2024-09-10T16:00:46.206920+0000","last_fullsized":"2024-09-10T16:00:46.206920+0000","mapping_epoch":45,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13365248,"num_objects":3263,"num_object_clones":0,"num_object_copies":6526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15433,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17280","reported_seq":18103,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.712140+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:45.712140+0000","last_peered":"2024-09-10T16:00:45.712140+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:45.712140+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13508608,"num_objects":3298,"num_object_clones":0,"num_object_copies":6596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14144,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-221}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17412","reported_seq":25221,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.768568+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:00:45.768568+0000","last_peered":"2024-09-10T16:00:45.768568+0000","last_clean":"2024-09-10T16:00:45.768568+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:00:45.768568+0000","last_undegraded":"2024-09-10T16:00:45.768568+0000","last_fullsized":"2024-09-10T16:00:45.768568+0000","mapping_epoch":49,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13815808,"num_objects":3373,"num_object_clones":0,"num_object_copies":6746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15386,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17087","reported_seq":17509,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.746144+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:45.746144+0000","last_peered":"2024-09-10T16:00:45.746144+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:45.746144+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13799424,"num_objects":3369,"num_object_clones":0,"num_object_copies":6738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14956,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-317}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17477","reported_seq":18324,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.723543+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:45.723543+0000","last_peered":"2024-09-10T16:00:45.723543+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:45.723543+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13832192,"num_objects":3377,"num_object_clones":0,"num_object_copies":6754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14669,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17421","reported_seq":18176,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.786814+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:45.786814+0000","last_peered":"2024-09-10T16:00:45.786814+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:45.786814+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504512,"num_objects":3297,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13839,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-329}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17400","reported_seq":18231,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.784465+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:45.784465+0000","last_peered":"2024-09-10T16:00:45.784465+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:45.784465+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787136,"num_objects":3366,"num_object_clones":0,"num_object_copies":6732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14006,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-170}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'17736","reported_seq":18501,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.789808+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:45.789808+0000","last_peered":"2024-09-10T16:00:45.789808+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:45.789808+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13771235,"num_objects":3363,"num_object_clones":0,"num_object_copies":6726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14555,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":217940475,"num_objects":53210,"num_object_clones":0,"num_object_copies":106420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239840,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14517,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":626348032,"data_stored":626271241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626526,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552496,"kb_used_data":159904,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93819344,"statfs":{"total":96636764160,"available":96071008256,"internally_reserved":0,"allocated":163741696,"data_stored":163392823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":401993390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149808,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659628,"kb_used_data":222620,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93712212,"statfs":{"total":96636764160,"available":95961305088,"internally_reserved":0,"allocated":227962880,"data_stored":227675887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":447475373},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574612,"kb_used_data":232000,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93797228,"statfs":{"total":96636764160,"available":96048361472,"internally_reserved":0,"allocated":237568000,"data_stored":237276935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,0,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":226668544,"data_stored":226664931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162865152,"data_stored":162799659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":236814336,"data_stored":236806651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:47.357 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:47.876 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:47.878 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:47.910 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-10T16:00:46.600774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":217940494,"num_objects":53212,"num_object_clones":0,"num_object_copies":106424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53212,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239842,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14519,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1786736,"kb_used_data":614524,"kb_used_omap":12,"kb_used_meta":1172147,"kb_avail":281328784,"statfs":{"total":289910292480,"available":288080674816,"internally_reserved":0,"allocated":629272576,"data_stored":628345645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1200278533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,6,4,9,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19185664,"num_objects":-4684,"num_object_clones":0,"num_object_copies":-9368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-229,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4684,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.658590"},"pg_stats":[{"pgid":"2.f","version":"50'17431","reported_seq":18618,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.769807+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:00:45.769807+0000","last_peered":"2024-09-10T16:00:45.769807+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:45.769807+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13676544,"num_objects":3339,"num_object_clones":0,"num_object_copies":6678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17431,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-84}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'17761","reported_seq":17877,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.777790+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:45.777790+0000","last_peered":"2024-09-10T16:00:45.777790+0000","last_clean":"2024-09-10T16:00:45.777790+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:45.777790+0000","last_undegraded":"2024-09-10T16:00:45.777790+0000","last_fullsized":"2024-09-10T16:00:45.777790+0000","mapping_epoch":36,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13570048,"num_objects":3313,"num_object_clones":0,"num_object_copies":6626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16097,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17304","reported_seq":17784,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.756136+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:45.756136+0000","last_peered":"2024-09-10T16:00:45.756136+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:45.756136+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13619224,"num_objects":3326,"num_object_clones":0,"num_object_copies":6652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15572,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-303}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17313","reported_seq":17402,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.770173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:45.770173+0000","last_peered":"2024-09-10T16:00:45.770173+0000","last_clean":"2024-09-10T16:00:45.770173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:45.770173+0000","last_undegraded":"2024-09-10T16:00:45.770173+0000","last_fullsized":"2024-09-10T16:00:45.770173+0000","mapping_epoch":36,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13799424,"num_objects":3369,"num_object_clones":0,"num_object_copies":6738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17313,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17074","reported_seq":17210,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.734058+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:45.734058+0000","last_peered":"2024-09-10T16:00:45.734058+0000","last_clean":"2024-09-10T16:00:45.734058+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:45.734058+0000","last_undegraded":"2024-09-10T16:00:45.734058+0000","last_fullsized":"2024-09-10T16:00:45.734058+0000","mapping_epoch":35,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13340672,"num_objects":3257,"num_object_clones":0,"num_object_copies":6514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14145,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'17571","reported_seq":25469,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:46.233950+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:46.233950+0000","last_peered":"2024-09-10T16:00:46.233950+0000","last_clean":"2024-09-10T16:00:46.233950+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:46.233950+0000","last_undegraded":"2024-09-10T16:00:46.233950+0000","last_fullsized":"2024-09-10T16:00:46.233950+0000","mapping_epoch":47,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":3296,"num_object_clones":0,"num_object_copies":6592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14289,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17043","reported_seq":17813,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.774019+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:45.774019+0000","last_peered":"2024-09-10T16:00:45.774019+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:45.774019+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13283328,"num_objects":3243,"num_object_clones":0,"num_object_copies":6486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-281}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17553","reported_seq":17694,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.726946+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:45.726946+0000","last_peered":"2024-09-10T16:00:45.726946+0000","last_clean":"2024-09-10T16:00:45.726946+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:45.726946+0000","last_undegraded":"2024-09-10T16:00:45.726946+0000","last_fullsized":"2024-09-10T16:00:45.726946+0000","mapping_epoch":35,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13766656,"num_objects":3361,"num_object_clones":0,"num_object_copies":6722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14312,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17456","reported_seq":25593,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:46.206920+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:46.206920+0000","last_peered":"2024-09-10T16:00:46.206920+0000","last_clean":"2024-09-10T16:00:46.206920+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:46.206920+0000","last_undegraded":"2024-09-10T16:00:46.206920+0000","last_fullsized":"2024-09-10T16:00:46.206920+0000","mapping_epoch":45,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13365248,"num_objects":3263,"num_object_clones":0,"num_object_copies":6526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15433,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17280","reported_seq":18103,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.712140+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:45.712140+0000","last_peered":"2024-09-10T16:00:45.712140+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:45.712140+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13508608,"num_objects":3298,"num_object_clones":0,"num_object_copies":6596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14144,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-221}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17412","reported_seq":25221,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.768568+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:00:45.768568+0000","last_peered":"2024-09-10T16:00:45.768568+0000","last_clean":"2024-09-10T16:00:45.768568+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:00:45.768568+0000","last_undegraded":"2024-09-10T16:00:45.768568+0000","last_fullsized":"2024-09-10T16:00:45.768568+0000","mapping_epoch":49,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13815808,"num_objects":3373,"num_object_clones":0,"num_object_copies":6746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15386,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17087","reported_seq":17509,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.746144+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:45.746144+0000","last_peered":"2024-09-10T16:00:45.746144+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:45.746144+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13799424,"num_objects":3369,"num_object_clones":0,"num_object_copies":6738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14956,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-317}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17477","reported_seq":18324,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.723543+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:45.723543+0000","last_peered":"2024-09-10T16:00:45.723543+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:45.723543+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13832192,"num_objects":3377,"num_object_clones":0,"num_object_copies":6754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14669,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17421","reported_seq":18176,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.786814+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:45.786814+0000","last_peered":"2024-09-10T16:00:45.786814+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:45.786814+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504512,"num_objects":3297,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13839,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-329}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17400","reported_seq":18231,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.784465+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:45.784465+0000","last_peered":"2024-09-10T16:00:45.784465+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:45.784465+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787136,"num_objects":3366,"num_object_clones":0,"num_object_copies":6732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14006,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-170}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'17736","reported_seq":18501,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.789808+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:45.789808+0000","last_peered":"2024-09-10T16:00:45.789808+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:45.789808+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13771235,"num_objects":3363,"num_object_clones":0,"num_object_copies":6726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14555,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":217940475,"num_objects":53210,"num_object_clones":0,"num_object_copies":106420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239840,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14517,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":626348032,"data_stored":626271241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626526,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552496,"kb_used_data":159904,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93819344,"statfs":{"total":96636764160,"available":96071008256,"internally_reserved":0,"allocated":163741696,"data_stored":163392823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":401993390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149808,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659628,"kb_used_data":222620,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93712212,"statfs":{"total":96636764160,"available":95961305088,"internally_reserved":0,"allocated":227962880,"data_stored":227675887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":447475373},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574612,"kb_used_data":232000,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93797228,"statfs":{"total":96636764160,"available":96048361472,"internally_reserved":0,"allocated":237568000,"data_stored":237276935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,0,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":226668544,"data_stored":226664931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162865152,"data_stored":162799659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":236814336,"data_stored":236806651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:47.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:47.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'17431", 'reported_seq': 18618, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:45.769807+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:00:45.769807+0000', 'last_peered': '2024-09-10T16:00:45.769807+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:00:45.769807+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'7400", 'ondisk_log_start': "44'7400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13676544, 'num_objects': 3339, 'num_object_clones': 0, 'num_object_copies': 6678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3423, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3339, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17431, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 550, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -84}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:47.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:47.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'17304", 'reported_seq': 17784, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:45.756136+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:00:45.756136+0000', 'last_peered': '2024-09-10T16:00:45.756136+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:00:45.756136+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'7300", 'ondisk_log_start': "44'7300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13619224, 'num_objects': 3326, 'num_object_clones': 0, 'num_object_copies': 6652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3326, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15572, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:47.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:47.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'17043", 'reported_seq': 17813, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:45.774019+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:00:45.774019+0000', 'last_peered': '2024-09-10T16:00:45.774019+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:00:45.774019+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'7000", 'ondisk_log_start': "44'7000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13283328, 'num_objects': 3243, 'num_object_clones': 0, 'num_object_copies': 6486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3243, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13693, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -281}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:47.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:47.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'17280", 'reported_seq': 18103, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:45.712140+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:00:45.712140+0000', 'last_peered': '2024-09-10T16:00:45.712140+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:00:45.712140+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'7200", 'ondisk_log_start': "44'7200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13508608, 'num_objects': 3298, 'num_object_clones': 0, 'num_object_copies': 6596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3298, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14144, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -221}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:47.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:47.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'17087", 'reported_seq': 17509, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:45.746144+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:00:45.746144+0000', 'last_peered': '2024-09-10T16:00:45.746144+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:00:45.746144+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'7000", 'ondisk_log_start': "44'7000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13799424, 'num_objects': 3369, 'num_object_clones': 0, 'num_object_copies': 6738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3369, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14956, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -317}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:47.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:47.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'17477", 'reported_seq': 18324, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:45.723543+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:00:45.723543+0000', 'last_peered': '2024-09-10T16:00:45.723543+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:00:45.723543+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'7400", 'ondisk_log_start': "44'7400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13832192, 'num_objects': 3377, 'num_object_clones': 0, 'num_object_copies': 6754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14669, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -101}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:47.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:47.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'17421", 'reported_seq': 18176, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:45.786814+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:00:45.786814+0000', 'last_peered': '2024-09-10T16:00:45.786814+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:00:45.786814+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'7400", 'ondisk_log_start': "44'7400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13504512, 'num_objects': 3297, 'num_object_clones': 0, 'num_object_copies': 6594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3297, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13839, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -329}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:47.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:47.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'17400", 'reported_seq': 18231, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:45.784465+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:00:45.784465+0000', 'last_peered': '2024-09-10T16:00:45.784465+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:00:45.784465+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'7300", 'ondisk_log_start': "44'7300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13787136, 'num_objects': 3366, 'num_object_clones': 0, 'num_object_copies': 6732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3366, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14006, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -170}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:47.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:47.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'17736", 'reported_seq': 18501, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:45.789808+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:00:45.789808+0000', 'last_peered': '2024-09-10T16:00:45.789808+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:00:45.789808+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'7700", 'ondisk_log_start': "44'7700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13771235, 'num_objects': 3363, 'num_object_clones': 0, 'num_object_copies': 6726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3363, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14555, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -331}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:47.914 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:48.618 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:48.618 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32131/106424 objects degraded (30.191%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":574,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":53212,"data_bytes":217940494,"bytes_used":1829617664,"bytes_avail":288080674816,"bytes_total":289910292480,"degraded_objects":32131,"degraded_total":106424,"degraded_ratio":0.30191498158310154,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":540},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-10T16:00:44.600014+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:00:48.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:48.650 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:49.202 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:49.203 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:49.222 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-10T16:00:48.601657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":217940494,"num_objects":53212,"num_object_clones":0,"num_object_copies":106424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53212,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239842,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14519,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1786736,"kb_used_data":614524,"kb_used_omap":12,"kb_used_meta":1172147,"kb_avail":281328784,"statfs":{"total":289910292480,"available":288080674816,"internally_reserved":0,"allocated":629272576,"data_stored":628345645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1200278533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,6,4,9,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19185664,"num_objects":-4684,"num_object_clones":0,"num_object_copies":-9368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-229,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4684,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004187"},"pg_stats":[{"pgid":"2.f","version":"50'17431","reported_seq":18618,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.769807+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:00:45.769807+0000","last_peered":"2024-09-10T16:00:45.769807+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:45.769807+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13676544,"num_objects":3339,"num_object_clones":0,"num_object_copies":6678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17431,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-84}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'17761","reported_seq":17877,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.777790+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:45.777790+0000","last_peered":"2024-09-10T16:00:45.777790+0000","last_clean":"2024-09-10T16:00:45.777790+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:45.777790+0000","last_undegraded":"2024-09-10T16:00:45.777790+0000","last_fullsized":"2024-09-10T16:00:45.777790+0000","mapping_epoch":36,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13570048,"num_objects":3313,"num_object_clones":0,"num_object_copies":6626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16097,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17304","reported_seq":17784,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.756136+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:45.756136+0000","last_peered":"2024-09-10T16:00:45.756136+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:45.756136+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13619224,"num_objects":3326,"num_object_clones":0,"num_object_copies":6652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15572,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-303}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17313","reported_seq":17402,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.770173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:45.770173+0000","last_peered":"2024-09-10T16:00:45.770173+0000","last_clean":"2024-09-10T16:00:45.770173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:45.770173+0000","last_undegraded":"2024-09-10T16:00:45.770173+0000","last_fullsized":"2024-09-10T16:00:45.770173+0000","mapping_epoch":36,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13799424,"num_objects":3369,"num_object_clones":0,"num_object_copies":6738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17313,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17074","reported_seq":17210,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.734058+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:45.734058+0000","last_peered":"2024-09-10T16:00:45.734058+0000","last_clean":"2024-09-10T16:00:45.734058+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:45.734058+0000","last_undegraded":"2024-09-10T16:00:45.734058+0000","last_fullsized":"2024-09-10T16:00:45.734058+0000","mapping_epoch":35,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13340672,"num_objects":3257,"num_object_clones":0,"num_object_copies":6514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14145,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'17571","reported_seq":25469,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:46.233950+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:46.233950+0000","last_peered":"2024-09-10T16:00:46.233950+0000","last_clean":"2024-09-10T16:00:46.233950+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:46.233950+0000","last_undegraded":"2024-09-10T16:00:46.233950+0000","last_fullsized":"2024-09-10T16:00:46.233950+0000","mapping_epoch":47,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":3296,"num_object_clones":0,"num_object_copies":6592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14289,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17043","reported_seq":17813,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.774019+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:45.774019+0000","last_peered":"2024-09-10T16:00:45.774019+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:45.774019+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13283328,"num_objects":3243,"num_object_clones":0,"num_object_copies":6486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-281}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17553","reported_seq":17694,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.726946+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:45.726946+0000","last_peered":"2024-09-10T16:00:45.726946+0000","last_clean":"2024-09-10T16:00:45.726946+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:45.726946+0000","last_undegraded":"2024-09-10T16:00:45.726946+0000","last_fullsized":"2024-09-10T16:00:45.726946+0000","mapping_epoch":35,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13766656,"num_objects":3361,"num_object_clones":0,"num_object_copies":6722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14312,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17456","reported_seq":25593,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:46.206920+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:46.206920+0000","last_peered":"2024-09-10T16:00:46.206920+0000","last_clean":"2024-09-10T16:00:46.206920+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:46.206920+0000","last_undegraded":"2024-09-10T16:00:46.206920+0000","last_fullsized":"2024-09-10T16:00:46.206920+0000","mapping_epoch":45,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13365248,"num_objects":3263,"num_object_clones":0,"num_object_copies":6526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15433,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17280","reported_seq":18103,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.712140+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:45.712140+0000","last_peered":"2024-09-10T16:00:45.712140+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:45.712140+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13508608,"num_objects":3298,"num_object_clones":0,"num_object_copies":6596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14144,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-221}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17412","reported_seq":25221,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:45.768568+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:00:45.768568+0000","last_peered":"2024-09-10T16:00:45.768568+0000","last_clean":"2024-09-10T16:00:45.768568+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:00:45.768568+0000","last_undegraded":"2024-09-10T16:00:45.768568+0000","last_fullsized":"2024-09-10T16:00:45.768568+0000","mapping_epoch":49,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13815808,"num_objects":3373,"num_object_clones":0,"num_object_copies":6746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15386,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17087","reported_seq":17509,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.746144+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:45.746144+0000","last_peered":"2024-09-10T16:00:45.746144+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:45.746144+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7000","ondisk_log_start":"44'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13799424,"num_objects":3369,"num_object_clones":0,"num_object_copies":6738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14956,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-317}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17477","reported_seq":18324,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.723543+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:45.723543+0000","last_peered":"2024-09-10T16:00:45.723543+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:45.723543+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13832192,"num_objects":3377,"num_object_clones":0,"num_object_copies":6754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14669,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17421","reported_seq":18176,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.786814+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:45.786814+0000","last_peered":"2024-09-10T16:00:45.786814+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:45.786814+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504512,"num_objects":3297,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13839,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-329}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17400","reported_seq":18231,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.784465+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:45.784465+0000","last_peered":"2024-09-10T16:00:45.784465+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:45.784465+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787136,"num_objects":3366,"num_object_clones":0,"num_object_copies":6732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14006,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-170}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'17736","reported_seq":18501,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:45.789808+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:45.789808+0000","last_peered":"2024-09-10T16:00:45.789808+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:45.789808+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13771235,"num_objects":3363,"num_object_clones":0,"num_object_copies":6726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14555,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":217940475,"num_objects":53210,"num_object_clones":0,"num_object_copies":106420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239840,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14517,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":626348032,"data_stored":626271241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626526,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552496,"kb_used_data":159904,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93819344,"statfs":{"total":96636764160,"available":96071008256,"internally_reserved":0,"allocated":163741696,"data_stored":163392823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":401993390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149808,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659628,"kb_used_data":222620,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93712212,"statfs":{"total":96636764160,"available":95961305088,"internally_reserved":0,"allocated":227962880,"data_stored":227675887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":447475373},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574612,"kb_used_data":232000,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93797228,"statfs":{"total":96636764160,"available":96048361472,"internally_reserved":0,"allocated":237568000,"data_stored":237276935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,0,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":226668544,"data_stored":226664931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162865152,"data_stored":162799659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":236814336,"data_stored":236806651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:52.224 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:53.380 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:53.382 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:53.404 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-10T16:00:52.603040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":208175630,"num_objects":50828,"num_object_clones":0,"num_object_copies":101656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50828,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242226,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14596,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160706,"ondisk_log_size":160706,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1767352,"kb_used_data":595120,"kb_used_omap":12,"kb_used_meta":1172147,"kb_avail":281348168,"statfs":{"total":289910292480,"available":288100524032,"internally_reserved":0,"allocated":609402880,"data_stored":608475949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1200278533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,15,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28950528,"num_objects":-7068,"num_object_clones":0,"num_object_copies":-14136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7068,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004765"},"pg_stats":[{"pgid":"2.f","version":"50'17602","reported_seq":18940,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.743325+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:00:50.743325+0000","last_peered":"2024-09-10T16:00:50.743325+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:50.743325+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12976128,"num_objects":3168,"num_object_clones":0,"num_object_copies":6336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17602,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-178}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'17927","reported_seq":18044,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.790057+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:50.790057+0000","last_peered":"2024-09-10T16:00:50.790057+0000","last_clean":"2024-09-10T16:00:50.790057+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:50.790057+0000","last_undegraded":"2024-09-10T16:00:50.790057+0000","last_fullsized":"2024-09-10T16:00:50.790057+0000","mapping_epoch":36,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12890112,"num_objects":3147,"num_object_clones":0,"num_object_copies":6294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16263,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17449","reported_seq":17929,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.743861+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:50.743861+0000","last_peered":"2024-09-10T16:00:50.743861+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:50.743861+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13025304,"num_objects":3181,"num_object_clones":0,"num_object_copies":6362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15717,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-448}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17456","reported_seq":17545,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.758150+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:50.758150+0000","last_peered":"2024-09-10T16:00:50.758150+0000","last_clean":"2024-09-10T16:00:50.758150+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:50.758150+0000","last_undegraded":"2024-09-10T16:00:50.758150+0000","last_fullsized":"2024-09-10T16:00:50.758150+0000","mapping_epoch":36,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13213696,"num_objects":3226,"num_object_clones":0,"num_object_copies":6452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17456,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17227","reported_seq":17363,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.758503+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:50.758503+0000","last_peered":"2024-09-10T16:00:50.758503+0000","last_clean":"2024-09-10T16:00:50.758503+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:50.758503+0000","last_undegraded":"2024-09-10T16:00:50.758503+0000","last_fullsized":"2024-09-10T16:00:50.758503+0000","mapping_epoch":35,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12713984,"num_objects":3104,"num_object_clones":0,"num_object_copies":6208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14298,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'17720","reported_seq":25618,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:51.348860+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:51.348860+0000","last_peered":"2024-09-10T16:00:51.348860+0000","last_clean":"2024-09-10T16:00:51.348860+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:51.348860+0000","last_undegraded":"2024-09-10T16:00:51.348860+0000","last_fullsized":"2024-09-10T16:00:51.348860+0000","mapping_epoch":47,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12890112,"num_objects":3147,"num_object_clones":0,"num_object_copies":6294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14438,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17188","reported_seq":17958,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.713633+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:50.713633+0000","last_peered":"2024-09-10T16:00:50.713633+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:50.713633+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12689408,"num_objects":3098,"num_object_clones":0,"num_object_copies":6196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13838,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17700","reported_seq":17841,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.669251+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:50.669251+0000","last_peered":"2024-09-10T16:00:50.669251+0000","last_clean":"2024-09-10T16:00:50.669251+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:50.669251+0000","last_undegraded":"2024-09-10T16:00:50.669251+0000","last_fullsized":"2024-09-10T16:00:50.669251+0000","mapping_epoch":35,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13164544,"num_objects":3214,"num_object_clones":0,"num_object_copies":6428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14459,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17578","reported_seq":25715,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:51.287160+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:51.287160+0000","last_peered":"2024-09-10T16:00:51.287160+0000","last_clean":"2024-09-10T16:00:51.287160+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:51.287160+0000","last_undegraded":"2024-09-10T16:00:51.287160+0000","last_fullsized":"2024-09-10T16:00:51.287160+0000","mapping_epoch":45,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12865536,"num_objects":3141,"num_object_clones":0,"num_object_copies":6282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15555,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17422","reported_seq":18247,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.758128+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:50.758128+0000","last_peered":"2024-09-10T16:00:50.758128+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:50.758128+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12926976,"num_objects":3156,"num_object_clones":0,"num_object_copies":6312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14286,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-363}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17576","reported_seq":25384,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.760252+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:00:50.760252+0000","last_peered":"2024-09-10T16:00:50.760252+0000","last_clean":"2024-09-10T16:00:50.760252+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:00:50.760252+0000","last_undegraded":"2024-09-10T16:00:50.760252+0000","last_fullsized":"2024-09-10T16:00:50.760252+0000","mapping_epoch":49,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13144064,"num_objects":3209,"num_object_clones":0,"num_object_copies":6418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15550,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17222","reported_seq":17644,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.709084+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:50.709084+0000","last_peered":"2024-09-10T16:00:50.709084+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:50.709084+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13246464,"num_objects":3234,"num_object_clones":0,"num_object_copies":6468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15091,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-452}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17627","reported_seq":18470,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.778958+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:50.778958+0000","last_peered":"2024-09-10T16:00:50.778958+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:50.778958+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13217792,"num_objects":3227,"num_object_clones":0,"num_object_copies":6454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14819,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17561","reported_seq":18316,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.769532+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:50.769532+0000","last_peered":"2024-09-10T16:00:50.769532+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:50.769532+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12931072,"num_objects":3157,"num_object_clones":0,"num_object_copies":6314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13979,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17541","reported_seq":18373,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.767316+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:50.767316+0000","last_peered":"2024-09-10T16:00:50.767316+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:50.767316+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13209600,"num_objects":3225,"num_object_clones":0,"num_object_copies":6450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14147,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-311}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'17907","reported_seq":18674,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.706344+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:50.706344+0000","last_peered":"2024-09-10T16:00:50.706344+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:50.706344+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13070819,"num_objects":3192,"num_object_clones":0,"num_object_copies":6384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14726,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":208175611,"num_objects":50826,"num_object_clones":0,"num_object_copies":101652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242224,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14594,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606691328,"data_stored":606614537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160703,"ondisk_log_size":160703,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626527,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549348,"kb_used_data":152588,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93822492,"statfs":{"total":96636764160,"available":96074231808,"internally_reserved":0,"allocated":156250112,"data_stored":155901239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":406253230},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149809,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643932,"kb_used_data":215228,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93727908,"statfs":{"total":96636764160,"available":95977377792,"internally_reserved":0,"allocated":220393472,"data_stored":220106479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":438955693},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574072,"kb_used_data":227304,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93797768,"statfs":{"total":96636764160,"available":96048914432,"internally_reserved":0,"allocated":232759296,"data_stored":232468231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":219242496,"data_stored":219238883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":155492352,"data_stored":155426859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":231956480,"data_stored":231948795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:53.405 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:53.997 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:53.997 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:54.023 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-10T16:00:52.603040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":208175630,"num_objects":50828,"num_object_clones":0,"num_object_copies":101656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50828,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242226,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14596,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160706,"ondisk_log_size":160706,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1767352,"kb_used_data":595120,"kb_used_omap":12,"kb_used_meta":1172147,"kb_avail":281348168,"statfs":{"total":289910292480,"available":288100524032,"internally_reserved":0,"allocated":609402880,"data_stored":608475949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1200278533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,15,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28950528,"num_objects":-7068,"num_object_clones":0,"num_object_copies":-14136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7068,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004765"},"pg_stats":[{"pgid":"2.f","version":"50'17602","reported_seq":18940,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.743325+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:00:50.743325+0000","last_peered":"2024-09-10T16:00:50.743325+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:50.743325+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12976128,"num_objects":3168,"num_object_clones":0,"num_object_copies":6336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17602,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-178}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'17927","reported_seq":18044,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.790057+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:50.790057+0000","last_peered":"2024-09-10T16:00:50.790057+0000","last_clean":"2024-09-10T16:00:50.790057+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:50.790057+0000","last_undegraded":"2024-09-10T16:00:50.790057+0000","last_fullsized":"2024-09-10T16:00:50.790057+0000","mapping_epoch":36,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12890112,"num_objects":3147,"num_object_clones":0,"num_object_copies":6294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16263,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17449","reported_seq":17929,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.743861+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:50.743861+0000","last_peered":"2024-09-10T16:00:50.743861+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:50.743861+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13025304,"num_objects":3181,"num_object_clones":0,"num_object_copies":6362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15717,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-448}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17456","reported_seq":17545,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.758150+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:50.758150+0000","last_peered":"2024-09-10T16:00:50.758150+0000","last_clean":"2024-09-10T16:00:50.758150+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:50.758150+0000","last_undegraded":"2024-09-10T16:00:50.758150+0000","last_fullsized":"2024-09-10T16:00:50.758150+0000","mapping_epoch":36,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13213696,"num_objects":3226,"num_object_clones":0,"num_object_copies":6452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17456,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17227","reported_seq":17363,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.758503+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:50.758503+0000","last_peered":"2024-09-10T16:00:50.758503+0000","last_clean":"2024-09-10T16:00:50.758503+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:50.758503+0000","last_undegraded":"2024-09-10T16:00:50.758503+0000","last_fullsized":"2024-09-10T16:00:50.758503+0000","mapping_epoch":35,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12713984,"num_objects":3104,"num_object_clones":0,"num_object_copies":6208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14298,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'17720","reported_seq":25618,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:51.348860+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:51.348860+0000","last_peered":"2024-09-10T16:00:51.348860+0000","last_clean":"2024-09-10T16:00:51.348860+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:51.348860+0000","last_undegraded":"2024-09-10T16:00:51.348860+0000","last_fullsized":"2024-09-10T16:00:51.348860+0000","mapping_epoch":47,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12890112,"num_objects":3147,"num_object_clones":0,"num_object_copies":6294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14438,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17188","reported_seq":17958,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.713633+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:50.713633+0000","last_peered":"2024-09-10T16:00:50.713633+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:50.713633+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12689408,"num_objects":3098,"num_object_clones":0,"num_object_copies":6196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13838,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17700","reported_seq":17841,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.669251+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:50.669251+0000","last_peered":"2024-09-10T16:00:50.669251+0000","last_clean":"2024-09-10T16:00:50.669251+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:50.669251+0000","last_undegraded":"2024-09-10T16:00:50.669251+0000","last_fullsized":"2024-09-10T16:00:50.669251+0000","mapping_epoch":35,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13164544,"num_objects":3214,"num_object_clones":0,"num_object_copies":6428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14459,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17578","reported_seq":25715,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:51.287160+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:51.287160+0000","last_peered":"2024-09-10T16:00:51.287160+0000","last_clean":"2024-09-10T16:00:51.287160+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:51.287160+0000","last_undegraded":"2024-09-10T16:00:51.287160+0000","last_fullsized":"2024-09-10T16:00:51.287160+0000","mapping_epoch":45,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12865536,"num_objects":3141,"num_object_clones":0,"num_object_copies":6282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15555,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17422","reported_seq":18247,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.758128+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:50.758128+0000","last_peered":"2024-09-10T16:00:50.758128+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:50.758128+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12926976,"num_objects":3156,"num_object_clones":0,"num_object_copies":6312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14286,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-363}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17576","reported_seq":25384,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.760252+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:00:50.760252+0000","last_peered":"2024-09-10T16:00:50.760252+0000","last_clean":"2024-09-10T16:00:50.760252+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:00:50.760252+0000","last_undegraded":"2024-09-10T16:00:50.760252+0000","last_fullsized":"2024-09-10T16:00:50.760252+0000","mapping_epoch":49,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13144064,"num_objects":3209,"num_object_clones":0,"num_object_copies":6418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15550,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17222","reported_seq":17644,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.709084+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:50.709084+0000","last_peered":"2024-09-10T16:00:50.709084+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:50.709084+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13246464,"num_objects":3234,"num_object_clones":0,"num_object_copies":6468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15091,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-452}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17627","reported_seq":18470,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.778958+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:50.778958+0000","last_peered":"2024-09-10T16:00:50.778958+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:50.778958+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13217792,"num_objects":3227,"num_object_clones":0,"num_object_copies":6454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14819,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17561","reported_seq":18316,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.769532+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:50.769532+0000","last_peered":"2024-09-10T16:00:50.769532+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:50.769532+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12931072,"num_objects":3157,"num_object_clones":0,"num_object_copies":6314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13979,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17541","reported_seq":18373,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.767316+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:50.767316+0000","last_peered":"2024-09-10T16:00:50.767316+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:50.767316+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13209600,"num_objects":3225,"num_object_clones":0,"num_object_copies":6450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14147,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-311}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'17907","reported_seq":18674,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.706344+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:50.706344+0000","last_peered":"2024-09-10T16:00:50.706344+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:50.706344+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13070819,"num_objects":3192,"num_object_clones":0,"num_object_copies":6384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14726,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":208175611,"num_objects":50826,"num_object_clones":0,"num_object_copies":101652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242224,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14594,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606691328,"data_stored":606614537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160703,"ondisk_log_size":160703,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626527,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549348,"kb_used_data":152588,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93822492,"statfs":{"total":96636764160,"available":96074231808,"internally_reserved":0,"allocated":156250112,"data_stored":155901239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":406253230},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149809,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643932,"kb_used_data":215228,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93727908,"statfs":{"total":96636764160,"available":95977377792,"internally_reserved":0,"allocated":220393472,"data_stored":220106479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":438955693},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574072,"kb_used_data":227304,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93797768,"statfs":{"total":96636764160,"available":96048914432,"internally_reserved":0,"allocated":232759296,"data_stored":232468231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":219242496,"data_stored":219238883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":155492352,"data_stored":155426859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":231956480,"data_stored":231948795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:54.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:54.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'17602", 'reported_seq': 18940, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:50.743325+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:00:50.743325+0000', 'last_peered': '2024-09-10T16:00:50.743325+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:00:50.743325+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'7600", 'ondisk_log_start': "44'7600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12976128, 'num_objects': 3168, 'num_object_clones': 0, 'num_object_copies': 6336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3346, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17602, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 627, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -178}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:54.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:54.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'17449", 'reported_seq': 17929, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:50.743861+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:00:50.743861+0000', 'last_peered': '2024-09-10T16:00:50.743861+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:00:50.743861+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'7400", 'ondisk_log_start': "44'7400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13025304, 'num_objects': 3181, 'num_object_clones': 0, 'num_object_copies': 6362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3181, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15717, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -448}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:54.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:54.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'17188", 'reported_seq': 17958, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:50.713633+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:00:50.713633+0000', 'last_peered': '2024-09-10T16:00:50.713633+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:00:50.713633+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'7100", 'ondisk_log_start': "44'7100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12689408, 'num_objects': 3098, 'num_object_clones': 0, 'num_object_copies': 6196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3098, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13838, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -426}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:54.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:54.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'17422", 'reported_seq': 18247, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:50.758128+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:00:50.758128+0000', 'last_peered': '2024-09-10T16:00:50.758128+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:00:50.758128+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'7400", 'ondisk_log_start': "44'7400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12926976, 'num_objects': 3156, 'num_object_clones': 0, 'num_object_copies': 6312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14286, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -363}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:54.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:54.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'17222", 'reported_seq': 17644, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:50.709084+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:00:50.709084+0000', 'last_peered': '2024-09-10T16:00:50.709084+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:00:50.709084+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'7200", 'ondisk_log_start': "44'7200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13246464, 'num_objects': 3234, 'num_object_clones': 0, 'num_object_copies': 6468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3234, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15091, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -452}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:54.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:54.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'17627", 'reported_seq': 18470, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:50.778958+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:00:50.778958+0000', 'last_peered': '2024-09-10T16:00:50.778958+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:00:50.778958+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'7600", 'ondisk_log_start': "44'7600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13217792, 'num_objects': 3227, 'num_object_clones': 0, 'num_object_copies': 6454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3227, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14819, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -251}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:54.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:54.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'17561", 'reported_seq': 18316, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:50.769532+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:00:50.769532+0000', 'last_peered': '2024-09-10T16:00:50.769532+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:00:50.769532+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'7500", 'ondisk_log_start': "44'7500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12931072, 'num_objects': 3157, 'num_object_clones': 0, 'num_object_copies': 6314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3157, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13979, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:54.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:54.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'17541", 'reported_seq': 18373, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:50.767316+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:00:50.767316+0000', 'last_peered': '2024-09-10T16:00:50.767316+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:00:50.767316+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'7500", 'ondisk_log_start': "44'7500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13209600, 'num_objects': 3225, 'num_object_clones': 0, 'num_object_copies': 6450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3225, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14147, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -311}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:54.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:54.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'17907", 'reported_seq': 18674, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:50.706344+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:00:50.706344+0000', 'last_peered': '2024-09-10T16:00:50.706344+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:00:50.706344+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'7900", 'ondisk_log_start': "44'7900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13070819, 'num_objects': 3192, 'num_object_clones': 0, 'num_object_copies': 6384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3192, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14726, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -502}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:54.026 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:00:54.724 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:54.724 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32054/101656 objects degraded (31.532%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":580,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":50828,"data_bytes":208175630,"bytes_used":1809768448,"bytes_avail":288100524032,"bytes_total":289910292480,"degraded_objects":32054,"degraded_total":101656,"degraded_ratio":0.31531832848036517,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":588},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-10T16:00:44.600014+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:00:54.745 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:00:54.745 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:55.298 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:55.300 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:55.319 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-10T16:00:54.603844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":208175630,"num_objects":50828,"num_object_clones":0,"num_object_copies":101656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50828,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242226,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14596,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160706,"ondisk_log_size":160706,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1767352,"kb_used_data":595120,"kb_used_omap":12,"kb_used_meta":1172147,"kb_avail":281348168,"statfs":{"total":289910292480,"available":288100524032,"internally_reserved":0,"allocated":609402880,"data_stored":608475949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1200278533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,15,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19746816,"num_objects":-4821,"num_object_clones":0,"num_object_copies":-9642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4821,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004749"},"pg_stats":[{"pgid":"2.f","version":"50'17602","reported_seq":18940,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.743325+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:00:50.743325+0000","last_peered":"2024-09-10T16:00:50.743325+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:50.743325+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12976128,"num_objects":3168,"num_object_clones":0,"num_object_copies":6336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17602,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-178}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'17927","reported_seq":18044,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.790057+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:50.790057+0000","last_peered":"2024-09-10T16:00:50.790057+0000","last_clean":"2024-09-10T16:00:50.790057+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:50.790057+0000","last_undegraded":"2024-09-10T16:00:50.790057+0000","last_fullsized":"2024-09-10T16:00:50.790057+0000","mapping_epoch":36,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12890112,"num_objects":3147,"num_object_clones":0,"num_object_copies":6294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16263,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17449","reported_seq":17929,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.743861+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:50.743861+0000","last_peered":"2024-09-10T16:00:50.743861+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:50.743861+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13025304,"num_objects":3181,"num_object_clones":0,"num_object_copies":6362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15717,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-448}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17456","reported_seq":17545,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.758150+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:50.758150+0000","last_peered":"2024-09-10T16:00:50.758150+0000","last_clean":"2024-09-10T16:00:50.758150+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:50.758150+0000","last_undegraded":"2024-09-10T16:00:50.758150+0000","last_fullsized":"2024-09-10T16:00:50.758150+0000","mapping_epoch":36,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13213696,"num_objects":3226,"num_object_clones":0,"num_object_copies":6452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17456,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17227","reported_seq":17363,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.758503+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:50.758503+0000","last_peered":"2024-09-10T16:00:50.758503+0000","last_clean":"2024-09-10T16:00:50.758503+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:50.758503+0000","last_undegraded":"2024-09-10T16:00:50.758503+0000","last_fullsized":"2024-09-10T16:00:50.758503+0000","mapping_epoch":35,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12713984,"num_objects":3104,"num_object_clones":0,"num_object_copies":6208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14298,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'17720","reported_seq":25618,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:51.348860+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:51.348860+0000","last_peered":"2024-09-10T16:00:51.348860+0000","last_clean":"2024-09-10T16:00:51.348860+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:51.348860+0000","last_undegraded":"2024-09-10T16:00:51.348860+0000","last_fullsized":"2024-09-10T16:00:51.348860+0000","mapping_epoch":47,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12890112,"num_objects":3147,"num_object_clones":0,"num_object_copies":6294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14438,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17188","reported_seq":17958,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.713633+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:50.713633+0000","last_peered":"2024-09-10T16:00:50.713633+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:50.713633+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7100","ondisk_log_start":"44'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12689408,"num_objects":3098,"num_object_clones":0,"num_object_copies":6196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13838,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17700","reported_seq":17841,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.669251+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:50.669251+0000","last_peered":"2024-09-10T16:00:50.669251+0000","last_clean":"2024-09-10T16:00:50.669251+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:50.669251+0000","last_undegraded":"2024-09-10T16:00:50.669251+0000","last_fullsized":"2024-09-10T16:00:50.669251+0000","mapping_epoch":35,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13164544,"num_objects":3214,"num_object_clones":0,"num_object_copies":6428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14459,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17578","reported_seq":25715,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:51.287160+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:51.287160+0000","last_peered":"2024-09-10T16:00:51.287160+0000","last_clean":"2024-09-10T16:00:51.287160+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:51.287160+0000","last_undegraded":"2024-09-10T16:00:51.287160+0000","last_fullsized":"2024-09-10T16:00:51.287160+0000","mapping_epoch":45,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12865536,"num_objects":3141,"num_object_clones":0,"num_object_copies":6282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15555,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17422","reported_seq":18247,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.758128+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:50.758128+0000","last_peered":"2024-09-10T16:00:50.758128+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:50.758128+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12926976,"num_objects":3156,"num_object_clones":0,"num_object_copies":6312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14286,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-363}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17576","reported_seq":25384,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:50.760252+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:00:50.760252+0000","last_peered":"2024-09-10T16:00:50.760252+0000","last_clean":"2024-09-10T16:00:50.760252+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:00:50.760252+0000","last_undegraded":"2024-09-10T16:00:50.760252+0000","last_fullsized":"2024-09-10T16:00:50.760252+0000","mapping_epoch":49,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13144064,"num_objects":3209,"num_object_clones":0,"num_object_copies":6418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15550,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17222","reported_seq":17644,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.709084+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:50.709084+0000","last_peered":"2024-09-10T16:00:50.709084+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:50.709084+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7200","ondisk_log_start":"44'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13246464,"num_objects":3234,"num_object_clones":0,"num_object_copies":6468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15091,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-452}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17627","reported_seq":18470,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.778958+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:50.778958+0000","last_peered":"2024-09-10T16:00:50.778958+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:50.778958+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13217792,"num_objects":3227,"num_object_clones":0,"num_object_copies":6454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14819,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17561","reported_seq":18316,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.769532+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:50.769532+0000","last_peered":"2024-09-10T16:00:50.769532+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:50.769532+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12931072,"num_objects":3157,"num_object_clones":0,"num_object_copies":6314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13979,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17541","reported_seq":18373,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.767316+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:50.767316+0000","last_peered":"2024-09-10T16:00:50.767316+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:50.767316+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13209600,"num_objects":3225,"num_object_clones":0,"num_object_copies":6450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14147,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-311}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'17907","reported_seq":18674,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:50.706344+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:50.706344+0000","last_peered":"2024-09-10T16:00:50.706344+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:50.706344+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13070819,"num_objects":3192,"num_object_clones":0,"num_object_copies":6384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14726,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":208175611,"num_objects":50826,"num_object_clones":0,"num_object_copies":101652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242224,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14594,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606691328,"data_stored":606614537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160703,"ondisk_log_size":160703,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626527,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549348,"kb_used_data":152588,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93822492,"statfs":{"total":96636764160,"available":96074231808,"internally_reserved":0,"allocated":156250112,"data_stored":155901239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":406253230},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149809,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643932,"kb_used_data":215228,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93727908,"statfs":{"total":96636764160,"available":95977377792,"internally_reserved":0,"allocated":220393472,"data_stored":220106479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":438955693},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574072,"kb_used_data":227304,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93797768,"statfs":{"total":96636764160,"available":96048914432,"internally_reserved":0,"allocated":232759296,"data_stored":232468231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":219242496,"data_stored":219238883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":155492352,"data_stored":155426859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":231956480,"data_stored":231948795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:58.322 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:58.892 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:58.894 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:58.916 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-10T16:00:58.605486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":198431246,"num_objects":48449,"num_object_clones":0,"num_object_copies":96898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48449,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244605,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14686,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160785,"ondisk_log_size":160785,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1802660,"kb_used_data":576288,"kb_used_omap":12,"kb_used_meta":1226291,"kb_avail":281312860,"statfs":{"total":289910292480,"available":288064368640,"internally_reserved":0,"allocated":590118912,"data_stored":589191981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1255721989},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,2,7,16,5,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19509248,"num_objects":-4763,"num_object_clones":0,"num_object_copies":-9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4763,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004711"},"pg_stats":[{"pgid":"2.f","version":"50'17768","reported_seq":19288,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.848236+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:00:55.848236+0000","last_peered":"2024-09-10T16:00:55.848236+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:55.848236+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12296192,"num_objects":3002,"num_object_clones":0,"num_object_copies":6004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17768,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-254}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18071","reported_seq":18188,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.732890+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:55.732890+0000","last_peered":"2024-09-10T16:00:55.732890+0000","last_clean":"2024-09-10T16:00:55.732890+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:55.732890+0000","last_undegraded":"2024-09-10T16:00:55.732890+0000","last_fullsized":"2024-09-10T16:00:55.732890+0000","mapping_epoch":36,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12300288,"num_objects":3003,"num_object_clones":0,"num_object_copies":6006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16407,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17615","reported_seq":18095,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.772683+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:55.772683+0000","last_peered":"2024-09-10T16:00:55.772683+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:55.772683+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12345368,"num_objects":3015,"num_object_clones":0,"num_object_copies":6030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3015,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15883,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-614}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17615","reported_seq":17704,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.744049+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:55.744049+0000","last_peered":"2024-09-10T16:00:55.744049+0000","last_clean":"2024-09-10T16:00:55.744049+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:55.744049+0000","last_undegraded":"2024-09-10T16:00:55.744049+0000","last_fullsized":"2024-09-10T16:00:55.744049+0000","mapping_epoch":36,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12562432,"num_objects":3067,"num_object_clones":0,"num_object_copies":6134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17615,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17347","reported_seq":17483,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.789543+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:55.789543+0000","last_peered":"2024-09-10T16:00:55.789543+0000","last_clean":"2024-09-10T16:00:55.789543+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:55.789543+0000","last_undegraded":"2024-09-10T16:00:55.789543+0000","last_fullsized":"2024-09-10T16:00:55.789543+0000","mapping_epoch":35,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12222464,"num_objects":2984,"num_object_clones":0,"num_object_copies":5968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14418,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'17890","reported_seq":25788,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:56.350025+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:56.350025+0000","last_peered":"2024-09-10T16:00:56.350025+0000","last_clean":"2024-09-10T16:00:56.350025+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:56.350025+0000","last_undegraded":"2024-09-10T16:00:56.350025+0000","last_fullsized":"2024-09-10T16:00:56.350025+0000","mapping_epoch":47,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12193792,"num_objects":2977,"num_object_clones":0,"num_object_copies":5954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14608,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17334","reported_seq":18104,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.766038+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:55.766038+0000","last_peered":"2024-09-10T16:00:55.766038+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:55.766038+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12091392,"num_objects":2952,"num_object_clones":0,"num_object_copies":5904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13984,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-572}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17825","reported_seq":17966,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.825558+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:55.825558+0000","last_peered":"2024-09-10T16:00:55.825558+0000","last_clean":"2024-09-10T16:00:55.825558+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:55.825558+0000","last_undegraded":"2024-09-10T16:00:55.825558+0000","last_fullsized":"2024-09-10T16:00:55.825558+0000","mapping_epoch":35,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12652544,"num_objects":3089,"num_object_clones":0,"num_object_copies":6178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14584,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17721","reported_seq":25856,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:56.352424+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:56.352424+0000","last_peered":"2024-09-10T16:00:56.352424+0000","last_clean":"2024-09-10T16:00:56.352424+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:56.352424+0000","last_undegraded":"2024-09-10T16:00:56.352424+0000","last_fullsized":"2024-09-10T16:00:56.352424+0000","mapping_epoch":45,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12279808,"num_objects":2998,"num_object_clones":0,"num_object_copies":5996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15698,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17566","reported_seq":18391,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.701350+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:55.701350+0000","last_peered":"2024-09-10T16:00:55.701350+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:55.701350+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12337152,"num_objects":3012,"num_object_clones":0,"num_object_copies":6024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14430,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-507}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17719","reported_seq":25529,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.782786+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:00:55.782786+0000","last_peered":"2024-09-10T16:00:55.782786+0000","last_clean":"2024-09-10T16:00:55.782786+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:00:55.782786+0000","last_undegraded":"2024-09-10T16:00:55.782786+0000","last_fullsized":"2024-09-10T16:00:55.782786+0000","mapping_epoch":49,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12558336,"num_objects":3066,"num_object_clones":0,"num_object_copies":6132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15693,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17375","reported_seq":17798,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.809895+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:55.809895+0000","last_peered":"2024-09-10T16:00:55.809895+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:55.809895+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12619776,"num_objects":3081,"num_object_clones":0,"num_object_copies":6162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15244,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17780","reported_seq":18626,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.838060+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:55.838060+0000","last_peered":"2024-09-10T16:00:55.838060+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:55.838060+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12591104,"num_objects":3074,"num_object_clones":0,"num_object_copies":6148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14972,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17707","reported_seq":18463,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.836416+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:55.836416+0000","last_peered":"2024-09-10T16:00:55.836416+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:55.836416+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12333056,"num_objects":3011,"num_object_clones":0,"num_object_copies":6022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14125,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17693","reported_seq":18525,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.774962+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:55.774962+0000","last_peered":"2024-09-10T16:00:55.774962+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:55.774962+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12587008,"num_objects":3073,"num_object_clones":0,"num_object_copies":6146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14299,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18056","reported_seq":18822,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.809145+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:55.809145+0000","last_peered":"2024-09-10T16:00:55.809145+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:55.809145+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12460515,"num_objects":3043,"num_object_clones":0,"num_object_copies":6086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14875,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-651}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":198431227,"num_objects":48447,"num_object_clones":0,"num_object_copies":96894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244603,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14684,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":587292672,"data_stored":587215881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626528,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586500,"kb_used_data":145316,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93785340,"statfs":{"total":96636764160,"available":96036188160,"internally_reserved":0,"allocated":148803584,"data_stored":148454711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":451735214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149810,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659132,"kb_used_data":208232,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93712708,"statfs":{"total":96636764160,"available":95961812992,"internally_reserved":0,"allocated":213229568,"data_stored":212942575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":461696685},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,10,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953581,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557028,"kb_used_data":222740,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93814812,"statfs":{"total":96636764160,"available":96066367488,"internally_reserved":0,"allocated":228085760,"data_stored":227794695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,5,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":211951616,"data_stored":211948003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":148062208,"data_stored":147996715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":227278848,"data_stored":227271163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:58.917 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:00:59.469 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:00:59.469 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:00:59.500 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-10T16:00:58.605486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":198431246,"num_objects":48449,"num_object_clones":0,"num_object_copies":96898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48449,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244605,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14686,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160785,"ondisk_log_size":160785,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1802660,"kb_used_data":576288,"kb_used_omap":12,"kb_used_meta":1226291,"kb_avail":281312860,"statfs":{"total":289910292480,"available":288064368640,"internally_reserved":0,"allocated":590118912,"data_stored":589191981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1255721989},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,2,7,16,5,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19509248,"num_objects":-4763,"num_object_clones":0,"num_object_copies":-9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4763,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004711"},"pg_stats":[{"pgid":"2.f","version":"50'17768","reported_seq":19288,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.848236+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:00:55.848236+0000","last_peered":"2024-09-10T16:00:55.848236+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:55.848236+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12296192,"num_objects":3002,"num_object_clones":0,"num_object_copies":6004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17768,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-254}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18071","reported_seq":18188,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.732890+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:55.732890+0000","last_peered":"2024-09-10T16:00:55.732890+0000","last_clean":"2024-09-10T16:00:55.732890+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:55.732890+0000","last_undegraded":"2024-09-10T16:00:55.732890+0000","last_fullsized":"2024-09-10T16:00:55.732890+0000","mapping_epoch":36,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12300288,"num_objects":3003,"num_object_clones":0,"num_object_copies":6006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16407,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17615","reported_seq":18095,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.772683+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:55.772683+0000","last_peered":"2024-09-10T16:00:55.772683+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:55.772683+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12345368,"num_objects":3015,"num_object_clones":0,"num_object_copies":6030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3015,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15883,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-614}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17615","reported_seq":17704,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.744049+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:55.744049+0000","last_peered":"2024-09-10T16:00:55.744049+0000","last_clean":"2024-09-10T16:00:55.744049+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:55.744049+0000","last_undegraded":"2024-09-10T16:00:55.744049+0000","last_fullsized":"2024-09-10T16:00:55.744049+0000","mapping_epoch":36,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12562432,"num_objects":3067,"num_object_clones":0,"num_object_copies":6134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17615,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17347","reported_seq":17483,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.789543+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:55.789543+0000","last_peered":"2024-09-10T16:00:55.789543+0000","last_clean":"2024-09-10T16:00:55.789543+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:55.789543+0000","last_undegraded":"2024-09-10T16:00:55.789543+0000","last_fullsized":"2024-09-10T16:00:55.789543+0000","mapping_epoch":35,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12222464,"num_objects":2984,"num_object_clones":0,"num_object_copies":5968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14418,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'17890","reported_seq":25788,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:56.350025+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:56.350025+0000","last_peered":"2024-09-10T16:00:56.350025+0000","last_clean":"2024-09-10T16:00:56.350025+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:56.350025+0000","last_undegraded":"2024-09-10T16:00:56.350025+0000","last_fullsized":"2024-09-10T16:00:56.350025+0000","mapping_epoch":47,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12193792,"num_objects":2977,"num_object_clones":0,"num_object_copies":5954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14608,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17334","reported_seq":18104,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.766038+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:55.766038+0000","last_peered":"2024-09-10T16:00:55.766038+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:55.766038+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12091392,"num_objects":2952,"num_object_clones":0,"num_object_copies":5904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13984,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-572}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17825","reported_seq":17966,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.825558+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:55.825558+0000","last_peered":"2024-09-10T16:00:55.825558+0000","last_clean":"2024-09-10T16:00:55.825558+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:55.825558+0000","last_undegraded":"2024-09-10T16:00:55.825558+0000","last_fullsized":"2024-09-10T16:00:55.825558+0000","mapping_epoch":35,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12652544,"num_objects":3089,"num_object_clones":0,"num_object_copies":6178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14584,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17721","reported_seq":25856,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:56.352424+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:56.352424+0000","last_peered":"2024-09-10T16:00:56.352424+0000","last_clean":"2024-09-10T16:00:56.352424+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:56.352424+0000","last_undegraded":"2024-09-10T16:00:56.352424+0000","last_fullsized":"2024-09-10T16:00:56.352424+0000","mapping_epoch":45,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12279808,"num_objects":2998,"num_object_clones":0,"num_object_copies":5996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15698,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17566","reported_seq":18391,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.701350+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:55.701350+0000","last_peered":"2024-09-10T16:00:55.701350+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:55.701350+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12337152,"num_objects":3012,"num_object_clones":0,"num_object_copies":6024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14430,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-507}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17719","reported_seq":25529,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.782786+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:00:55.782786+0000","last_peered":"2024-09-10T16:00:55.782786+0000","last_clean":"2024-09-10T16:00:55.782786+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:00:55.782786+0000","last_undegraded":"2024-09-10T16:00:55.782786+0000","last_fullsized":"2024-09-10T16:00:55.782786+0000","mapping_epoch":49,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12558336,"num_objects":3066,"num_object_clones":0,"num_object_copies":6132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15693,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17375","reported_seq":17798,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.809895+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:55.809895+0000","last_peered":"2024-09-10T16:00:55.809895+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:55.809895+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12619776,"num_objects":3081,"num_object_clones":0,"num_object_copies":6162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15244,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17780","reported_seq":18626,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.838060+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:55.838060+0000","last_peered":"2024-09-10T16:00:55.838060+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:55.838060+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12591104,"num_objects":3074,"num_object_clones":0,"num_object_copies":6148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14972,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17707","reported_seq":18463,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.836416+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:55.836416+0000","last_peered":"2024-09-10T16:00:55.836416+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:55.836416+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12333056,"num_objects":3011,"num_object_clones":0,"num_object_copies":6022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14125,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17693","reported_seq":18525,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.774962+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:55.774962+0000","last_peered":"2024-09-10T16:00:55.774962+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:55.774962+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12587008,"num_objects":3073,"num_object_clones":0,"num_object_copies":6146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14299,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18056","reported_seq":18822,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.809145+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:55.809145+0000","last_peered":"2024-09-10T16:00:55.809145+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:55.809145+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12460515,"num_objects":3043,"num_object_clones":0,"num_object_copies":6086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14875,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-651}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":198431227,"num_objects":48447,"num_object_clones":0,"num_object_copies":96894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244603,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14684,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":587292672,"data_stored":587215881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626528,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586500,"kb_used_data":145316,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93785340,"statfs":{"total":96636764160,"available":96036188160,"internally_reserved":0,"allocated":148803584,"data_stored":148454711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":451735214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149810,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659132,"kb_used_data":208232,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93712708,"statfs":{"total":96636764160,"available":95961812992,"internally_reserved":0,"allocated":213229568,"data_stored":212942575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":461696685},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,10,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953581,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557028,"kb_used_data":222740,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93814812,"statfs":{"total":96636764160,"available":96066367488,"internally_reserved":0,"allocated":228085760,"data_stored":227794695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,5,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":211951616,"data_stored":211948003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":148062208,"data_stored":147996715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":227278848,"data_stored":227271163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:00:59.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:00:59.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'17768", 'reported_seq': 19288, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:55.848236+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:00:55.848236+0000', 'last_peered': '2024-09-10T16:00:55.848236+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:00:55.848236+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'7700", 'ondisk_log_start': "44'7700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12296192, 'num_objects': 3002, 'num_object_clones': 0, 'num_object_copies': 6004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3256, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17768, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 717, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -254}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:00:59.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'17615", 'reported_seq': 18095, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:55.772683+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:00:55.772683+0000', 'last_peered': '2024-09-10T16:00:55.772683+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:00:55.772683+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'7600", 'ondisk_log_start': "44'7600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12345368, 'num_objects': 3015, 'num_object_clones': 0, 'num_object_copies': 6030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3015, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15883, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:59.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:00:59.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'17334", 'reported_seq': 18104, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:55.766038+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:00:55.766038+0000', 'last_peered': '2024-09-10T16:00:55.766038+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:00:55.766038+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'7300", 'ondisk_log_start': "44'7300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12091392, 'num_objects': 2952, 'num_object_clones': 0, 'num_object_copies': 5904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13984, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -572}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:00:59.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'17566", 'reported_seq': 18391, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:55.701350+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:00:55.701350+0000', 'last_peered': '2024-09-10T16:00:55.701350+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:00:55.701350+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'7500", 'ondisk_log_start': "44'7500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12337152, 'num_objects': 3012, 'num_object_clones': 0, 'num_object_copies': 6024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3012, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14430, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -507}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:00:59.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'17375", 'reported_seq': 17798, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:55.809895+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:00:55.809895+0000', 'last_peered': '2024-09-10T16:00:55.809895+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:00:55.809895+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'7300", 'ondisk_log_start': "44'7300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12619776, 'num_objects': 3081, 'num_object_clones': 0, 'num_object_copies': 6162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3081, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15244, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -605}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:00:59.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:00:59.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'17780", 'reported_seq': 18626, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:55.838060+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:00:55.838060+0000', 'last_peered': '2024-09-10T16:00:55.838060+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:00:55.838060+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'7700", 'ondisk_log_start': "44'7700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12591104, 'num_objects': 3074, 'num_object_clones': 0, 'num_object_copies': 6148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3074, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14972, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -404}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:00:59.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'17707", 'reported_seq': 18463, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:55.836416+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:00:55.836416+0000', 'last_peered': '2024-09-10T16:00:55.836416+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:00:55.836416+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'7700", 'ondisk_log_start': "44'7700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12333056, 'num_objects': 3011, 'num_object_clones': 0, 'num_object_copies': 6022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3011, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14125, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -615}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:00:59.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'17693", 'reported_seq': 18525, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:55.774962+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:00:55.774962+0000', 'last_peered': '2024-09-10T16:00:55.774962+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:00:55.774962+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'7600", 'ondisk_log_start': "44'7600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12587008, 'num_objects': 3073, 'num_object_clones': 0, 'num_object_copies': 6146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3073, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14299, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -463}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.503 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:00:59.503 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'18056", 'reported_seq': 18822, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:00:55.809145+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:00:55.809145+0000', 'last_peered': '2024-09-10T16:00:55.809145+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:00:55.809145+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'8000", 'ondisk_log_start': "44'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12460515, 'num_objects': 3043, 'num_object_clones': 0, 'num_object_copies': 6086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3043, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14875, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -651}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:00:59.503 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:00.195 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:00.195 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31964/96898 objects degraded (32.987%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":586,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":48449,"data_bytes":198431246,"bytes_used":1845923840,"bytes_avail":288064368640,"bytes_total":289910292480,"degraded_objects":31964,"degraded_total":96898,"degraded_ratio":0.32987264959029083,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":167,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":396},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-10T16:00:44.600014+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:01:00.227 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:00.227 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:00.797 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:00.797 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:00.831 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-10T16:01:00.606118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":198431246,"num_objects":48449,"num_object_clones":0,"num_object_copies":96898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48449,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244605,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14686,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160785,"ondisk_log_size":160785,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1802660,"kb_used_data":576288,"kb_used_omap":12,"kb_used_meta":1226291,"kb_avail":281312860,"statfs":{"total":289910292480,"available":288064368640,"internally_reserved":0,"allocated":590118912,"data_stored":589191981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1255721989},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,2,7,16,5,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19509248,"num_objects":-4763,"num_object_clones":0,"num_object_copies":-9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4763,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004460"},"pg_stats":[{"pgid":"2.f","version":"50'17768","reported_seq":19288,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.848236+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:00:55.848236+0000","last_peered":"2024-09-10T16:00:55.848236+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:00:55.848236+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12296192,"num_objects":3002,"num_object_clones":0,"num_object_copies":6004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17768,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-254}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18071","reported_seq":18188,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.732890+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:00:55.732890+0000","last_peered":"2024-09-10T16:00:55.732890+0000","last_clean":"2024-09-10T16:00:55.732890+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:00:55.732890+0000","last_undegraded":"2024-09-10T16:00:55.732890+0000","last_fullsized":"2024-09-10T16:00:55.732890+0000","mapping_epoch":36,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12300288,"num_objects":3003,"num_object_clones":0,"num_object_copies":6006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16407,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17615","reported_seq":18095,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.772683+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:00:55.772683+0000","last_peered":"2024-09-10T16:00:55.772683+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:00:55.772683+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12345368,"num_objects":3015,"num_object_clones":0,"num_object_copies":6030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3015,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15883,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-614}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17615","reported_seq":17704,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.744049+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:00:55.744049+0000","last_peered":"2024-09-10T16:00:55.744049+0000","last_clean":"2024-09-10T16:00:55.744049+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:00:55.744049+0000","last_undegraded":"2024-09-10T16:00:55.744049+0000","last_fullsized":"2024-09-10T16:00:55.744049+0000","mapping_epoch":36,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12562432,"num_objects":3067,"num_object_clones":0,"num_object_copies":6134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17615,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17347","reported_seq":17483,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.789543+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:00:55.789543+0000","last_peered":"2024-09-10T16:00:55.789543+0000","last_clean":"2024-09-10T16:00:55.789543+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:00:55.789543+0000","last_undegraded":"2024-09-10T16:00:55.789543+0000","last_fullsized":"2024-09-10T16:00:55.789543+0000","mapping_epoch":35,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12222464,"num_objects":2984,"num_object_clones":0,"num_object_copies":5968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14418,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'17890","reported_seq":25788,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:56.350025+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:00:56.350025+0000","last_peered":"2024-09-10T16:00:56.350025+0000","last_clean":"2024-09-10T16:00:56.350025+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:00:56.350025+0000","last_undegraded":"2024-09-10T16:00:56.350025+0000","last_fullsized":"2024-09-10T16:00:56.350025+0000","mapping_epoch":47,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12193792,"num_objects":2977,"num_object_clones":0,"num_object_copies":5954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14608,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17334","reported_seq":18104,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.766038+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:00:55.766038+0000","last_peered":"2024-09-10T16:00:55.766038+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:00:55.766038+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12091392,"num_objects":2952,"num_object_clones":0,"num_object_copies":5904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13984,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-572}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17825","reported_seq":17966,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.825558+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:00:55.825558+0000","last_peered":"2024-09-10T16:00:55.825558+0000","last_clean":"2024-09-10T16:00:55.825558+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:00:55.825558+0000","last_undegraded":"2024-09-10T16:00:55.825558+0000","last_fullsized":"2024-09-10T16:00:55.825558+0000","mapping_epoch":35,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12652544,"num_objects":3089,"num_object_clones":0,"num_object_copies":6178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14584,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17721","reported_seq":25856,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:56.352424+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:00:56.352424+0000","last_peered":"2024-09-10T16:00:56.352424+0000","last_clean":"2024-09-10T16:00:56.352424+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:00:56.352424+0000","last_undegraded":"2024-09-10T16:00:56.352424+0000","last_fullsized":"2024-09-10T16:00:56.352424+0000","mapping_epoch":45,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12279808,"num_objects":2998,"num_object_clones":0,"num_object_copies":5996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15698,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17566","reported_seq":18391,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.701350+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:00:55.701350+0000","last_peered":"2024-09-10T16:00:55.701350+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:00:55.701350+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12337152,"num_objects":3012,"num_object_clones":0,"num_object_copies":6024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14430,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-507}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17719","reported_seq":25529,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:55.782786+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:00:55.782786+0000","last_peered":"2024-09-10T16:00:55.782786+0000","last_clean":"2024-09-10T16:00:55.782786+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:00:55.782786+0000","last_undegraded":"2024-09-10T16:00:55.782786+0000","last_fullsized":"2024-09-10T16:00:55.782786+0000","mapping_epoch":49,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12558336,"num_objects":3066,"num_object_clones":0,"num_object_copies":6132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15693,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17375","reported_seq":17798,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.809895+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:00:55.809895+0000","last_peered":"2024-09-10T16:00:55.809895+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:00:55.809895+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7300","ondisk_log_start":"44'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12619776,"num_objects":3081,"num_object_clones":0,"num_object_copies":6162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15244,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17780","reported_seq":18626,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.838060+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:00:55.838060+0000","last_peered":"2024-09-10T16:00:55.838060+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:00:55.838060+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12591104,"num_objects":3074,"num_object_clones":0,"num_object_copies":6148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14972,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17707","reported_seq":18463,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.836416+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:00:55.836416+0000","last_peered":"2024-09-10T16:00:55.836416+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:00:55.836416+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12333056,"num_objects":3011,"num_object_clones":0,"num_object_copies":6022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14125,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17693","reported_seq":18525,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.774962+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:00:55.774962+0000","last_peered":"2024-09-10T16:00:55.774962+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:00:55.774962+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12587008,"num_objects":3073,"num_object_clones":0,"num_object_copies":6146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14299,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18056","reported_seq":18822,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:00:55.809145+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:00:55.809145+0000","last_peered":"2024-09-10T16:00:55.809145+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:00:55.809145+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12460515,"num_objects":3043,"num_object_clones":0,"num_object_copies":6086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14875,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-651}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":198431227,"num_objects":48447,"num_object_clones":0,"num_object_copies":96894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244603,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14684,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":587292672,"data_stored":587215881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626528,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586500,"kb_used_data":145316,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93785340,"statfs":{"total":96636764160,"available":96036188160,"internally_reserved":0,"allocated":148803584,"data_stored":148454711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":451735214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149810,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659132,"kb_used_data":208232,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93712708,"statfs":{"total":96636764160,"available":95961812992,"internally_reserved":0,"allocated":213229568,"data_stored":212942575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":461696685},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,10,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953581,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557028,"kb_used_data":222740,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93814812,"statfs":{"total":96636764160,"available":96066367488,"internally_reserved":0,"allocated":228085760,"data_stored":227794695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,5,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":211951616,"data_stored":211948003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":148062208,"data_stored":147996715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":227278848,"data_stored":227271163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:03.833 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:04.407 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:04.408 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:04.435 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-10T16:01:02.606769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":189231630,"num_objects":46203,"num_object_clones":0,"num_object_copies":92406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46203,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246851,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14752,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1780252,"kb_used_data":558056,"kb_used_omap":12,"kb_used_meta":1222131,"kb_avail":281335268,"statfs":{"total":289910292480,"available":288087314432,"internally_reserved":0,"allocated":571449344,"data_stored":570522413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1251462149},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,3,8,13,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28708864,"num_objects":-7009,"num_object_clones":0,"num_object_copies":-14018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7009,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004336"},"pg_stats":[{"pgid":"2.f","version":"50'17906","reported_seq":19559,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.837033+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:00.837033+0000","last_peered":"2024-09-10T16:01:00.837033+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:00.837033+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11730944,"num_objects":2864,"num_object_clones":0,"num_object_copies":5728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17906,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":783,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-326}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18205","reported_seq":18322,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.828410+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:00.828410+0000","last_peered":"2024-09-10T16:01:00.828410+0000","last_clean":"2024-09-10T16:01:00.828410+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:00.828410+0000","last_undegraded":"2024-09-10T16:01:00.828410+0000","last_fullsized":"2024-09-10T16:01:00.828410+0000","mapping_epoch":36,"log_start":"44'8200","ondisk_log_start":"44'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11751424,"num_objects":2869,"num_object_clones":0,"num_object_copies":5738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16541,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17749","reported_seq":18229,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.744162+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:00.744162+0000","last_peered":"2024-09-10T16:01:00.744162+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:00.744162+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11796504,"num_objects":2881,"num_object_clones":0,"num_object_copies":5762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16017,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-748}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17763","reported_seq":17852,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.806592+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:00.806592+0000","last_peered":"2024-09-10T16:01:00.806592+0000","last_clean":"2024-09-10T16:01:00.806592+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:00.806592+0000","last_undegraded":"2024-09-10T16:01:00.806592+0000","last_fullsized":"2024-09-10T16:01:00.806592+0000","mapping_epoch":36,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11956224,"num_objects":2919,"num_object_clones":0,"num_object_copies":5838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17763,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17485","reported_seq":17619,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.760340+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:00.760340+0000","last_peered":"2024-09-10T16:01:00.760340+0000","last_clean":"2024-09-10T16:01:00.760340+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:00.760340+0000","last_undegraded":"2024-09-10T16:01:00.760340+0000","last_fullsized":"2024-09-10T16:01:00.760340+0000","mapping_epoch":35,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11657216,"num_objects":2846,"num_object_clones":0,"num_object_copies":5692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14556,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18030","reported_seq":25928,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:01.429888+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:01.429888+0000","last_peered":"2024-09-10T16:01:01.429888+0000","last_clean":"2024-09-10T16:01:01.429888+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:01.429888+0000","last_undegraded":"2024-09-10T16:01:01.429888+0000","last_fullsized":"2024-09-10T16:01:01.429888+0000","mapping_epoch":47,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11620352,"num_objects":2837,"num_object_clones":0,"num_object_copies":5674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14748,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17472","reported_seq":18242,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.867841+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:00.867841+0000","last_peered":"2024-09-10T16:01:00.867841+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:00.867841+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11526144,"num_objects":2814,"num_object_clones":0,"num_object_copies":5628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14122,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17968","reported_seq":18109,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.821391+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:00.821391+0000","last_peered":"2024-09-10T16:01:00.821391+0000","last_clean":"2024-09-10T16:01:00.821391+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:00.821391+0000","last_undegraded":"2024-09-10T16:01:00.821391+0000","last_fullsized":"2024-09-10T16:01:00.821391+0000","mapping_epoch":35,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12066816,"num_objects":2946,"num_object_clones":0,"num_object_copies":5892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14727,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17858","reported_seq":25995,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:01.378023+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:01.378023+0000","last_peered":"2024-09-10T16:01:01.378023+0000","last_clean":"2024-09-10T16:01:01.378023+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:01.378023+0000","last_undegraded":"2024-09-10T16:01:01.378023+0000","last_fullsized":"2024-09-10T16:01:01.378023+0000","mapping_epoch":45,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11718656,"num_objects":2861,"num_object_clones":0,"num_object_copies":5722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15835,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17700","reported_seq":18525,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.844079+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:00.844079+0000","last_peered":"2024-09-10T16:01:00.844079+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:00.844079+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11788288,"num_objects":2878,"num_object_clones":0,"num_object_copies":5756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14564,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-641}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17843","reported_seq":25653,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.754197+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:00.754197+0000","last_peered":"2024-09-10T16:01:00.754197+0000","last_clean":"2024-09-10T16:01:00.754197+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:00.754197+0000","last_undegraded":"2024-09-10T16:01:00.754197+0000","last_fullsized":"2024-09-10T16:01:00.754197+0000","mapping_epoch":49,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12050432,"num_objects":2942,"num_object_clones":0,"num_object_copies":5884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15817,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17550","reported_seq":17973,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.817921+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:00.817921+0000","last_peered":"2024-09-10T16:01:00.817921+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:00.817921+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11902976,"num_objects":2906,"num_object_clones":0,"num_object_copies":5812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15419,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-780}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17930","reported_seq":18777,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.879367+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:00.879367+0000","last_peered":"2024-09-10T16:01:00.879367+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:00.879367+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11976704,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15122,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17856","reported_seq":18612,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.624563+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:00.624563+0000","last_peered":"2024-09-10T16:01:00.624563+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:00.624563+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11722752,"num_objects":2862,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14274,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17828","reported_seq":18658,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.766631+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:00.766631+0000","last_peered":"2024-09-10T16:01:00.766631+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:00.766631+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12034048,"num_objects":2938,"num_object_clones":0,"num_object_copies":5876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14434,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18185","reported_seq":18952,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.802552+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:00.802552+0000","last_peered":"2024-09-10T16:01:00.802552+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:00.802552+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11932131,"num_objects":2914,"num_object_clones":0,"num_object_copies":5828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15004,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-780}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":189231611,"num_objects":46201,"num_object_clones":0,"num_object_copies":92402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246849,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14750,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":568823808,"data_stored":568747017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626529,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567200,"kb_used_data":138508,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93804640,"statfs":{"total":96636764160,"available":96055951360,"internally_reserved":0,"allocated":141832192,"data_stored":141483319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":438955694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149811,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656484,"kb_used_data":201428,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93715356,"statfs":{"total":96636764160,"available":95964524544,"internally_reserved":0,"allocated":206262272,"data_stored":205975279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":465956525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556568,"kb_used_data":218120,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93815272,"statfs":{"total":96636764160,"available":96066838528,"internally_reserved":0,"allocated":223354880,"data_stored":223063815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":205103104,"data_stored":205099491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":141094912,"data_stored":141029419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222625792,"data_stored":222618107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:04.437 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:04.992 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:04.995 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:05.013 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-10T16:01:04.607405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":189231630,"num_objects":46203,"num_object_clones":0,"num_object_copies":92406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46203,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246851,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14752,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1780252,"kb_used_data":558056,"kb_used_omap":12,"kb_used_meta":1222131,"kb_avail":281335268,"statfs":{"total":289910292480,"available":288087314432,"internally_reserved":0,"allocated":571449344,"data_stored":570522413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1251462149},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,3,8,13,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18944000,"num_objects":-4625,"num_object_clones":0,"num_object_copies":-9250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4625,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004365"},"pg_stats":[{"pgid":"2.f","version":"50'17906","reported_seq":19559,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.837033+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:00.837033+0000","last_peered":"2024-09-10T16:01:00.837033+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:00.837033+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11730944,"num_objects":2864,"num_object_clones":0,"num_object_copies":5728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17906,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":783,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-326}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18205","reported_seq":18322,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.828410+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:00.828410+0000","last_peered":"2024-09-10T16:01:00.828410+0000","last_clean":"2024-09-10T16:01:00.828410+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:00.828410+0000","last_undegraded":"2024-09-10T16:01:00.828410+0000","last_fullsized":"2024-09-10T16:01:00.828410+0000","mapping_epoch":36,"log_start":"44'8200","ondisk_log_start":"44'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11751424,"num_objects":2869,"num_object_clones":0,"num_object_copies":5738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16541,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17749","reported_seq":18229,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.744162+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:00.744162+0000","last_peered":"2024-09-10T16:01:00.744162+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:00.744162+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11796504,"num_objects":2881,"num_object_clones":0,"num_object_copies":5762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16017,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-748}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17763","reported_seq":17852,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.806592+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:00.806592+0000","last_peered":"2024-09-10T16:01:00.806592+0000","last_clean":"2024-09-10T16:01:00.806592+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:00.806592+0000","last_undegraded":"2024-09-10T16:01:00.806592+0000","last_fullsized":"2024-09-10T16:01:00.806592+0000","mapping_epoch":36,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11956224,"num_objects":2919,"num_object_clones":0,"num_object_copies":5838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17763,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17485","reported_seq":17619,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.760340+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:00.760340+0000","last_peered":"2024-09-10T16:01:00.760340+0000","last_clean":"2024-09-10T16:01:00.760340+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:00.760340+0000","last_undegraded":"2024-09-10T16:01:00.760340+0000","last_fullsized":"2024-09-10T16:01:00.760340+0000","mapping_epoch":35,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11657216,"num_objects":2846,"num_object_clones":0,"num_object_copies":5692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14556,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18030","reported_seq":25928,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:01.429888+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:01.429888+0000","last_peered":"2024-09-10T16:01:01.429888+0000","last_clean":"2024-09-10T16:01:01.429888+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:01.429888+0000","last_undegraded":"2024-09-10T16:01:01.429888+0000","last_fullsized":"2024-09-10T16:01:01.429888+0000","mapping_epoch":47,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11620352,"num_objects":2837,"num_object_clones":0,"num_object_copies":5674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14748,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17472","reported_seq":18242,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.867841+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:00.867841+0000","last_peered":"2024-09-10T16:01:00.867841+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:00.867841+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11526144,"num_objects":2814,"num_object_clones":0,"num_object_copies":5628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14122,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17968","reported_seq":18109,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.821391+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:00.821391+0000","last_peered":"2024-09-10T16:01:00.821391+0000","last_clean":"2024-09-10T16:01:00.821391+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:00.821391+0000","last_undegraded":"2024-09-10T16:01:00.821391+0000","last_fullsized":"2024-09-10T16:01:00.821391+0000","mapping_epoch":35,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12066816,"num_objects":2946,"num_object_clones":0,"num_object_copies":5892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14727,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17858","reported_seq":25995,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:01.378023+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:01.378023+0000","last_peered":"2024-09-10T16:01:01.378023+0000","last_clean":"2024-09-10T16:01:01.378023+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:01.378023+0000","last_undegraded":"2024-09-10T16:01:01.378023+0000","last_fullsized":"2024-09-10T16:01:01.378023+0000","mapping_epoch":45,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11718656,"num_objects":2861,"num_object_clones":0,"num_object_copies":5722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15835,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17700","reported_seq":18525,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.844079+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:00.844079+0000","last_peered":"2024-09-10T16:01:00.844079+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:00.844079+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11788288,"num_objects":2878,"num_object_clones":0,"num_object_copies":5756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14564,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-641}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17843","reported_seq":25653,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.754197+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:00.754197+0000","last_peered":"2024-09-10T16:01:00.754197+0000","last_clean":"2024-09-10T16:01:00.754197+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:00.754197+0000","last_undegraded":"2024-09-10T16:01:00.754197+0000","last_fullsized":"2024-09-10T16:01:00.754197+0000","mapping_epoch":49,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12050432,"num_objects":2942,"num_object_clones":0,"num_object_copies":5884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15817,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17550","reported_seq":17973,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.817921+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:00.817921+0000","last_peered":"2024-09-10T16:01:00.817921+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:00.817921+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11902976,"num_objects":2906,"num_object_clones":0,"num_object_copies":5812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15419,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-780}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17930","reported_seq":18777,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.879367+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:00.879367+0000","last_peered":"2024-09-10T16:01:00.879367+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:00.879367+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11976704,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15122,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17856","reported_seq":18612,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.624563+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:00.624563+0000","last_peered":"2024-09-10T16:01:00.624563+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:00.624563+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11722752,"num_objects":2862,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14274,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17828","reported_seq":18658,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.766631+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:00.766631+0000","last_peered":"2024-09-10T16:01:00.766631+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:00.766631+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12034048,"num_objects":2938,"num_object_clones":0,"num_object_copies":5876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14434,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18185","reported_seq":18952,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.802552+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:00.802552+0000","last_peered":"2024-09-10T16:01:00.802552+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:00.802552+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11932131,"num_objects":2914,"num_object_clones":0,"num_object_copies":5828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15004,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-780}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":189231611,"num_objects":46201,"num_object_clones":0,"num_object_copies":92402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246849,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14750,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":568823808,"data_stored":568747017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626529,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567200,"kb_used_data":138508,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93804640,"statfs":{"total":96636764160,"available":96055951360,"internally_reserved":0,"allocated":141832192,"data_stored":141483319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":438955694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149811,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656484,"kb_used_data":201428,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93715356,"statfs":{"total":96636764160,"available":95964524544,"internally_reserved":0,"allocated":206262272,"data_stored":205975279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":465956525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556568,"kb_used_data":218120,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93815272,"statfs":{"total":96636764160,"available":96066838528,"internally_reserved":0,"allocated":223354880,"data_stored":223063815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":205103104,"data_stored":205099491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":141094912,"data_stored":141029419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222625792,"data_stored":222618107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:05.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:05.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'17906", 'reported_seq': 19559, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:00.837033+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:01:00.837033+0000', 'last_peered': '2024-09-10T16:01:00.837033+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:01:00.837033+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'7900", 'ondisk_log_start': "44'7900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11730944, 'num_objects': 2864, 'num_object_clones': 0, 'num_object_copies': 5728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3190, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2864, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17906, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 783, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -326}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:05.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'17749", 'reported_seq': 18229, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:00.744162+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:01:00.744162+0000', 'last_peered': '2024-09-10T16:01:00.744162+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:01:00.744162+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'7700", 'ondisk_log_start': "44'7700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11796504, 'num_objects': 2881, 'num_object_clones': 0, 'num_object_copies': 5762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2881, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16017, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -748}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'17472", 'reported_seq': 18242, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:00.867841+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:01:00.867841+0000', 'last_peered': '2024-09-10T16:01:00.867841+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:01:00.867841+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'7400", 'ondisk_log_start': "44'7400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11526144, 'num_objects': 2814, 'num_object_clones': 0, 'num_object_copies': 5628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2814, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14122, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -710}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'17700", 'reported_seq': 18525, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:00.844079+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:01:00.844079+0000', 'last_peered': '2024-09-10T16:01:00.844079+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:01:00.844079+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'7600", 'ondisk_log_start': "44'7600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11788288, 'num_objects': 2878, 'num_object_clones': 0, 'num_object_copies': 5756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2878, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14564, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -641}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'17550", 'reported_seq': 17973, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:00.817921+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:01:00.817921+0000', 'last_peered': '2024-09-10T16:01:00.817921+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:01:00.817921+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'7500", 'ondisk_log_start': "44'7500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11902976, 'num_objects': 2906, 'num_object_clones': 0, 'num_object_copies': 5812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2906, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15419, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -780}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'17930", 'reported_seq': 18777, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:00.879367+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:01:00.879367+0000', 'last_peered': '2024-09-10T16:01:00.879367+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:01:00.879367+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'7900", 'ondisk_log_start': "44'7900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11976704, 'num_objects': 2924, 'num_object_clones': 0, 'num_object_copies': 5848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15122, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -554}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:05.015 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'17856", 'reported_seq': 18612, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:00.624563+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:01:00.624563+0000', 'last_peered': '2024-09-10T16:01:00.624563+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:01:00.624563+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'7800", 'ondisk_log_start': "44'7800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11722752, 'num_objects': 2862, 'num_object_clones': 0, 'num_object_copies': 5724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14274, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -764}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'17828", 'reported_seq': 18658, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:00.766631+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:01:00.766631+0000', 'last_peered': '2024-09-10T16:01:00.766631+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:01:00.766631+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'7800", 'ondisk_log_start': "44'7800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12034048, 'num_objects': 2938, 'num_object_clones': 0, 'num_object_copies': 5876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2938, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14434, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:05.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'18185", 'reported_seq': 18952, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:00.802552+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:01:00.802552+0000', 'last_peered': '2024-09-10T16:01:00.802552+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:01:00.802552+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'8100", 'ondisk_log_start': "44'8100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11932131, 'num_objects': 2914, 'num_object_clones': 0, 'num_object_copies': 5828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2914, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15004, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -780}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:05.016 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:05.695 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:05.695 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31898/92406 objects degraded (34.519%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":591,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":46203,"data_bytes":189231630,"bytes_used":1822978048,"bytes_avail":288087314432,"bytes_total":289910292480,"degraded_objects":31898,"degraded_total":92406,"degraded_ratio":0.34519403501937096,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":156,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":385},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-10T16:00:44.600014+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:01:05.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:05.717 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:06.263 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:06.264 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:06.292 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-10T16:01:04.607405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":189231630,"num_objects":46203,"num_object_clones":0,"num_object_copies":92406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46203,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246851,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14752,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1780252,"kb_used_data":558056,"kb_used_omap":12,"kb_used_meta":1222131,"kb_avail":281335268,"statfs":{"total":289910292480,"available":288087314432,"internally_reserved":0,"allocated":571449344,"data_stored":570522413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1251462149},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,3,8,13,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18944000,"num_objects":-4625,"num_object_clones":0,"num_object_copies":-9250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4625,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004365"},"pg_stats":[{"pgid":"2.f","version":"50'17906","reported_seq":19559,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.837033+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:00.837033+0000","last_peered":"2024-09-10T16:01:00.837033+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:00.837033+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11730944,"num_objects":2864,"num_object_clones":0,"num_object_copies":5728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17906,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":783,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-326}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18205","reported_seq":18322,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.828410+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:00.828410+0000","last_peered":"2024-09-10T16:01:00.828410+0000","last_clean":"2024-09-10T16:01:00.828410+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:00.828410+0000","last_undegraded":"2024-09-10T16:01:00.828410+0000","last_fullsized":"2024-09-10T16:01:00.828410+0000","mapping_epoch":36,"log_start":"44'8200","ondisk_log_start":"44'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11751424,"num_objects":2869,"num_object_clones":0,"num_object_copies":5738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16541,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17749","reported_seq":18229,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.744162+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:00.744162+0000","last_peered":"2024-09-10T16:01:00.744162+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:00.744162+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11796504,"num_objects":2881,"num_object_clones":0,"num_object_copies":5762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16017,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-748}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17763","reported_seq":17852,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.806592+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:00.806592+0000","last_peered":"2024-09-10T16:01:00.806592+0000","last_clean":"2024-09-10T16:01:00.806592+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:00.806592+0000","last_undegraded":"2024-09-10T16:01:00.806592+0000","last_fullsized":"2024-09-10T16:01:00.806592+0000","mapping_epoch":36,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11956224,"num_objects":2919,"num_object_clones":0,"num_object_copies":5838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17763,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17485","reported_seq":17619,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.760340+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:00.760340+0000","last_peered":"2024-09-10T16:01:00.760340+0000","last_clean":"2024-09-10T16:01:00.760340+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:00.760340+0000","last_undegraded":"2024-09-10T16:01:00.760340+0000","last_fullsized":"2024-09-10T16:01:00.760340+0000","mapping_epoch":35,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11657216,"num_objects":2846,"num_object_clones":0,"num_object_copies":5692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14556,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18030","reported_seq":25928,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:01.429888+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:01.429888+0000","last_peered":"2024-09-10T16:01:01.429888+0000","last_clean":"2024-09-10T16:01:01.429888+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:01.429888+0000","last_undegraded":"2024-09-10T16:01:01.429888+0000","last_fullsized":"2024-09-10T16:01:01.429888+0000","mapping_epoch":47,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11620352,"num_objects":2837,"num_object_clones":0,"num_object_copies":5674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14748,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17472","reported_seq":18242,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.867841+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:00.867841+0000","last_peered":"2024-09-10T16:01:00.867841+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:00.867841+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7400","ondisk_log_start":"44'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11526144,"num_objects":2814,"num_object_clones":0,"num_object_copies":5628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14122,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'17968","reported_seq":18109,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.821391+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:00.821391+0000","last_peered":"2024-09-10T16:01:00.821391+0000","last_clean":"2024-09-10T16:01:00.821391+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:00.821391+0000","last_undegraded":"2024-09-10T16:01:00.821391+0000","last_fullsized":"2024-09-10T16:01:00.821391+0000","mapping_epoch":35,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12066816,"num_objects":2946,"num_object_clones":0,"num_object_copies":5892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14727,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'17858","reported_seq":25995,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:01.378023+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:01.378023+0000","last_peered":"2024-09-10T16:01:01.378023+0000","last_clean":"2024-09-10T16:01:01.378023+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:01.378023+0000","last_undegraded":"2024-09-10T16:01:01.378023+0000","last_fullsized":"2024-09-10T16:01:01.378023+0000","mapping_epoch":45,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11718656,"num_objects":2861,"num_object_clones":0,"num_object_copies":5722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15835,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17700","reported_seq":18525,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.844079+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:00.844079+0000","last_peered":"2024-09-10T16:01:00.844079+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:00.844079+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11788288,"num_objects":2878,"num_object_clones":0,"num_object_copies":5756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14564,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-641}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'17843","reported_seq":25653,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:00.754197+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:00.754197+0000","last_peered":"2024-09-10T16:01:00.754197+0000","last_clean":"2024-09-10T16:01:00.754197+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:00.754197+0000","last_undegraded":"2024-09-10T16:01:00.754197+0000","last_fullsized":"2024-09-10T16:01:00.754197+0000","mapping_epoch":49,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12050432,"num_objects":2942,"num_object_clones":0,"num_object_copies":5884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15817,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17550","reported_seq":17973,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.817921+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:00.817921+0000","last_peered":"2024-09-10T16:01:00.817921+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:00.817921+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7500","ondisk_log_start":"44'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11902976,"num_objects":2906,"num_object_clones":0,"num_object_copies":5812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15419,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-780}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'17930","reported_seq":18777,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.879367+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:00.879367+0000","last_peered":"2024-09-10T16:01:00.879367+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:00.879367+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11976704,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15122,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'17856","reported_seq":18612,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.624563+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:00.624563+0000","last_peered":"2024-09-10T16:01:00.624563+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:00.624563+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11722752,"num_objects":2862,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14274,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17828","reported_seq":18658,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.766631+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:00.766631+0000","last_peered":"2024-09-10T16:01:00.766631+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:00.766631+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12034048,"num_objects":2938,"num_object_clones":0,"num_object_copies":5876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14434,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18185","reported_seq":18952,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:00.802552+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:00.802552+0000","last_peered":"2024-09-10T16:01:00.802552+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:00.802552+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11932131,"num_objects":2914,"num_object_clones":0,"num_object_copies":5828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15004,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-780}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":189231611,"num_objects":46201,"num_object_clones":0,"num_object_copies":92402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246849,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14750,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":568823808,"data_stored":568747017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626529,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567200,"kb_used_data":138508,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93804640,"statfs":{"total":96636764160,"available":96055951360,"internally_reserved":0,"allocated":141832192,"data_stored":141483319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":438955694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149811,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656484,"kb_used_data":201428,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93715356,"statfs":{"total":96636764160,"available":95964524544,"internally_reserved":0,"allocated":206262272,"data_stored":205975279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":465956525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556568,"kb_used_data":218120,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93815272,"statfs":{"total":96636764160,"available":96066838528,"internally_reserved":0,"allocated":223354880,"data_stored":223063815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":205103104,"data_stored":205099491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":141094912,"data_stored":141029419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222625792,"data_stored":222618107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:09.294 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:09.850 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:09.851 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:09.879 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-10T16:01:08.608626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":179061262,"num_objects":43720,"num_object_clones":0,"num_object_copies":87440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249334,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14835,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1760488,"kb_used_data":538276,"kb_used_omap":12,"kb_used_meta":1222131,"kb_avail":281355032,"statfs":{"total":289910292480,"available":288107552768,"internally_reserved":0,"allocated":551194624,"data_stored":550267693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1251462149},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,8,8,15,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19369984,"num_objects":-4729,"num_object_clones":0,"num_object_copies":-9458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4729,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004000"},"pg_stats":[{"pgid":"2.f","version":"50'18082","reported_seq":19901,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.905954+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:05.905954+0000","last_peered":"2024-09-10T16:01:05.905954+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:05.905954+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11010048,"num_objects":2688,"num_object_clones":0,"num_object_copies":5376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18082,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":866,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-419}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18354","reported_seq":18471,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.795592+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:05.795592+0000","last_peered":"2024-09-10T16:01:05.795592+0000","last_clean":"2024-09-10T16:01:05.795592+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:05.795592+0000","last_undegraded":"2024-09-10T16:01:05.795592+0000","last_fullsized":"2024-09-10T16:01:05.795592+0000","mapping_epoch":36,"log_start":"45'8300","ondisk_log_start":"45'8300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11141120,"num_objects":2720,"num_object_clones":0,"num_object_copies":5440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16690,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17893","reported_seq":18373,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.745577+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:05.745577+0000","last_peered":"2024-09-10T16:01:05.745577+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:05.745577+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11206680,"num_objects":2737,"num_object_clones":0,"num_object_copies":5474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16161,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-892}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17931","reported_seq":18020,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.677512+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:05.677512+0000","last_peered":"2024-09-10T16:01:05.677512+0000","last_clean":"2024-09-10T16:01:05.677512+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:05.677512+0000","last_undegraded":"2024-09-10T16:01:05.677512+0000","last_fullsized":"2024-09-10T16:01:05.677512+0000","mapping_epoch":36,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268096,"num_objects":2751,"num_object_clones":0,"num_object_copies":5502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17931,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17630","reported_seq":17767,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.849918+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:05.849918+0000","last_peered":"2024-09-10T16:01:05.849918+0000","last_clean":"2024-09-10T16:01:05.849918+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:05.849918+0000","last_undegraded":"2024-09-10T16:01:05.849918+0000","last_fullsized":"2024-09-10T16:01:05.849918+0000","mapping_epoch":35,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11063296,"num_objects":2701,"num_object_clones":0,"num_object_copies":5402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14701,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18155","reported_seq":26053,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:06.485698+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:06.485698+0000","last_peered":"2024-09-10T16:01:06.485698+0000","last_clean":"2024-09-10T16:01:06.485698+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:06.485698+0000","last_undegraded":"2024-09-10T16:01:06.485698+0000","last_fullsized":"2024-09-10T16:01:06.485698+0000","mapping_epoch":47,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11108352,"num_objects":2712,"num_object_clones":0,"num_object_copies":5424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14873,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17622","reported_seq":18391,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.854150+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:05.854150+0000","last_peered":"2024-09-10T16:01:05.854150+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:05.854150+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10911744,"num_objects":2664,"num_object_clones":0,"num_object_copies":5328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14272,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18140","reported_seq":18281,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.778102+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:05.778102+0000","last_peered":"2024-09-10T16:01:05.778102+0000","last_clean":"2024-09-10T16:01:05.778102+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:05.778102+0000","last_undegraded":"2024-09-10T16:01:05.778102+0000","last_fullsized":"2024-09-10T16:01:05.778102+0000","mapping_epoch":35,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11362304,"num_objects":2774,"num_object_clones":0,"num_object_copies":5548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14899,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18006","reported_seq":26142,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:06.506486+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:06.506486+0000","last_peered":"2024-09-10T16:01:06.506486+0000","last_clean":"2024-09-10T16:01:06.506486+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:06.506486+0000","last_undegraded":"2024-09-10T16:01:06.506486+0000","last_fullsized":"2024-09-10T16:01:06.506486+0000","mapping_epoch":45,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15983,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17857","reported_seq":18682,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.709076+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:05.709076+0000","last_peered":"2024-09-10T16:01:05.709076+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:05.709076+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11145216,"num_objects":2721,"num_object_clones":0,"num_object_copies":5442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14721,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18016","reported_seq":25826,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.723828+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:05.723828+0000","last_peered":"2024-09-10T16:01:05.723828+0000","last_clean":"2024-09-10T16:01:05.723828+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:05.723828+0000","last_undegraded":"2024-09-10T16:01:05.723828+0000","last_fullsized":"2024-09-10T16:01:05.723828+0000","mapping_epoch":49,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11341824,"num_objects":2769,"num_object_clones":0,"num_object_copies":5538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15990,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17702","reported_seq":18125,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.810083+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:05.810083+0000","last_peered":"2024-09-10T16:01:05.810083+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:05.810083+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11280384,"num_objects":2754,"num_object_clones":0,"num_object_copies":5508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15571,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-932}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18084","reported_seq":18932,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.906762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:05.906762+0000","last_peered":"2024-09-10T16:01:05.906762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:05.906762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11345920,"num_objects":2770,"num_object_clones":0,"num_object_copies":5540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15276,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18015","reported_seq":18771,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.859793+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:05.859793+0000","last_peered":"2024-09-10T16:01:05.859793+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:05.859793+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11071488,"num_objects":2703,"num_object_clones":0,"num_object_copies":5406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14433,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17975","reported_seq":18807,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.883322+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:05.883322+0000","last_peered":"2024-09-10T16:01:05.883322+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:05.883322+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11431936,"num_objects":2791,"num_object_clones":0,"num_object_copies":5582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14581,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-745}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18349","reported_seq":19116,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.774532+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:05.774532+0000","last_peered":"2024-09-10T16:01:05.774532+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:05.774532+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'8300","ondisk_log_start":"44'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260387,"num_objects":2750,"num_object_clones":0,"num_object_copies":5500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15168,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":179061243,"num_objects":43718,"num_object_clones":0,"num_object_copies":87436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43718,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249332,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14833,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548433920,"data_stored":548357129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626530,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564088,"kb_used_data":131240,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93807752,"statfs":{"total":96636764160,"available":96059138048,"internally_reserved":0,"allocated":134389760,"data_stored":134040887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443215534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149812,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640780,"kb_used_data":194024,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93731060,"statfs":{"total":96636764160,"available":95980605440,"internally_reserved":0,"allocated":198680576,"data_stored":198393583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":457436845},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555620,"kb_used_data":213012,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93816220,"statfs":{"total":96636764160,"available":96067809280,"internally_reserved":0,"allocated":218124288,"data_stored":217833223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":197427200,"data_stored":197423587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":133500928,"data_stored":133435435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217505792,"data_stored":217498107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:09.880 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:10.434 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:10.437 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:10.456 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-10T16:01:08.608626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":179061262,"num_objects":43720,"num_object_clones":0,"num_object_copies":87440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249334,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14835,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1760488,"kb_used_data":538276,"kb_used_omap":12,"kb_used_meta":1222131,"kb_avail":281355032,"statfs":{"total":289910292480,"available":288107552768,"internally_reserved":0,"allocated":551194624,"data_stored":550267693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1251462149},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,8,8,15,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19369984,"num_objects":-4729,"num_object_clones":0,"num_object_copies":-9458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4729,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004000"},"pg_stats":[{"pgid":"2.f","version":"50'18082","reported_seq":19901,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.905954+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:05.905954+0000","last_peered":"2024-09-10T16:01:05.905954+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:05.905954+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11010048,"num_objects":2688,"num_object_clones":0,"num_object_copies":5376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18082,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":866,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-419}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18354","reported_seq":18471,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.795592+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:05.795592+0000","last_peered":"2024-09-10T16:01:05.795592+0000","last_clean":"2024-09-10T16:01:05.795592+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:05.795592+0000","last_undegraded":"2024-09-10T16:01:05.795592+0000","last_fullsized":"2024-09-10T16:01:05.795592+0000","mapping_epoch":36,"log_start":"45'8300","ondisk_log_start":"45'8300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11141120,"num_objects":2720,"num_object_clones":0,"num_object_copies":5440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16690,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17893","reported_seq":18373,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.745577+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:05.745577+0000","last_peered":"2024-09-10T16:01:05.745577+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:05.745577+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11206680,"num_objects":2737,"num_object_clones":0,"num_object_copies":5474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16161,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-892}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17931","reported_seq":18020,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.677512+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:05.677512+0000","last_peered":"2024-09-10T16:01:05.677512+0000","last_clean":"2024-09-10T16:01:05.677512+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:05.677512+0000","last_undegraded":"2024-09-10T16:01:05.677512+0000","last_fullsized":"2024-09-10T16:01:05.677512+0000","mapping_epoch":36,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268096,"num_objects":2751,"num_object_clones":0,"num_object_copies":5502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17931,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17630","reported_seq":17767,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.849918+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:05.849918+0000","last_peered":"2024-09-10T16:01:05.849918+0000","last_clean":"2024-09-10T16:01:05.849918+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:05.849918+0000","last_undegraded":"2024-09-10T16:01:05.849918+0000","last_fullsized":"2024-09-10T16:01:05.849918+0000","mapping_epoch":35,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11063296,"num_objects":2701,"num_object_clones":0,"num_object_copies":5402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14701,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18155","reported_seq":26053,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:06.485698+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:06.485698+0000","last_peered":"2024-09-10T16:01:06.485698+0000","last_clean":"2024-09-10T16:01:06.485698+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:06.485698+0000","last_undegraded":"2024-09-10T16:01:06.485698+0000","last_fullsized":"2024-09-10T16:01:06.485698+0000","mapping_epoch":47,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11108352,"num_objects":2712,"num_object_clones":0,"num_object_copies":5424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14873,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17622","reported_seq":18391,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.854150+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:05.854150+0000","last_peered":"2024-09-10T16:01:05.854150+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:05.854150+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10911744,"num_objects":2664,"num_object_clones":0,"num_object_copies":5328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14272,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18140","reported_seq":18281,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.778102+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:05.778102+0000","last_peered":"2024-09-10T16:01:05.778102+0000","last_clean":"2024-09-10T16:01:05.778102+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:05.778102+0000","last_undegraded":"2024-09-10T16:01:05.778102+0000","last_fullsized":"2024-09-10T16:01:05.778102+0000","mapping_epoch":35,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11362304,"num_objects":2774,"num_object_clones":0,"num_object_copies":5548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14899,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18006","reported_seq":26142,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:06.506486+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:06.506486+0000","last_peered":"2024-09-10T16:01:06.506486+0000","last_clean":"2024-09-10T16:01:06.506486+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:06.506486+0000","last_undegraded":"2024-09-10T16:01:06.506486+0000","last_fullsized":"2024-09-10T16:01:06.506486+0000","mapping_epoch":45,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15983,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17857","reported_seq":18682,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.709076+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:05.709076+0000","last_peered":"2024-09-10T16:01:05.709076+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:05.709076+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11145216,"num_objects":2721,"num_object_clones":0,"num_object_copies":5442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14721,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18016","reported_seq":25826,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.723828+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:05.723828+0000","last_peered":"2024-09-10T16:01:05.723828+0000","last_clean":"2024-09-10T16:01:05.723828+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:05.723828+0000","last_undegraded":"2024-09-10T16:01:05.723828+0000","last_fullsized":"2024-09-10T16:01:05.723828+0000","mapping_epoch":49,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11341824,"num_objects":2769,"num_object_clones":0,"num_object_copies":5538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15990,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17702","reported_seq":18125,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.810083+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:05.810083+0000","last_peered":"2024-09-10T16:01:05.810083+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:05.810083+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11280384,"num_objects":2754,"num_object_clones":0,"num_object_copies":5508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15571,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-932}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18084","reported_seq":18932,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.906762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:05.906762+0000","last_peered":"2024-09-10T16:01:05.906762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:05.906762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11345920,"num_objects":2770,"num_object_clones":0,"num_object_copies":5540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15276,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18015","reported_seq":18771,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.859793+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:05.859793+0000","last_peered":"2024-09-10T16:01:05.859793+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:05.859793+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11071488,"num_objects":2703,"num_object_clones":0,"num_object_copies":5406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14433,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17975","reported_seq":18807,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.883322+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:05.883322+0000","last_peered":"2024-09-10T16:01:05.883322+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:05.883322+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11431936,"num_objects":2791,"num_object_clones":0,"num_object_copies":5582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14581,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-745}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18349","reported_seq":19116,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.774532+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:05.774532+0000","last_peered":"2024-09-10T16:01:05.774532+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:05.774532+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'8300","ondisk_log_start":"44'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260387,"num_objects":2750,"num_object_clones":0,"num_object_copies":5500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15168,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":179061243,"num_objects":43718,"num_object_clones":0,"num_object_copies":87436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43718,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249332,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14833,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548433920,"data_stored":548357129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626530,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564088,"kb_used_data":131240,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93807752,"statfs":{"total":96636764160,"available":96059138048,"internally_reserved":0,"allocated":134389760,"data_stored":134040887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443215534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149812,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640780,"kb_used_data":194024,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93731060,"statfs":{"total":96636764160,"available":95980605440,"internally_reserved":0,"allocated":198680576,"data_stored":198393583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":457436845},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555620,"kb_used_data":213012,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93816220,"statfs":{"total":96636764160,"available":96067809280,"internally_reserved":0,"allocated":218124288,"data_stored":217833223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":197427200,"data_stored":197423587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":133500928,"data_stored":133435435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217505792,"data_stored":217498107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:10.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:10.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'18082", 'reported_seq': 19901, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:05.905954+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:01:05.905954+0000', 'last_peered': '2024-09-10T16:01:05.905954+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:01:05.905954+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "44'8000", 'ondisk_log_start': "44'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11010048, 'num_objects': 2688, 'num_object_clones': 0, 'num_object_copies': 5376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2688, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18082, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 866, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -419}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:10.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'17893", 'reported_seq': 18373, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:05.745577+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:01:05.745577+0000', 'last_peered': '2024-09-10T16:01:05.745577+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:01:05.745577+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'7800", 'ondisk_log_start': "44'7800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11206680, 'num_objects': 2737, 'num_object_clones': 0, 'num_object_copies': 5474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2737, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16161, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -892}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:10.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:10.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'17622", 'reported_seq': 18391, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:05.854150+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:01:05.854150+0000', 'last_peered': '2024-09-10T16:01:05.854150+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:01:05.854150+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'7600", 'ondisk_log_start': "44'7600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10911744, 'num_objects': 2664, 'num_object_clones': 0, 'num_object_copies': 5328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2664, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14272, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -860}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:10.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'17857", 'reported_seq': 18682, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:05.709076+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:01:05.709076+0000', 'last_peered': '2024-09-10T16:01:05.709076+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:01:05.709076+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'7800", 'ondisk_log_start': "44'7800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11145216, 'num_objects': 2721, 'num_object_clones': 0, 'num_object_copies': 5442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14721, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -798}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:10.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'17702", 'reported_seq': 18125, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:05.810083+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:01:05.810083+0000', 'last_peered': '2024-09-10T16:01:05.810083+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:01:05.810083+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'7700", 'ondisk_log_start': "44'7700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11280384, 'num_objects': 2754, 'num_object_clones': 0, 'num_object_copies': 5508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2754, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15571, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -932}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:10.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:10.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'18084", 'reported_seq': 18932, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:05.906762+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:01:05.906762+0000', 'last_peered': '2024-09-10T16:01:05.906762+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:01:05.906762+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "44'8000", 'ondisk_log_start': "44'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11345920, 'num_objects': 2770, 'num_object_clones': 0, 'num_object_copies': 5540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15276, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -708}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:10.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'18015", 'reported_seq': 18771, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:05.859793+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:01:05.859793+0000', 'last_peered': '2024-09-10T16:01:05.859793+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:01:05.859793+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'8000", 'ondisk_log_start': "44'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11071488, 'num_objects': 2703, 'num_object_clones': 0, 'num_object_copies': 5406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2703, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14433, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -923}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:10.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'17975", 'reported_seq': 18807, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:05.883322+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:01:05.883322+0000', 'last_peered': '2024-09-10T16:01:05.883322+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:01:05.883322+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "44'7900", 'ondisk_log_start': "44'7900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11431936, 'num_objects': 2791, 'num_object_clones': 0, 'num_object_copies': 5582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2791, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14581, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -745}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:10.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'18349", 'reported_seq': 19116, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:05.774532+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:01:05.774532+0000', 'last_peered': '2024-09-10T16:01:05.774532+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:01:05.774532+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "44'8300", 'ondisk_log_start': "44'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11260387, 'num_objects': 2750, 'num_object_clones': 0, 'num_object_copies': 5500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2750, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15168, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -944}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:10.459 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:11.136 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:11.136 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31815/87440 objects degraded (36.385%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":597,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":43720,"data_bytes":179061262,"bytes_used":1802739712,"bytes_avail":288107552768,"bytes_total":289910292480,"degraded_objects":31815,"degraded_total":87440,"degraded_ratio":0.36384949679780421,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":393},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-10T16:00:44.600014+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:01:11.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:11.161 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:11.737 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:11.738 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:11.757 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-10T16:01:10.609387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":179061262,"num_objects":43720,"num_object_clones":0,"num_object_copies":87440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249334,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14835,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1760488,"kb_used_data":538276,"kb_used_omap":12,"kb_used_meta":1222131,"kb_avail":281355032,"statfs":{"total":289910292480,"available":288107552768,"internally_reserved":0,"allocated":551194624,"data_stored":550267693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1251462149},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,8,8,15,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19369984,"num_objects":-4729,"num_object_clones":0,"num_object_copies":-9458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4729,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003901"},"pg_stats":[{"pgid":"2.f","version":"50'18082","reported_seq":19901,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.905954+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:05.905954+0000","last_peered":"2024-09-10T16:01:05.905954+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:05.905954+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11010048,"num_objects":2688,"num_object_clones":0,"num_object_copies":5376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18082,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":866,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-419}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18354","reported_seq":18471,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.795592+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:05.795592+0000","last_peered":"2024-09-10T16:01:05.795592+0000","last_clean":"2024-09-10T16:01:05.795592+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:05.795592+0000","last_undegraded":"2024-09-10T16:01:05.795592+0000","last_fullsized":"2024-09-10T16:01:05.795592+0000","mapping_epoch":36,"log_start":"45'8300","ondisk_log_start":"45'8300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11141120,"num_objects":2720,"num_object_clones":0,"num_object_copies":5440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16690,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'17893","reported_seq":18373,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.745577+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:05.745577+0000","last_peered":"2024-09-10T16:01:05.745577+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:05.745577+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11206680,"num_objects":2737,"num_object_clones":0,"num_object_copies":5474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16161,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-892}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'17931","reported_seq":18020,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.677512+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:05.677512+0000","last_peered":"2024-09-10T16:01:05.677512+0000","last_clean":"2024-09-10T16:01:05.677512+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:05.677512+0000","last_undegraded":"2024-09-10T16:01:05.677512+0000","last_fullsized":"2024-09-10T16:01:05.677512+0000","mapping_epoch":36,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268096,"num_objects":2751,"num_object_clones":0,"num_object_copies":5502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17931,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17630","reported_seq":17767,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.849918+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:05.849918+0000","last_peered":"2024-09-10T16:01:05.849918+0000","last_clean":"2024-09-10T16:01:05.849918+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:05.849918+0000","last_undegraded":"2024-09-10T16:01:05.849918+0000","last_fullsized":"2024-09-10T16:01:05.849918+0000","mapping_epoch":35,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11063296,"num_objects":2701,"num_object_clones":0,"num_object_copies":5402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14701,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18155","reported_seq":26053,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:06.485698+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:06.485698+0000","last_peered":"2024-09-10T16:01:06.485698+0000","last_clean":"2024-09-10T16:01:06.485698+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:06.485698+0000","last_undegraded":"2024-09-10T16:01:06.485698+0000","last_fullsized":"2024-09-10T16:01:06.485698+0000","mapping_epoch":47,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11108352,"num_objects":2712,"num_object_clones":0,"num_object_copies":5424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14873,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17622","reported_seq":18391,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.854150+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:05.854150+0000","last_peered":"2024-09-10T16:01:05.854150+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:05.854150+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7600","ondisk_log_start":"44'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10911744,"num_objects":2664,"num_object_clones":0,"num_object_copies":5328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14272,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18140","reported_seq":18281,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.778102+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:05.778102+0000","last_peered":"2024-09-10T16:01:05.778102+0000","last_clean":"2024-09-10T16:01:05.778102+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:05.778102+0000","last_undegraded":"2024-09-10T16:01:05.778102+0000","last_fullsized":"2024-09-10T16:01:05.778102+0000","mapping_epoch":35,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11362304,"num_objects":2774,"num_object_clones":0,"num_object_copies":5548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14899,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18006","reported_seq":26142,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:06.506486+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:06.506486+0000","last_peered":"2024-09-10T16:01:06.506486+0000","last_clean":"2024-09-10T16:01:06.506486+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:06.506486+0000","last_undegraded":"2024-09-10T16:01:06.506486+0000","last_fullsized":"2024-09-10T16:01:06.506486+0000","mapping_epoch":45,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15983,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'17857","reported_seq":18682,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.709076+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:05.709076+0000","last_peered":"2024-09-10T16:01:05.709076+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:05.709076+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11145216,"num_objects":2721,"num_object_clones":0,"num_object_copies":5442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14721,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18016","reported_seq":25826,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:05.723828+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:05.723828+0000","last_peered":"2024-09-10T16:01:05.723828+0000","last_clean":"2024-09-10T16:01:05.723828+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:05.723828+0000","last_undegraded":"2024-09-10T16:01:05.723828+0000","last_fullsized":"2024-09-10T16:01:05.723828+0000","mapping_epoch":49,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11341824,"num_objects":2769,"num_object_clones":0,"num_object_copies":5538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15990,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17702","reported_seq":18125,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.810083+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:05.810083+0000","last_peered":"2024-09-10T16:01:05.810083+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:05.810083+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11280384,"num_objects":2754,"num_object_clones":0,"num_object_copies":5508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15571,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-932}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18084","reported_seq":18932,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.906762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:05.906762+0000","last_peered":"2024-09-10T16:01:05.906762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:05.906762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11345920,"num_objects":2770,"num_object_clones":0,"num_object_copies":5540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15276,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18015","reported_seq":18771,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.859793+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:05.859793+0000","last_peered":"2024-09-10T16:01:05.859793+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:05.859793+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11071488,"num_objects":2703,"num_object_clones":0,"num_object_copies":5406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14433,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'17975","reported_seq":18807,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.883322+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:05.883322+0000","last_peered":"2024-09-10T16:01:05.883322+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:05.883322+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11431936,"num_objects":2791,"num_object_clones":0,"num_object_copies":5582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14581,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-745}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18349","reported_seq":19116,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:05.774532+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:05.774532+0000","last_peered":"2024-09-10T16:01:05.774532+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:05.774532+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"44'8300","ondisk_log_start":"44'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260387,"num_objects":2750,"num_object_clones":0,"num_object_copies":5500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15168,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":179061243,"num_objects":43718,"num_object_clones":0,"num_object_copies":87436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43718,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249332,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14833,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548433920,"data_stored":548357129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626530,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564088,"kb_used_data":131240,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93807752,"statfs":{"total":96636764160,"available":96059138048,"internally_reserved":0,"allocated":134389760,"data_stored":134040887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443215534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149812,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640780,"kb_used_data":194024,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93731060,"statfs":{"total":96636764160,"available":95980605440,"internally_reserved":0,"allocated":198680576,"data_stored":198393583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":457436845},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555620,"kb_used_data":213012,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93816220,"statfs":{"total":96636764160,"available":96067809280,"internally_reserved":0,"allocated":218124288,"data_stored":217833223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":197427200,"data_stored":197423587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":133500928,"data_stored":133435435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217505792,"data_stored":217498107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:14.759 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:15.286 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:15.289 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:15.310 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-10T16:01:14.611008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":168768014,"num_objects":41207,"num_object_clones":0,"num_object_copies":82414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31738,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251847,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14912,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1744848,"kb_used_data":518476,"kb_used_omap":12,"kb_used_meta":1226291,"kb_avail":281370672,"statfs":{"total":289910292480,"available":288123568128,"internally_reserved":0,"allocated":530919424,"data_stored":529992493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1255721989},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3,6,12,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20463616,"num_objects":-4996,"num_object_clones":0,"num_object_copies":-9992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4996,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004239"},"pg_stats":[{"pgid":"2.f","version":"50'18250","reported_seq":20220,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.916586+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:10.916586+0000","last_peered":"2024-09-10T16:01:10.916586+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:10.916586+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10321920,"num_objects":2520,"num_object_clones":0,"num_object_copies":5040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18250,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-510}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18515","reported_seq":18632,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:10.811610+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:10.811610+0000","last_peered":"2024-09-10T16:01:10.811610+0000","last_clean":"2024-09-10T16:01:10.811610+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:10.811610+0000","last_undegraded":"2024-09-10T16:01:10.811610+0000","last_fullsized":"2024-09-10T16:01:10.811610+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10481664,"num_objects":2559,"num_object_clones":0,"num_object_copies":5118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16851,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18044","reported_seq":18524,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.742276+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:10.742276+0000","last_peered":"2024-09-10T16:01:10.742276+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:10.742276+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10588184,"num_objects":2586,"num_object_clones":0,"num_object_copies":5172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2586,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16312,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1043}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18094","reported_seq":18183,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:10.836536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:10.836536+0000","last_peered":"2024-09-10T16:01:10.836536+0000","last_clean":"2024-09-10T16:01:10.836536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:10.836536+0000","last_undegraded":"2024-09-10T16:01:10.836536+0000","last_fullsized":"2024-09-10T16:01:10.836536+0000","mapping_epoch":36,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10600448,"num_objects":2588,"num_object_clones":0,"num_object_copies":5176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18094,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17769","reported_seq":17905,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:10.920608+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:10.920608+0000","last_peered":"2024-09-10T16:01:10.920608+0000","last_clean":"2024-09-10T16:01:10.920608+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:10.920608+0000","last_undegraded":"2024-09-10T16:01:10.920608+0000","last_fullsized":"2024-09-10T16:01:10.920608+0000","mapping_epoch":35,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10493952,"num_objects":2562,"num_object_clones":0,"num_object_copies":5124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14840,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18315","reported_seq":26212,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:11.492477+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:11.492477+0000","last_peered":"2024-09-10T16:01:11.492477+0000","last_clean":"2024-09-10T16:01:11.492477+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:11.492477+0000","last_undegraded":"2024-09-10T16:01:11.492477+0000","last_fullsized":"2024-09-10T16:01:11.492477+0000","mapping_epoch":47,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10452992,"num_objects":2552,"num_object_clones":0,"num_object_copies":5104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15033,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17770","reported_seq":18541,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.878251+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:10.878251+0000","last_peered":"2024-09-10T16:01:10.878251+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:10.878251+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10305536,"num_objects":2516,"num_object_clones":0,"num_object_copies":5032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14420,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18296","reported_seq":18437,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:10.866155+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:10.866155+0000","last_peered":"2024-09-10T16:01:10.866155+0000","last_clean":"2024-09-10T16:01:10.866155+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:10.866155+0000","last_undegraded":"2024-09-10T16:01:10.866155+0000","last_fullsized":"2024-09-10T16:01:10.866155+0000","mapping_epoch":35,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10723328,"num_objects":2618,"num_object_clones":0,"num_object_copies":5236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15055,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18193","reported_seq":26330,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:11.558103+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:11.558103+0000","last_peered":"2024-09-10T16:01:11.558103+0000","last_clean":"2024-09-10T16:01:11.558103+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:11.558103+0000","last_undegraded":"2024-09-10T16:01:11.558103+0000","last_fullsized":"2024-09-10T16:01:11.558103+0000","mapping_epoch":45,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10346496,"num_objects":2526,"num_object_clones":0,"num_object_copies":5052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16170,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18010","reported_seq":18835,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.918308+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:10.918308+0000","last_peered":"2024-09-10T16:01:10.918308+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:10.918308+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10518528,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14874,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-951}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18164","reported_seq":25974,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:10.812282+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:10.812282+0000","last_peered":"2024-09-10T16:01:10.812282+0000","last_clean":"2024-09-10T16:01:10.812282+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:10.812282+0000","last_undegraded":"2024-09-10T16:01:10.812282+0000","last_fullsized":"2024-09-10T16:01:10.812282+0000","mapping_epoch":49,"log_start":"45'8100","ondisk_log_start":"45'8100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10735616,"num_objects":2621,"num_object_clones":0,"num_object_copies":5242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16138,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17867","reported_seq":18290,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.786131+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:10.786131+0000","last_peered":"2024-09-10T16:01:10.786131+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:10.786131+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10604544,"num_objects":2589,"num_object_clones":0,"num_object_copies":5178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15736,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1097}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18242","reported_seq":19090,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.726686+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:10.726686+0000","last_peered":"2024-09-10T16:01:10.726686+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:10.726686+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10698752,"num_objects":2612,"num_object_clones":0,"num_object_copies":5224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15434,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-866}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18164","reported_seq":18920,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.841885+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:10.841885+0000","last_peered":"2024-09-10T16:01:10.841885+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:10.841885+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10461184,"num_objects":2554,"num_object_clones":0,"num_object_copies":5108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14582,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1072}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18127","reported_seq":18958,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.925777+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:10.925777+0000","last_peered":"2024-09-10T16:01:10.925777+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:10.925777+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10809344,"num_objects":2639,"num_object_clones":0,"num_object_copies":5278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14733,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18504","reported_seq":19270,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.932693+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:10.932693+0000","last_peered":"2024-09-10T16:01:10.932693+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:10.932693+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10625507,"num_objects":2595,"num_object_clones":0,"num_object_copies":5190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15323,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":168767995,"num_objects":41205,"num_object_clones":0,"num_object_copies":82410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31738,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41205,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251845,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14910,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":527867904,"data_stored":527791113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626531,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548396,"kb_used_data":123848,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93823444,"statfs":{"total":96636764160,"available":96075206656,"internally_reserved":0,"allocated":126820352,"data_stored":126471479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":434695854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149813,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637604,"kb_used_data":186708,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93734236,"statfs":{"total":96636764160,"available":95983857664,"internally_reserved":0,"allocated":191188992,"data_stored":190901999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":461696685},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953584,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558848,"kb_used_data":207920,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93812992,"statfs":{"total":96636764160,"available":96064503808,"internally_reserved":0,"allocated":212910080,"data_stored":212619015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":189792256,"data_stored":189788643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":125747200,"data_stored":125681707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":212328448,"data_stored":212320763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:15.312 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:15.851 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:15.854 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:15.872 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-10T16:01:14.611008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":168768014,"num_objects":41207,"num_object_clones":0,"num_object_copies":82414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31738,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251847,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14912,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1744848,"kb_used_data":518476,"kb_used_omap":12,"kb_used_meta":1226291,"kb_avail":281370672,"statfs":{"total":289910292480,"available":288123568128,"internally_reserved":0,"allocated":530919424,"data_stored":529992493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1255721989},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3,6,12,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20463616,"num_objects":-4996,"num_object_clones":0,"num_object_copies":-9992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4996,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004239"},"pg_stats":[{"pgid":"2.f","version":"50'18250","reported_seq":20220,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.916586+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:10.916586+0000","last_peered":"2024-09-10T16:01:10.916586+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:10.916586+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10321920,"num_objects":2520,"num_object_clones":0,"num_object_copies":5040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18250,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-510}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18515","reported_seq":18632,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:10.811610+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:10.811610+0000","last_peered":"2024-09-10T16:01:10.811610+0000","last_clean":"2024-09-10T16:01:10.811610+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:10.811610+0000","last_undegraded":"2024-09-10T16:01:10.811610+0000","last_fullsized":"2024-09-10T16:01:10.811610+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10481664,"num_objects":2559,"num_object_clones":0,"num_object_copies":5118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16851,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18044","reported_seq":18524,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.742276+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:10.742276+0000","last_peered":"2024-09-10T16:01:10.742276+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:10.742276+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10588184,"num_objects":2586,"num_object_clones":0,"num_object_copies":5172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2586,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16312,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1043}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18094","reported_seq":18183,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:10.836536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:10.836536+0000","last_peered":"2024-09-10T16:01:10.836536+0000","last_clean":"2024-09-10T16:01:10.836536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:10.836536+0000","last_undegraded":"2024-09-10T16:01:10.836536+0000","last_fullsized":"2024-09-10T16:01:10.836536+0000","mapping_epoch":36,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10600448,"num_objects":2588,"num_object_clones":0,"num_object_copies":5176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18094,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17769","reported_seq":17905,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:10.920608+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:10.920608+0000","last_peered":"2024-09-10T16:01:10.920608+0000","last_clean":"2024-09-10T16:01:10.920608+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:10.920608+0000","last_undegraded":"2024-09-10T16:01:10.920608+0000","last_fullsized":"2024-09-10T16:01:10.920608+0000","mapping_epoch":35,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10493952,"num_objects":2562,"num_object_clones":0,"num_object_copies":5124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14840,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18315","reported_seq":26212,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:11.492477+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:11.492477+0000","last_peered":"2024-09-10T16:01:11.492477+0000","last_clean":"2024-09-10T16:01:11.492477+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:11.492477+0000","last_undegraded":"2024-09-10T16:01:11.492477+0000","last_fullsized":"2024-09-10T16:01:11.492477+0000","mapping_epoch":47,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10452992,"num_objects":2552,"num_object_clones":0,"num_object_copies":5104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15033,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17770","reported_seq":18541,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.878251+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:10.878251+0000","last_peered":"2024-09-10T16:01:10.878251+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:10.878251+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7700","ondisk_log_start":"44'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10305536,"num_objects":2516,"num_object_clones":0,"num_object_copies":5032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14420,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18296","reported_seq":18437,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:10.866155+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:10.866155+0000","last_peered":"2024-09-10T16:01:10.866155+0000","last_clean":"2024-09-10T16:01:10.866155+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:10.866155+0000","last_undegraded":"2024-09-10T16:01:10.866155+0000","last_fullsized":"2024-09-10T16:01:10.866155+0000","mapping_epoch":35,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10723328,"num_objects":2618,"num_object_clones":0,"num_object_copies":5236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15055,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18193","reported_seq":26330,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:11.558103+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:11.558103+0000","last_peered":"2024-09-10T16:01:11.558103+0000","last_clean":"2024-09-10T16:01:11.558103+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:11.558103+0000","last_undegraded":"2024-09-10T16:01:11.558103+0000","last_fullsized":"2024-09-10T16:01:11.558103+0000","mapping_epoch":45,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10346496,"num_objects":2526,"num_object_clones":0,"num_object_copies":5052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16170,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18010","reported_seq":18835,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.918308+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:10.918308+0000","last_peered":"2024-09-10T16:01:10.918308+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:10.918308+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"44'8000","ondisk_log_start":"44'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10518528,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14874,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-951}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18164","reported_seq":25974,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:10.812282+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:10.812282+0000","last_peered":"2024-09-10T16:01:10.812282+0000","last_clean":"2024-09-10T16:01:10.812282+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:10.812282+0000","last_undegraded":"2024-09-10T16:01:10.812282+0000","last_fullsized":"2024-09-10T16:01:10.812282+0000","mapping_epoch":49,"log_start":"45'8100","ondisk_log_start":"45'8100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10735616,"num_objects":2621,"num_object_clones":0,"num_object_copies":5242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16138,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'17867","reported_seq":18290,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.786131+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:10.786131+0000","last_peered":"2024-09-10T16:01:10.786131+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:10.786131+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"44'7800","ondisk_log_start":"44'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10604544,"num_objects":2589,"num_object_clones":0,"num_object_copies":5178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15736,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1097}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18242","reported_seq":19090,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.726686+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:10.726686+0000","last_peered":"2024-09-10T16:01:10.726686+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:10.726686+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10698752,"num_objects":2612,"num_object_clones":0,"num_object_copies":5224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15434,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-866}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18164","reported_seq":18920,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.841885+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:10.841885+0000","last_peered":"2024-09-10T16:01:10.841885+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:10.841885+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10461184,"num_objects":2554,"num_object_clones":0,"num_object_copies":5108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14582,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1072}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18127","reported_seq":18958,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.925777+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:10.925777+0000","last_peered":"2024-09-10T16:01:10.925777+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:10.925777+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10809344,"num_objects":2639,"num_object_clones":0,"num_object_copies":5278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14733,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18504","reported_seq":19270,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:10.932693+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:10.932693+0000","last_peered":"2024-09-10T16:01:10.932693+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:10.932693+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10625507,"num_objects":2595,"num_object_clones":0,"num_object_copies":5190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15323,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":168767995,"num_objects":41205,"num_object_clones":0,"num_object_copies":82410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31738,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41205,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251845,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14910,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":527867904,"data_stored":527791113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626531,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548396,"kb_used_data":123848,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93823444,"statfs":{"total":96636764160,"available":96075206656,"internally_reserved":0,"allocated":126820352,"data_stored":126471479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":434695854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149813,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637604,"kb_used_data":186708,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93734236,"statfs":{"total":96636764160,"available":95983857664,"internally_reserved":0,"allocated":191188992,"data_stored":190901999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":461696685},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953584,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558848,"kb_used_data":207920,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93812992,"statfs":{"total":96636764160,"available":96064503808,"internally_reserved":0,"allocated":212910080,"data_stored":212619015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":189792256,"data_stored":189788643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":125747200,"data_stored":125681707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":212328448,"data_stored":212320763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:15.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:15.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'18250", 'reported_seq': 20220, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:10.916586+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:01:10.916586+0000', 'last_peered': '2024-09-10T16:01:10.916586+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:01:10.916586+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'8200", 'ondisk_log_start': "46'8200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10321920, 'num_objects': 2520, 'num_object_clones': 0, 'num_object_copies': 5040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3030, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2520, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18250, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 943, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -510}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:15.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'18044", 'reported_seq': 18524, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:10.742276+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:01:10.742276+0000', 'last_peered': '2024-09-10T16:01:10.742276+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:01:10.742276+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "44'8000", 'ondisk_log_start': "44'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10588184, 'num_objects': 2586, 'num_object_clones': 0, 'num_object_copies': 5172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2586, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16312, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1043}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:15.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:15.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'17770", 'reported_seq': 18541, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:10.878251+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:01:10.878251+0000', 'last_peered': '2024-09-10T16:01:10.878251+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:01:10.878251+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'7700", 'ondisk_log_start': "44'7700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10305536, 'num_objects': 2516, 'num_object_clones': 0, 'num_object_copies': 5032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2516, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14420, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1008}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:15.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'18010", 'reported_seq': 18835, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:10.918308+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:01:10.918308+0000', 'last_peered': '2024-09-10T16:01:10.918308+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:01:10.918308+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "44'8000", 'ondisk_log_start': "44'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10518528, 'num_objects': 2568, 'num_object_clones': 0, 'num_object_copies': 5136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2568, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14874, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -951}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:15.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'17867", 'reported_seq': 18290, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:10.786131+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:01:10.786131+0000', 'last_peered': '2024-09-10T16:01:10.786131+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:01:10.786131+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "44'7800", 'ondisk_log_start': "44'7800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10604544, 'num_objects': 2589, 'num_object_clones': 0, 'num_object_copies': 5178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2589, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15736, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1097}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:15.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:15.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'18242", 'reported_seq': 19090, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:10.726686+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:01:10.726686+0000', 'last_peered': '2024-09-10T16:01:10.726686+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:01:10.726686+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'8200", 'ondisk_log_start': "46'8200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10698752, 'num_objects': 2612, 'num_object_clones': 0, 'num_object_copies': 5224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15434, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -866}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:15.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'18164", 'reported_seq': 18920, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:10.841885+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:01:10.841885+0000', 'last_peered': '2024-09-10T16:01:10.841885+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:01:10.841885+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "44'8100", 'ondisk_log_start': "44'8100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10461184, 'num_objects': 2554, 'num_object_clones': 0, 'num_object_copies': 5108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2554, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14582, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1072}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:15.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'18127", 'reported_seq': 18958, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:10.925777+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:01:10.925777+0000', 'last_peered': '2024-09-10T16:01:10.925777+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:01:10.925777+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'8100", 'ondisk_log_start': "46'8100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10809344, 'num_objects': 2639, 'num_object_clones': 0, 'num_object_copies': 5278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2639, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14733, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -897}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:15.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'18504", 'reported_seq': 19270, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:10.932693+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:01:10.932693+0000', 'last_peered': '2024-09-10T16:01:10.932693+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:01:10.932693+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10625507, 'num_objects': 2595, 'num_object_clones': 0, 'num_object_copies': 5190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2595, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15323, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1099}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:15.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:16.594 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:16.595 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31738/82414 objects degraded (38.510%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":602,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":41207,"data_bytes":168768014,"bytes_used":1786724352,"bytes_avail":288123568128,"bytes_total":289910292480,"degraded_objects":31738,"degraded_total":82414,"degraded_ratio":0.38510447254107311,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":160,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":416},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:16.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:16.620 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:17.175 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:17.178 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:17.199 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-10T16:01:16.611652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":159924750,"num_objects":39048,"num_object_clones":0,"num_object_copies":78096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31664,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254006,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14986,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1724048,"kb_used_data":505992,"kb_used_omap":12,"kb_used_meta":1217971,"kb_avail":281391472,"statfs":{"total":289910292480,"available":288144867328,"internally_reserved":0,"allocated":518135808,"data_stored":517208877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1247202309},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,6,9,17,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29306880,"num_objects":-7155,"num_object_clones":0,"num_object_copies":-14310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7155,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004246"},"pg_stats":[{"pgid":"2.f","version":"50'18392","reported_seq":20514,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.957049+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:15.957049+0000","last_peered":"2024-09-10T16:01:15.957049+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:15.957049+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9740288,"num_objects":2378,"num_object_clones":0,"num_object_copies":4756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18392,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1017,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-578}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18665","reported_seq":18782,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.856393+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:15.856393+0000","last_peered":"2024-09-10T16:01:15.856393+0000","last_clean":"2024-09-10T16:01:15.856393+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:15.856393+0000","last_undegraded":"2024-09-10T16:01:15.856393+0000","last_fullsized":"2024-09-10T16:01:15.856393+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9867264,"num_objects":2409,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18187","reported_seq":18667,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.861043+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:15.861043+0000","last_peered":"2024-09-10T16:01:15.861043+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:15.861043+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10002456,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16455,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1186}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18247","reported_seq":18336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.894796+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:15.894796+0000","last_peered":"2024-09-10T16:01:15.894796+0000","last_clean":"2024-09-10T16:01:15.894796+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:15.894796+0000","last_undegraded":"2024-09-10T16:01:15.894796+0000","last_fullsized":"2024-09-10T16:01:15.894796+0000","mapping_epoch":36,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9973760,"num_objects":2435,"num_object_clones":0,"num_object_copies":4870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18247,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17922","reported_seq":18059,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.878045+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:15.878045+0000","last_peered":"2024-09-10T16:01:15.878045+0000","last_clean":"2024-09-10T16:01:15.878045+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:15.878045+0000","last_undegraded":"2024-09-10T16:01:15.878045+0000","last_fullsized":"2024-09-10T16:01:15.878045+0000","mapping_epoch":35,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9867264,"num_objects":2409,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14993,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18315","reported_seq":26212,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:11.492477+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:11.492477+0000","last_peered":"2024-09-10T16:01:11.492477+0000","last_clean":"2024-09-10T16:01:11.492477+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:11.492477+0000","last_undegraded":"2024-09-10T16:01:11.492477+0000","last_fullsized":"2024-09-10T16:01:11.492477+0000","mapping_epoch":47,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10452992,"num_objects":2552,"num_object_clones":0,"num_object_copies":5104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15033,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17929","reported_seq":18700,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.848028+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:15.848028+0000","last_peered":"2024-09-10T16:01:15.848028+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:15.848028+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9654272,"num_objects":2357,"num_object_clones":0,"num_object_copies":4714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14579,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18461","reported_seq":18602,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.905116+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:15.905116+0000","last_peered":"2024-09-10T16:01:15.905116+0000","last_clean":"2024-09-10T16:01:15.905116+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:15.905116+0000","last_undegraded":"2024-09-10T16:01:15.905116+0000","last_fullsized":"2024-09-10T16:01:15.905116+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10047488,"num_objects":2453,"num_object_clones":0,"num_object_copies":4906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15220,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18193","reported_seq":26330,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:11.558103+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:11.558103+0000","last_peered":"2024-09-10T16:01:11.558103+0000","last_clean":"2024-09-10T16:01:11.558103+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:11.558103+0000","last_undegraded":"2024-09-10T16:01:11.558103+0000","last_fullsized":"2024-09-10T16:01:11.558103+0000","mapping_epoch":45,"log_start":"44'8100","ondisk_log_start":"44'8100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10346496,"num_objects":2526,"num_object_clones":0,"num_object_copies":5052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16170,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18145","reported_seq":18969,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.940294+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:15.940294+0000","last_peered":"2024-09-10T16:01:15.940294+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:15.940294+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9965568,"num_objects":2433,"num_object_clones":0,"num_object_copies":4866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15009,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1086}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18319","reported_seq":26129,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.811952+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:15.811952+0000","last_peered":"2024-09-10T16:01:15.811952+0000","last_clean":"2024-09-10T16:01:15.811952+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:15.811952+0000","last_undegraded":"2024-09-10T16:01:15.811952+0000","last_fullsized":"2024-09-10T16:01:15.811952+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10100736,"num_objects":2466,"num_object_clones":0,"num_object_copies":4932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16293,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18020","reported_seq":18443,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.895852+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:15.895852+0000","last_peered":"2024-09-10T16:01:15.895852+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:15.895852+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9977856,"num_objects":2436,"num_object_clones":0,"num_object_copies":4872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15889,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1250}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18400","reported_seq":19248,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.894085+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:15.894085+0000","last_peered":"2024-09-10T16:01:15.894085+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:15.894085+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10051584,"num_objects":2454,"num_object_clones":0,"num_object_copies":4908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15592,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1024}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18329","reported_seq":19085,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.865027+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:15.865027+0000","last_peered":"2024-09-10T16:01:15.865027+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:15.865027+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9785344,"num_objects":2389,"num_object_clones":0,"num_object_copies":4778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14747,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18286","reported_seq":19118,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.943845+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:15.943845+0000","last_peered":"2024-09-10T16:01:15.943845+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:15.943845+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10158080,"num_objects":2480,"num_object_clones":0,"num_object_copies":4960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14892,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1056}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18673","reported_seq":19439,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.927041+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:15.927041+0000","last_peered":"2024-09-10T16:01:15.927041+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:15.927041+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9933283,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15492,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":159924731,"num_objects":39046,"num_object_clones":0,"num_object_copies":78092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31664,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39046,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254004,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14984,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":515051520,"data_stored":514974729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160883,"ondisk_log_size":160883,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626531,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548396,"kb_used_data":123848,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93823444,"statfs":{"total":96636764160,"available":96075206656,"internally_reserved":0,"allocated":126820352,"data_stored":126471479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":434695854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149814,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634296,"kb_used_data":179240,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93737544,"statfs":{"total":96636764160,"available":95987245056,"internally_reserved":0,"allocated":183541760,"data_stored":183254767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":465956525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953585,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541356,"kb_used_data":202904,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93830484,"statfs":{"total":96636764160,"available":96082415616,"internally_reserved":0,"allocated":207773696,"data_stored":207482631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":182038528,"data_stored":182034915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":125747200,"data_stored":125681707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":207265792,"data_stored":207258107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:20.202 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:20.829 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:20.830 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:20.850 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-10T16:01:20.613123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":158581262,"num_objects":38720,"num_object_clones":0,"num_object_copies":77440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31664,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254334,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14986,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1738788,"kb_used_data":498544,"kb_used_omap":12,"kb_used_meta":1240179,"kb_avail":281376732,"statfs":{"total":289910292480,"available":288129773568,"internally_reserved":0,"allocated":510509056,"data_stored":509582125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1269943301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,4,10,13,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20480000,"num_objects":-5000,"num_object_clones":0,"num_object_copies":-10000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5000,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004496"},"pg_stats":[{"pgid":"2.f","version":"50'18392","reported_seq":20514,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.957049+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:15.957049+0000","last_peered":"2024-09-10T16:01:15.957049+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:15.957049+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9740288,"num_objects":2378,"num_object_clones":0,"num_object_copies":4756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18392,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1017,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-578}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18665","reported_seq":18782,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.856393+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:15.856393+0000","last_peered":"2024-09-10T16:01:15.856393+0000","last_clean":"2024-09-10T16:01:15.856393+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:15.856393+0000","last_undegraded":"2024-09-10T16:01:15.856393+0000","last_fullsized":"2024-09-10T16:01:15.856393+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9867264,"num_objects":2409,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18187","reported_seq":18667,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.861043+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:15.861043+0000","last_peered":"2024-09-10T16:01:15.861043+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:15.861043+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10002456,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16455,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1186}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18247","reported_seq":18336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.894796+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:15.894796+0000","last_peered":"2024-09-10T16:01:15.894796+0000","last_clean":"2024-09-10T16:01:15.894796+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:15.894796+0000","last_undegraded":"2024-09-10T16:01:15.894796+0000","last_fullsized":"2024-09-10T16:01:15.894796+0000","mapping_epoch":36,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9973760,"num_objects":2435,"num_object_clones":0,"num_object_copies":4870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18247,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17922","reported_seq":18059,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.878045+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:15.878045+0000","last_peered":"2024-09-10T16:01:15.878045+0000","last_clean":"2024-09-10T16:01:15.878045+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:15.878045+0000","last_undegraded":"2024-09-10T16:01:15.878045+0000","last_fullsized":"2024-09-10T16:01:15.878045+0000","mapping_epoch":35,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9867264,"num_objects":2409,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14993,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18479","reported_seq":26377,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:16.535724+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:16.535724+0000","last_peered":"2024-09-10T16:01:16.535724+0000","last_clean":"2024-09-10T16:01:16.535724+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:16.535724+0000","last_undegraded":"2024-09-10T16:01:16.535724+0000","last_fullsized":"2024-09-10T16:01:16.535724+0000","mapping_epoch":47,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9781248,"num_objects":2388,"num_object_clones":0,"num_object_copies":4776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15197,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17929","reported_seq":18700,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.848028+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:15.848028+0000","last_peered":"2024-09-10T16:01:15.848028+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:15.848028+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9654272,"num_objects":2357,"num_object_clones":0,"num_object_copies":4714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14579,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18461","reported_seq":18602,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.905116+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:15.905116+0000","last_peered":"2024-09-10T16:01:15.905116+0000","last_clean":"2024-09-10T16:01:15.905116+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:15.905116+0000","last_undegraded":"2024-09-10T16:01:15.905116+0000","last_fullsized":"2024-09-10T16:01:15.905116+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10047488,"num_objects":2453,"num_object_clones":0,"num_object_copies":4906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15220,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18357","reported_seq":26494,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:16.635493+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:16.635493+0000","last_peered":"2024-09-10T16:01:16.635493+0000","last_clean":"2024-09-10T16:01:16.635493+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:16.635493+0000","last_undegraded":"2024-09-10T16:01:16.635493+0000","last_fullsized":"2024-09-10T16:01:16.635493+0000","mapping_epoch":45,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9674752,"num_objects":2362,"num_object_clones":0,"num_object_copies":4724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16334,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18145","reported_seq":18969,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.940294+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:15.940294+0000","last_peered":"2024-09-10T16:01:15.940294+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:15.940294+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9965568,"num_objects":2433,"num_object_clones":0,"num_object_copies":4866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15009,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1086}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18319","reported_seq":26129,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.811952+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:15.811952+0000","last_peered":"2024-09-10T16:01:15.811952+0000","last_clean":"2024-09-10T16:01:15.811952+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:15.811952+0000","last_undegraded":"2024-09-10T16:01:15.811952+0000","last_fullsized":"2024-09-10T16:01:15.811952+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10100736,"num_objects":2466,"num_object_clones":0,"num_object_copies":4932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16293,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18020","reported_seq":18443,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.895852+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:15.895852+0000","last_peered":"2024-09-10T16:01:15.895852+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:15.895852+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9977856,"num_objects":2436,"num_object_clones":0,"num_object_copies":4872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15889,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1250}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18400","reported_seq":19248,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.894085+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:15.894085+0000","last_peered":"2024-09-10T16:01:15.894085+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:15.894085+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10051584,"num_objects":2454,"num_object_clones":0,"num_object_copies":4908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15592,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1024}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18329","reported_seq":19085,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.865027+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:15.865027+0000","last_peered":"2024-09-10T16:01:15.865027+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:15.865027+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9785344,"num_objects":2389,"num_object_clones":0,"num_object_copies":4778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14747,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18286","reported_seq":19118,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.943845+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:15.943845+0000","last_peered":"2024-09-10T16:01:15.943845+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:15.943845+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10158080,"num_objects":2480,"num_object_clones":0,"num_object_copies":4960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14892,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1056}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18673","reported_seq":19439,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.927041+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:15.927041+0000","last_peered":"2024-09-10T16:01:15.927041+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:15.927041+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9933283,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15492,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":158581243,"num_objects":38718,"num_object_clones":0,"num_object_copies":77436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31664,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38718,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254332,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14984,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":507285504,"data_stored":507208713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626532,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563136,"kb_used_data":116400,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93808704,"statfs":{"total":96636764160,"available":96060112896,"internally_reserved":0,"allocated":119193600,"data_stored":118844727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457436846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149814,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634296,"kb_used_data":179240,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93737544,"statfs":{"total":96636764160,"available":95987245056,"internally_reserved":0,"allocated":183541760,"data_stored":183254767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":465956525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953585,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541356,"kb_used_data":202904,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93830484,"statfs":{"total":96636764160,"available":96082415616,"internally_reserved":0,"allocated":207773696,"data_stored":207482631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":182038528,"data_stored":182034915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":117981184,"data_stored":117915691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":207265792,"data_stored":207258107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:20.852 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:21.401 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:21.402 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:21.420 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-10T16:01:20.613123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":158581262,"num_objects":38720,"num_object_clones":0,"num_object_copies":77440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31664,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254334,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14986,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1738788,"kb_used_data":498544,"kb_used_omap":12,"kb_used_meta":1240179,"kb_avail":281376732,"statfs":{"total":289910292480,"available":288129773568,"internally_reserved":0,"allocated":510509056,"data_stored":509582125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1269943301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,4,10,13,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20480000,"num_objects":-5000,"num_object_clones":0,"num_object_copies":-10000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5000,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004496"},"pg_stats":[{"pgid":"2.f","version":"50'18392","reported_seq":20514,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.957049+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:15.957049+0000","last_peered":"2024-09-10T16:01:15.957049+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:15.957049+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9740288,"num_objects":2378,"num_object_clones":0,"num_object_copies":4756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18392,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1017,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-578}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18665","reported_seq":18782,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.856393+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:15.856393+0000","last_peered":"2024-09-10T16:01:15.856393+0000","last_clean":"2024-09-10T16:01:15.856393+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:15.856393+0000","last_undegraded":"2024-09-10T16:01:15.856393+0000","last_fullsized":"2024-09-10T16:01:15.856393+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9867264,"num_objects":2409,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18187","reported_seq":18667,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.861043+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:15.861043+0000","last_peered":"2024-09-10T16:01:15.861043+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:15.861043+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10002456,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16455,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1186}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18247","reported_seq":18336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.894796+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:15.894796+0000","last_peered":"2024-09-10T16:01:15.894796+0000","last_clean":"2024-09-10T16:01:15.894796+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:15.894796+0000","last_undegraded":"2024-09-10T16:01:15.894796+0000","last_fullsized":"2024-09-10T16:01:15.894796+0000","mapping_epoch":36,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9973760,"num_objects":2435,"num_object_clones":0,"num_object_copies":4870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18247,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'17922","reported_seq":18059,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.878045+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:15.878045+0000","last_peered":"2024-09-10T16:01:15.878045+0000","last_clean":"2024-09-10T16:01:15.878045+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:15.878045+0000","last_undegraded":"2024-09-10T16:01:15.878045+0000","last_fullsized":"2024-09-10T16:01:15.878045+0000","mapping_epoch":35,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9867264,"num_objects":2409,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14993,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18479","reported_seq":26377,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:16.535724+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:16.535724+0000","last_peered":"2024-09-10T16:01:16.535724+0000","last_clean":"2024-09-10T16:01:16.535724+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:16.535724+0000","last_undegraded":"2024-09-10T16:01:16.535724+0000","last_fullsized":"2024-09-10T16:01:16.535724+0000","mapping_epoch":47,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9781248,"num_objects":2388,"num_object_clones":0,"num_object_copies":4776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15197,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'17929","reported_seq":18700,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.848028+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:15.848028+0000","last_peered":"2024-09-10T16:01:15.848028+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:15.848028+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"44'7900","ondisk_log_start":"44'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9654272,"num_objects":2357,"num_object_clones":0,"num_object_copies":4714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14579,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18461","reported_seq":18602,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.905116+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:15.905116+0000","last_peered":"2024-09-10T16:01:15.905116+0000","last_clean":"2024-09-10T16:01:15.905116+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:15.905116+0000","last_undegraded":"2024-09-10T16:01:15.905116+0000","last_fullsized":"2024-09-10T16:01:15.905116+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10047488,"num_objects":2453,"num_object_clones":0,"num_object_copies":4906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15220,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951489+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951489+0000","last_peered":"2024-09-10T16:00:38.951489+0000","last_clean":"2024-09-10T16:00:38.951489+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951489+0000","last_undegraded":"2024-09-10T16:00:38.951489+0000","last_fullsized":"2024-09-10T16:00:38.951489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18357","reported_seq":26494,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:16.635493+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:16.635493+0000","last_peered":"2024-09-10T16:01:16.635493+0000","last_clean":"2024-09-10T16:01:16.635493+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:16.635493+0000","last_undegraded":"2024-09-10T16:01:16.635493+0000","last_fullsized":"2024-09-10T16:01:16.635493+0000","mapping_epoch":45,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9674752,"num_objects":2362,"num_object_clones":0,"num_object_copies":4724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16334,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18145","reported_seq":18969,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.940294+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:15.940294+0000","last_peered":"2024-09-10T16:01:15.940294+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:15.940294+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9965568,"num_objects":2433,"num_object_clones":0,"num_object_copies":4866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15009,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1086}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18319","reported_seq":26129,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:15.811952+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:15.811952+0000","last_peered":"2024-09-10T16:01:15.811952+0000","last_clean":"2024-09-10T16:01:15.811952+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:15.811952+0000","last_undegraded":"2024-09-10T16:01:15.811952+0000","last_fullsized":"2024-09-10T16:01:15.811952+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10100736,"num_objects":2466,"num_object_clones":0,"num_object_copies":4932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16293,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18020","reported_seq":18443,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.895852+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:15.895852+0000","last_peered":"2024-09-10T16:01:15.895852+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:15.895852+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9977856,"num_objects":2436,"num_object_clones":0,"num_object_copies":4872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15889,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1250}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18400","reported_seq":19248,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.894085+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:15.894085+0000","last_peered":"2024-09-10T16:01:15.894085+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:15.894085+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10051584,"num_objects":2454,"num_object_clones":0,"num_object_copies":4908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15592,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1024}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18329","reported_seq":19085,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.865027+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:15.865027+0000","last_peered":"2024-09-10T16:01:15.865027+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:15.865027+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9785344,"num_objects":2389,"num_object_clones":0,"num_object_copies":4778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14747,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1237}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18286","reported_seq":19118,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.943845+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:15.943845+0000","last_peered":"2024-09-10T16:01:15.943845+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:15.943845+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10158080,"num_objects":2480,"num_object_clones":0,"num_object_copies":4960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14892,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1056}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18673","reported_seq":19439,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:15.927041+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:15.927041+0000","last_peered":"2024-09-10T16:01:15.927041+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:15.927041+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9933283,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15492,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":158581243,"num_objects":38718,"num_object_clones":0,"num_object_copies":77436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31664,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38718,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254332,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14984,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":507285504,"data_stored":507208713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626532,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563136,"kb_used_data":116400,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93808704,"statfs":{"total":96636764160,"available":96060112896,"internally_reserved":0,"allocated":119193600,"data_stored":118844727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457436846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149814,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634296,"kb_used_data":179240,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93737544,"statfs":{"total":96636764160,"available":95987245056,"internally_reserved":0,"allocated":183541760,"data_stored":183254767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":465956525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953585,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541356,"kb_used_data":202904,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93830484,"statfs":{"total":96636764160,"available":96082415616,"internally_reserved":0,"allocated":207773696,"data_stored":207482631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":182038528,"data_stored":182034915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":117981184,"data_stored":117915691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":207265792,"data_stored":207258107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:21.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:21.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'18392", 'reported_seq': 20514, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:15.957049+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:01:15.957049+0000', 'last_peered': '2024-09-10T16:01:15.957049+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:01:15.957049+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9740288, 'num_objects': 2378, 'num_object_clones': 0, 'num_object_copies': 4756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2956, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2378, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18392, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1017, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -578}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:21.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'18187", 'reported_seq': 18667, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:15.861043+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:01:15.861043+0000', 'last_peered': '2024-09-10T16:01:15.861043+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:01:15.861043+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'8100", 'ondisk_log_start': "46'8100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10002456, 'num_objects': 2443, 'num_object_clones': 0, 'num_object_copies': 4886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2443, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16455, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1186}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:21.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:21.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'17929", 'reported_seq': 18700, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:15.848028+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:01:15.848028+0000', 'last_peered': '2024-09-10T16:01:15.848028+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:01:15.848028+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "44'7900", 'ondisk_log_start': "44'7900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9654272, 'num_objects': 2357, 'num_object_clones': 0, 'num_object_copies': 4714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2357, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14579, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1167}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:21.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'18145", 'reported_seq': 18969, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:15.940294+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:01:15.940294+0000', 'last_peered': '2024-09-10T16:01:15.940294+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:01:15.940294+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'8100", 'ondisk_log_start': "46'8100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9965568, 'num_objects': 2433, 'num_object_clones': 0, 'num_object_copies': 4866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2433, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15009, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1086}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:21.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'18020", 'reported_seq': 18443, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:15.895852+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:01:15.895852+0000', 'last_peered': '2024-09-10T16:01:15.895852+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:01:15.895852+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9977856, 'num_objects': 2436, 'num_object_clones': 0, 'num_object_copies': 4872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2436, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15889, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1250}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:21.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:21.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'18400", 'reported_seq': 19248, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:15.894085+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:01:15.894085+0000', 'last_peered': '2024-09-10T16:01:15.894085+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:01:15.894085+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10051584, 'num_objects': 2454, 'num_object_clones': 0, 'num_object_copies': 4908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2454, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15592, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1024}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:21.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'18329", 'reported_seq': 19085, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:15.865027+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:01:15.865027+0000', 'last_peered': '2024-09-10T16:01:15.865027+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:01:15.865027+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9785344, 'num_objects': 2389, 'num_object_clones': 0, 'num_object_copies': 4778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2389, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14747, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:21.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'18286", 'reported_seq': 19118, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:15.943845+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:01:15.943845+0000', 'last_peered': '2024-09-10T16:01:15.943845+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:01:15.943845+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'8200", 'ondisk_log_start': "46'8200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10158080, 'num_objects': 2480, 'num_object_clones': 0, 'num_object_copies': 4960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2480, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14892, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1056}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:21.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'18673", 'reported_seq': 19439, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:15.927041+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:01:15.927041+0000', 'last_peered': '2024-09-10T16:01:15.927041+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:01:15.927041+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'8600", 'ondisk_log_start': "46'8600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9933283, 'num_objects': 2426, 'num_object_clones': 0, 'num_object_copies': 4852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15492, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:21.424 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:22.635 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31664/77440 objects degraded (40.888%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":608,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":38720,"data_bytes":158581262,"bytes_used":1780518912,"bytes_avail":288129773568,"bytes_total":289910292480,"degraded_objects":31664,"degraded_total":77440,"degraded_ratio":0.40888429752066119,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":416},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:22.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:22.659 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:23.196 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:23.196 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:23.215 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-10T16:01:22.613745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":148132366,"num_objects":36169,"num_object_clones":0,"num_object_copies":72338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36169,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256885,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15066,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1722976,"kb_used_data":478548,"kb_used_omap":12,"kb_used_meta":1244339,"kb_avail":281392544,"statfs":{"total":289910292480,"available":288145965056,"internally_reserved":0,"allocated":490033152,"data_stored":489106221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1274203141},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,15,4,6,11,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30928896,"num_objects":-7551,"num_object_clones":0,"num_object_copies":-15102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-231,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7551,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004358"},"pg_stats":[{"pgid":"2.f","version":"50'18536","reported_seq":20817,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.167188+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:21.167188+0000","last_peered":"2024-09-10T16:01:21.167188+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:21.167188+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9150464,"num_objects":2234,"num_object_clones":0,"num_object_copies":4468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2876,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18536,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1097,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-642}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18819","reported_seq":18934,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:20.843261+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:20.843261+0000","last_peered":"2024-09-10T16:01:20.843261+0000","last_clean":"2024-09-10T16:01:20.843261+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:20.843261+0000","last_undegraded":"2024-09-10T16:01:20.843261+0000","last_fullsized":"2024-09-10T16:01:20.843261+0000","mapping_epoch":36,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9236480,"num_objects":2255,"num_object_clones":0,"num_object_copies":4510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18325","reported_seq":18805,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:20.878205+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:20.878205+0000","last_peered":"2024-09-10T16:01:20.878205+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:20.878205+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437208,"num_objects":2305,"num_object_clones":0,"num_object_copies":4610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2305,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16593,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1324}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18393","reported_seq":18482,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:20.778016+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:20.778016+0000","last_peered":"2024-09-10T16:01:20.778016+0000","last_clean":"2024-09-10T16:01:20.778016+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:20.778016+0000","last_undegraded":"2024-09-10T16:01:20.778016+0000","last_fullsized":"2024-09-10T16:01:20.778016+0000","mapping_epoch":36,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9375744,"num_objects":2289,"num_object_clones":0,"num_object_copies":4578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18393,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18091","reported_seq":18228,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:21.067278+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:21.067278+0000","last_peered":"2024-09-10T16:01:21.067278+0000","last_clean":"2024-09-10T16:01:21.067278+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:21.067278+0000","last_undegraded":"2024-09-10T16:01:21.067278+0000","last_fullsized":"2024-09-10T16:01:21.067278+0000","mapping_epoch":35,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9175040,"num_objects":2240,"num_object_clones":0,"num_object_copies":4480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15162,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18655","reported_seq":26553,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:21.637121+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:21.637121+0000","last_peered":"2024-09-10T16:01:21.637121+0000","last_clean":"2024-09-10T16:01:21.637121+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:21.637121+0000","last_undegraded":"2024-09-10T16:01:21.637121+0000","last_fullsized":"2024-09-10T16:01:21.637121+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9060352,"num_objects":2212,"num_object_clones":0,"num_object_copies":4424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15373,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18079","reported_seq":18850,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:20.927539+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:20.927539+0000","last_peered":"2024-09-10T16:01:20.927539+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:20.927539+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9039872,"num_objects":2207,"num_object_clones":0,"num_object_copies":4414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14729,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1317}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18624","reported_seq":18765,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:20.975450+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:20.975450+0000","last_peered":"2024-09-10T16:01:20.975450+0000","last_clean":"2024-09-10T16:01:20.975450+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:20.975450+0000","last_undegraded":"2024-09-10T16:01:20.975450+0000","last_fullsized":"2024-09-10T16:01:20.975450+0000","mapping_epoch":35,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9379840,"num_objects":2290,"num_object_clones":0,"num_object_copies":4580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15383,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18515","reported_seq":26652,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:21.673962+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:21.673962+0000","last_peered":"2024-09-10T16:01:21.673962+0000","last_clean":"2024-09-10T16:01:21.673962+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:21.673962+0000","last_undegraded":"2024-09-10T16:01:21.673962+0000","last_fullsized":"2024-09-10T16:01:21.673962+0000","mapping_epoch":45,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9027584,"num_objects":2204,"num_object_clones":0,"num_object_copies":4408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16492,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18309","reported_seq":19134,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.029165+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:21.029165+0000","last_peered":"2024-09-10T16:01:21.029165+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:21.029165+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9293824,"num_objects":2269,"num_object_clones":0,"num_object_copies":4538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15173,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1250}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18477","reported_seq":26285,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:20.844437+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:20.844437+0000","last_peered":"2024-09-10T16:01:20.844437+0000","last_clean":"2024-09-10T16:01:20.844437+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:20.844437+0000","last_undegraded":"2024-09-10T16:01:20.844437+0000","last_fullsized":"2024-09-10T16:01:20.844437+0000","mapping_epoch":49,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9453568,"num_objects":2308,"num_object_clones":0,"num_object_copies":4616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16451,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18185","reported_seq":18608,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:20.809221+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:20.809221+0000","last_peered":"2024-09-10T16:01:20.809221+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:20.809221+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9302016,"num_objects":2271,"num_object_clones":0,"num_object_copies":4542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16054,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1415}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18576","reported_seq":19424,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.088006+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:21.088006+0000","last_peered":"2024-09-10T16:01:21.088006+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:21.088006+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9330688,"num_objects":2278,"num_object_clones":0,"num_object_copies":4556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15768,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1200}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18474","reported_seq":19230,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.121685+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:21.121685+0000","last_peered":"2024-09-10T16:01:21.121685+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:21.121685+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9191424,"num_objects":2244,"num_object_clones":0,"num_object_copies":4488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14892,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18446","reported_seq":19278,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.133552+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:21.133552+0000","last_peered":"2024-09-10T16:01:21.133552+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:21.133552+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9502720,"num_objects":2320,"num_object_clones":0,"num_object_copies":4640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15052,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1216}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18858","reported_seq":19621,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:21.145005+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:21.145005+0000","last_peered":"2024-09-10T16:01:21.145005+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:21.145005+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9175523,"num_objects":2241,"num_object_clones":0,"num_object_copies":4482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1453}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":148132347,"num_objects":36167,"num_object_clones":0,"num_object_copies":72334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36167,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256883,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15064,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":486707200,"data_stored":486630409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626533,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563896,"kb_used_data":108828,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93807944,"statfs":{"total":96636764160,"available":96059334656,"internally_reserved":0,"allocated":111439872,"data_stored":111090999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":465956526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149815,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618520,"kb_used_data":171768,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93753320,"statfs":{"total":96636764160,"available":96003399680,"internally_reserved":0,"allocated":175890432,"data_stored":175603439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":457436845},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,0,2,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953586,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540560,"kb_used_data":197952,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93831280,"statfs":{"total":96636764160,"available":96083230720,"internally_reserved":0,"allocated":202702848,"data_stored":202411783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,10,1,0,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":174206976,"data_stored":174203363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110288896,"data_stored":110223403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":202211328,"data_stored":202203643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:26.217 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:26.797 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:26.800 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:26.824 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-10T16:01:26.614975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":139559438,"num_objects":34076,"num_object_clones":0,"num_object_copies":68152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34076,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258978,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15139,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1719624,"kb_used_data":466884,"kb_used_omap":12,"kb_used_meta":1252659,"kb_avail":281395896,"statfs":{"total":289910292480,"available":288149397504,"internally_reserved":0,"allocated":478089216,"data_stored":477162285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1282722821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,8,7,10,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29208576,"num_objects":-7131,"num_object_clones":0,"num_object_copies":-14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003966"},"pg_stats":[{"pgid":"2.f","version":"50'18677","reported_seq":21103,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.208013+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:26.208013+0000","last_peered":"2024-09-10T16:01:26.208013+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:26.208013+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8572928,"num_objects":2093,"num_object_clones":0,"num_object_copies":4186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18677,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1170,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-710}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18961","reported_seq":19078,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:25.911839+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:25.911839+0000","last_peered":"2024-09-10T16:01:25.911839+0000","last_clean":"2024-09-10T16:01:25.911839+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:25.911839+0000","last_undegraded":"2024-09-10T16:01:25.911839+0000","last_fullsized":"2024-09-10T16:01:25.911839+0000","mapping_epoch":36,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8654848,"num_objects":2113,"num_object_clones":0,"num_object_copies":4226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17297,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18467","reported_seq":18946,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:25.917490+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:25.917490+0000","last_peered":"2024-09-10T16:01:25.917490+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:25.917490+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8855576,"num_objects":2163,"num_object_clones":0,"num_object_copies":4326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16735,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1466}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18554","reported_seq":18644,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:25.909920+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:25.909920+0000","last_peered":"2024-09-10T16:01:25.909920+0000","last_clean":"2024-09-10T16:01:25.909920+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:25.909920+0000","last_undegraded":"2024-09-10T16:01:25.909920+0000","last_fullsized":"2024-09-10T16:01:25.909920+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8716288,"num_objects":2128,"num_object_clones":0,"num_object_copies":4256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18554,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18228","reported_seq":18365,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.100216+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:26.100216+0000","last_peered":"2024-09-10T16:01:26.100216+0000","last_clean":"2024-09-10T16:01:26.100216+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:26.100216+0000","last_undegraded":"2024-09-10T16:01:26.100216+0000","last_fullsized":"2024-09-10T16:01:26.100216+0000","mapping_epoch":35,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8613888,"num_objects":2103,"num_object_clones":0,"num_object_copies":4206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15299,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18655","reported_seq":26553,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:21.637121+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:21.637121+0000","last_peered":"2024-09-10T16:01:21.637121+0000","last_clean":"2024-09-10T16:01:21.637121+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:21.637121+0000","last_undegraded":"2024-09-10T16:01:21.637121+0000","last_fullsized":"2024-09-10T16:01:21.637121+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9060352,"num_objects":2212,"num_object_clones":0,"num_object_copies":4424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15373,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18235","reported_seq":19006,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.087628+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:26.087628+0000","last_peered":"2024-09-10T16:01:26.087628+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:26.087628+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8400896,"num_objects":2051,"num_object_clones":0,"num_object_copies":4102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14885,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1473}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18778","reported_seq":18919,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.144593+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:26.144593+0000","last_peered":"2024-09-10T16:01:26.144593+0000","last_clean":"2024-09-10T16:01:26.144593+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:26.144593+0000","last_undegraded":"2024-09-10T16:01:26.144593+0000","last_fullsized":"2024-09-10T16:01:26.144593+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8749056,"num_objects":2136,"num_object_clones":0,"num_object_copies":4272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15537,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18515","reported_seq":26652,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:21.673962+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:21.673962+0000","last_peered":"2024-09-10T16:01:21.673962+0000","last_clean":"2024-09-10T16:01:21.673962+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:21.673962+0000","last_undegraded":"2024-09-10T16:01:21.673962+0000","last_fullsized":"2024-09-10T16:01:21.673962+0000","mapping_epoch":45,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9027584,"num_objects":2204,"num_object_clones":0,"num_object_copies":4408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16492,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18462","reported_seq":19287,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.049040+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:26.049040+0000","last_peered":"2024-09-10T16:01:26.049040+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:26.049040+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8667136,"num_objects":2116,"num_object_clones":0,"num_object_copies":4232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15326,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1403}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18618","reported_seq":26424,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:25.814619+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:25.814619+0000","last_peered":"2024-09-10T16:01:25.814619+0000","last_clean":"2024-09-10T16:01:25.814619+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:25.814619+0000","last_undegraded":"2024-09-10T16:01:25.814619+0000","last_fullsized":"2024-09-10T16:01:25.814619+0000","mapping_epoch":49,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8876032,"num_objects":2167,"num_object_clones":0,"num_object_copies":4334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16592,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18349","reported_seq":18772,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:25.899208+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:25.899208+0000","last_peered":"2024-09-10T16:01:25.899208+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:25.899208+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8630272,"num_objects":2107,"num_object_clones":0,"num_object_copies":4214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16218,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1579}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18723","reported_seq":19571,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.165534+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:26.165534+0000","last_peered":"2024-09-10T16:01:26.165534+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:26.165534+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8728576,"num_objects":2131,"num_object_clones":0,"num_object_copies":4262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15915,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18633","reported_seq":19389,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.166470+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:26.166470+0000","last_peered":"2024-09-10T16:01:26.166470+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:26.166470+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8540160,"num_objects":2085,"num_object_clones":0,"num_object_copies":4170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15051,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1541}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18606","reported_seq":19438,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.098228+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:26.098228+0000","last_peered":"2024-09-10T16:01:26.098228+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:26.098228+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8847360,"num_objects":2160,"num_object_clones":0,"num_object_copies":4320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15212,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18994","reported_seq":19761,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.135885+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:26.135885+0000","last_peered":"2024-09-10T16:01:26.135885+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:26.135885+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8618467,"num_objects":2105,"num_object_clones":0,"num_object_copies":4210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15813,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1589}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":139559419,"num_objects":34074,"num_object_clones":0,"num_object_copies":68148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258976,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15137,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":474451968,"data_stored":474375177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626533,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563896,"kb_used_data":108828,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93807944,"statfs":{"total":96636764160,"available":96059334656,"internally_reserved":0,"allocated":111439872,"data_stored":111090999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":465956526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149816,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615580,"kb_used_data":164684,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93756260,"statfs":{"total":96636764160,"available":96006410240,"internally_reserved":0,"allocated":168636416,"data_stored":168349423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":461696685},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953587,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540148,"kb_used_data":193372,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93831692,"statfs":{"total":96636764160,"available":96083652608,"internally_reserved":0,"allocated":198012928,"data_stored":197721863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166801891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110288896,"data_stored":110223403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":197357568,"data_stored":197349883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:26.826 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:27.371 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:27.371 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:27.394 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-10T16:01:26.614975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":139559438,"num_objects":34076,"num_object_clones":0,"num_object_copies":68152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34076,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258978,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15139,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1719624,"kb_used_data":466884,"kb_used_omap":12,"kb_used_meta":1252659,"kb_avail":281395896,"statfs":{"total":289910292480,"available":288149397504,"internally_reserved":0,"allocated":478089216,"data_stored":477162285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1282722821},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,8,7,10,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29208576,"num_objects":-7131,"num_object_clones":0,"num_object_copies":-14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003966"},"pg_stats":[{"pgid":"2.f","version":"50'18677","reported_seq":21103,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.208013+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:26.208013+0000","last_peered":"2024-09-10T16:01:26.208013+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:26.208013+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8572928,"num_objects":2093,"num_object_clones":0,"num_object_copies":4186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18677,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1170,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-710}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18961","reported_seq":19078,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:25.911839+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:25.911839+0000","last_peered":"2024-09-10T16:01:25.911839+0000","last_clean":"2024-09-10T16:01:25.911839+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:25.911839+0000","last_undegraded":"2024-09-10T16:01:25.911839+0000","last_fullsized":"2024-09-10T16:01:25.911839+0000","mapping_epoch":36,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8654848,"num_objects":2113,"num_object_clones":0,"num_object_copies":4226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17297,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18467","reported_seq":18946,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:25.917490+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:25.917490+0000","last_peered":"2024-09-10T16:01:25.917490+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:25.917490+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8855576,"num_objects":2163,"num_object_clones":0,"num_object_copies":4326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16735,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1466}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18554","reported_seq":18644,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:25.909920+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:25.909920+0000","last_peered":"2024-09-10T16:01:25.909920+0000","last_clean":"2024-09-10T16:01:25.909920+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:25.909920+0000","last_undegraded":"2024-09-10T16:01:25.909920+0000","last_fullsized":"2024-09-10T16:01:25.909920+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8716288,"num_objects":2128,"num_object_clones":0,"num_object_copies":4256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18554,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18228","reported_seq":18365,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.100216+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:26.100216+0000","last_peered":"2024-09-10T16:01:26.100216+0000","last_clean":"2024-09-10T16:01:26.100216+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:26.100216+0000","last_undegraded":"2024-09-10T16:01:26.100216+0000","last_fullsized":"2024-09-10T16:01:26.100216+0000","mapping_epoch":35,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8613888,"num_objects":2103,"num_object_clones":0,"num_object_copies":4206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15299,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18655","reported_seq":26553,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:21.637121+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:21.637121+0000","last_peered":"2024-09-10T16:01:21.637121+0000","last_clean":"2024-09-10T16:01:21.637121+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:21.637121+0000","last_undegraded":"2024-09-10T16:01:21.637121+0000","last_fullsized":"2024-09-10T16:01:21.637121+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9060352,"num_objects":2212,"num_object_clones":0,"num_object_copies":4424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15373,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18235","reported_seq":19006,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.087628+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:26.087628+0000","last_peered":"2024-09-10T16:01:26.087628+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:26.087628+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8400896,"num_objects":2051,"num_object_clones":0,"num_object_copies":4102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14885,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1473}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18778","reported_seq":18919,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.144593+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:26.144593+0000","last_peered":"2024-09-10T16:01:26.144593+0000","last_clean":"2024-09-10T16:01:26.144593+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:26.144593+0000","last_undegraded":"2024-09-10T16:01:26.144593+0000","last_fullsized":"2024-09-10T16:01:26.144593+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8749056,"num_objects":2136,"num_object_clones":0,"num_object_copies":4272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15537,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18515","reported_seq":26652,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:21.673962+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:21.673962+0000","last_peered":"2024-09-10T16:01:21.673962+0000","last_clean":"2024-09-10T16:01:21.673962+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:21.673962+0000","last_undegraded":"2024-09-10T16:01:21.673962+0000","last_fullsized":"2024-09-10T16:01:21.673962+0000","mapping_epoch":45,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9027584,"num_objects":2204,"num_object_clones":0,"num_object_copies":4408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16492,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18462","reported_seq":19287,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.049040+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:26.049040+0000","last_peered":"2024-09-10T16:01:26.049040+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:26.049040+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8667136,"num_objects":2116,"num_object_clones":0,"num_object_copies":4232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15326,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1403}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18618","reported_seq":26424,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:25.814619+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:25.814619+0000","last_peered":"2024-09-10T16:01:25.814619+0000","last_clean":"2024-09-10T16:01:25.814619+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:25.814619+0000","last_undegraded":"2024-09-10T16:01:25.814619+0000","last_fullsized":"2024-09-10T16:01:25.814619+0000","mapping_epoch":49,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8876032,"num_objects":2167,"num_object_clones":0,"num_object_copies":4334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16592,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18349","reported_seq":18772,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:25.899208+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:25.899208+0000","last_peered":"2024-09-10T16:01:25.899208+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:25.899208+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8630272,"num_objects":2107,"num_object_clones":0,"num_object_copies":4214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16218,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1579}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18723","reported_seq":19571,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.165534+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:26.165534+0000","last_peered":"2024-09-10T16:01:26.165534+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:26.165534+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8728576,"num_objects":2131,"num_object_clones":0,"num_object_copies":4262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15915,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18633","reported_seq":19389,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.166470+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:26.166470+0000","last_peered":"2024-09-10T16:01:26.166470+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:26.166470+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8540160,"num_objects":2085,"num_object_clones":0,"num_object_copies":4170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15051,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1541}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18606","reported_seq":19438,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.098228+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:26.098228+0000","last_peered":"2024-09-10T16:01:26.098228+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:26.098228+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8847360,"num_objects":2160,"num_object_clones":0,"num_object_copies":4320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15212,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18994","reported_seq":19761,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.135885+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:26.135885+0000","last_peered":"2024-09-10T16:01:26.135885+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:26.135885+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8618467,"num_objects":2105,"num_object_clones":0,"num_object_copies":4210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15813,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1589}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":139559419,"num_objects":34074,"num_object_clones":0,"num_object_copies":68148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258976,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15137,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":474451968,"data_stored":474375177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626533,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563896,"kb_used_data":108828,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93807944,"statfs":{"total":96636764160,"available":96059334656,"internally_reserved":0,"allocated":111439872,"data_stored":111090999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":465956526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149816,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615580,"kb_used_data":164684,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93756260,"statfs":{"total":96636764160,"available":96006410240,"internally_reserved":0,"allocated":168636416,"data_stored":168349423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":461696685},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953587,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540148,"kb_used_data":193372,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93831692,"statfs":{"total":96636764160,"available":96083652608,"internally_reserved":0,"allocated":198012928,"data_stored":197721863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166801891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110288896,"data_stored":110223403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":197357568,"data_stored":197349883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:27.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:27.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'18677", 'reported_seq': 21103, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.208013+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:01:26.208013+0000', 'last_peered': '2024-09-10T16:01:26.208013+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:01:26.208013+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'8600", 'ondisk_log_start': "46'8600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8572928, 'num_objects': 2093, 'num_object_clones': 0, 'num_object_copies': 4186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2803, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2093, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18677, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1170, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -710}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:27.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:27.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'18467", 'reported_seq': 18946, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:25.917490+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:01:25.917490+0000', 'last_peered': '2024-09-10T16:01:25.917490+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:01:25.917490+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'8400", 'ondisk_log_start': "46'8400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8855576, 'num_objects': 2163, 'num_object_clones': 0, 'num_object_copies': 4326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2163, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16735, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1466}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:27.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:27.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'18235", 'reported_seq': 19006, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.087628+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:01:26.087628+0000', 'last_peered': '2024-09-10T16:01:26.087628+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:01:26.087628+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'8200", 'ondisk_log_start': "46'8200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8400896, 'num_objects': 2051, 'num_object_clones': 0, 'num_object_copies': 4102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14885, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1473}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:27.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:27.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'18462", 'reported_seq': 19287, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.049040+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:01:26.049040+0000', 'last_peered': '2024-09-10T16:01:26.049040+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:01:26.049040+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'8400", 'ondisk_log_start': "46'8400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8667136, 'num_objects': 2116, 'num_object_clones': 0, 'num_object_copies': 4232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2116, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15326, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1403}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:27.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:27.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'18349", 'reported_seq': 18772, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:25.899208+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:01:25.899208+0000', 'last_peered': '2024-09-10T16:01:25.899208+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:01:25.899208+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8630272, 'num_objects': 2107, 'num_object_clones': 0, 'num_object_copies': 4214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2107, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16218, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1579}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:27.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:27.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'18723", 'reported_seq': 19571, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.165534+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:01:26.165534+0000', 'last_peered': '2024-09-10T16:01:26.165534+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:01:26.165534+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8728576, 'num_objects': 2131, 'num_object_clones': 0, 'num_object_copies': 4262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15915, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1347}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:27.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:27.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'18633", 'reported_seq': 19389, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.166470+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:01:26.166470+0000', 'last_peered': '2024-09-10T16:01:26.166470+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:01:26.166470+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'8600", 'ondisk_log_start': "46'8600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8540160, 'num_objects': 2085, 'num_object_clones': 0, 'num_object_copies': 4170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2085, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15051, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1541}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:27.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:27.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'18606", 'reported_seq': 19438, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.098228+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:01:26.098228+0000', 'last_peered': '2024-09-10T16:01:26.098228+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:01:26.098228+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'8600", 'ondisk_log_start': "46'8600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8847360, 'num_objects': 2160, 'num_object_clones': 0, 'num_object_copies': 4320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15212, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1376}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:27.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:27.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'18994", 'reported_seq': 19761, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:26.135885+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:01:26.135885+0000', 'last_peered': '2024-09-10T16:01:26.135885+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:01:26.135885+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8618467, 'num_objects': 2105, 'num_object_clones': 0, 'num_object_copies': 4210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15813, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1589}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:27.397 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:28.076 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:28.076 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31511/68152 objects degraded (46.236%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":614,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":34076,"data_bytes":139559438,"bytes_used":1760894976,"bytes_avail":288149397504,"bytes_total":289910292480,"degraded_objects":31511,"degraded_total":68152,"degraded_ratio":0.46236354032163407,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":227,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":594},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:28.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:28.112 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:28.669 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:28.669 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:28.693 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-10T16:01:28.615591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":138396174,"num_objects":33792,"num_object_clones":0,"num_object_copies":67584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33792,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259262,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15139,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160842,"ondisk_log_size":160842,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1700176,"kb_used_data":459924,"kb_used_omap":12,"kb_used_meta":1240179,"kb_avail":281415344,"statfs":{"total":289910292480,"available":288169312256,"internally_reserved":0,"allocated":470962176,"data_stored":470035245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1269943301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,7,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21528576,"num_objects":-5256,"num_object_clones":0,"num_object_copies":-10512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5256,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003938"},"pg_stats":[{"pgid":"2.f","version":"50'18677","reported_seq":21103,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.208013+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:26.208013+0000","last_peered":"2024-09-10T16:01:26.208013+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:26.208013+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8572928,"num_objects":2093,"num_object_clones":0,"num_object_copies":4186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2803,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18677,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1170,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-710}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'18961","reported_seq":19078,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:25.911839+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:25.911839+0000","last_peered":"2024-09-10T16:01:25.911839+0000","last_clean":"2024-09-10T16:01:25.911839+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:25.911839+0000","last_undegraded":"2024-09-10T16:01:25.911839+0000","last_fullsized":"2024-09-10T16:01:25.911839+0000","mapping_epoch":36,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8654848,"num_objects":2113,"num_object_clones":0,"num_object_copies":4226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17297,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18467","reported_seq":18946,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:25.917490+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:25.917490+0000","last_peered":"2024-09-10T16:01:25.917490+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:25.917490+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8855576,"num_objects":2163,"num_object_clones":0,"num_object_copies":4326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16735,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1466}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18554","reported_seq":18644,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:25.909920+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:25.909920+0000","last_peered":"2024-09-10T16:01:25.909920+0000","last_clean":"2024-09-10T16:01:25.909920+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:25.909920+0000","last_undegraded":"2024-09-10T16:01:25.909920+0000","last_fullsized":"2024-09-10T16:01:25.909920+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8716288,"num_objects":2128,"num_object_clones":0,"num_object_copies":4256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18554,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18228","reported_seq":18365,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.100216+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:26.100216+0000","last_peered":"2024-09-10T16:01:26.100216+0000","last_clean":"2024-09-10T16:01:26.100216+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:26.100216+0000","last_undegraded":"2024-09-10T16:01:26.100216+0000","last_fullsized":"2024-09-10T16:01:26.100216+0000","mapping_epoch":35,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8613888,"num_objects":2103,"num_object_clones":0,"num_object_copies":4206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15299,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18790","reported_seq":26688,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.735771+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:26.735771+0000","last_peered":"2024-09-10T16:01:26.735771+0000","last_clean":"2024-09-10T16:01:26.735771+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:26.735771+0000","last_undegraded":"2024-09-10T16:01:26.735771+0000","last_fullsized":"2024-09-10T16:01:26.735771+0000","mapping_epoch":47,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8507392,"num_objects":2077,"num_object_clones":0,"num_object_copies":4154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15508,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18235","reported_seq":19006,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.087628+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:26.087628+0000","last_peered":"2024-09-10T16:01:26.087628+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:26.087628+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8400896,"num_objects":2051,"num_object_clones":0,"num_object_copies":4102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14885,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1473}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18778","reported_seq":18919,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.144593+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:26.144593+0000","last_peered":"2024-09-10T16:01:26.144593+0000","last_clean":"2024-09-10T16:01:26.144593+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:26.144593+0000","last_undegraded":"2024-09-10T16:01:26.144593+0000","last_fullsized":"2024-09-10T16:01:26.144593+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8749056,"num_objects":2136,"num_object_clones":0,"num_object_copies":4272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15537,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18664","reported_seq":26801,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:26.656075+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:26.656075+0000","last_peered":"2024-09-10T16:01:26.656075+0000","last_clean":"2024-09-10T16:01:26.656075+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:26.656075+0000","last_undegraded":"2024-09-10T16:01:26.656075+0000","last_fullsized":"2024-09-10T16:01:26.656075+0000","mapping_epoch":45,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8417280,"num_objects":2055,"num_object_clones":0,"num_object_copies":4110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16641,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18462","reported_seq":19287,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.049040+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:26.049040+0000","last_peered":"2024-09-10T16:01:26.049040+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:26.049040+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8667136,"num_objects":2116,"num_object_clones":0,"num_object_copies":4232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15326,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1403}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18618","reported_seq":26424,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:25.814619+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:25.814619+0000","last_peered":"2024-09-10T16:01:25.814619+0000","last_clean":"2024-09-10T16:01:25.814619+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:25.814619+0000","last_undegraded":"2024-09-10T16:01:25.814619+0000","last_fullsized":"2024-09-10T16:01:25.814619+0000","mapping_epoch":49,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8876032,"num_objects":2167,"num_object_clones":0,"num_object_copies":4334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16592,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18349","reported_seq":18772,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:25.899208+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:25.899208+0000","last_peered":"2024-09-10T16:01:25.899208+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:25.899208+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8630272,"num_objects":2107,"num_object_clones":0,"num_object_copies":4214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16218,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1579}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18723","reported_seq":19571,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.165534+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:26.165534+0000","last_peered":"2024-09-10T16:01:26.165534+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:26.165534+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8728576,"num_objects":2131,"num_object_clones":0,"num_object_copies":4262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15915,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18633","reported_seq":19389,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.166470+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:26.166470+0000","last_peered":"2024-09-10T16:01:26.166470+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:26.166470+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8540160,"num_objects":2085,"num_object_clones":0,"num_object_copies":4170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15051,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1541}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18606","reported_seq":19438,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.098228+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:26.098228+0000","last_peered":"2024-09-10T16:01:26.098228+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:26.098228+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8847360,"num_objects":2160,"num_object_clones":0,"num_object_copies":4320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15212,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'18994","reported_seq":19761,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:26.135885+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:26.135885+0000","last_peered":"2024-09-10T16:01:26.135885+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:26.135885+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8618467,"num_objects":2105,"num_object_clones":0,"num_object_copies":4210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15813,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1589}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":138396155,"num_objects":33790,"num_object_clones":0,"num_object_copies":67580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33790,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259260,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15137,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":467152896,"data_stored":467076105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626534,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544448,"kb_used_data":101868,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93827392,"statfs":{"total":96636764160,"available":96079249408,"internally_reserved":0,"allocated":104312832,"data_stored":103963959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453177006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149816,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615580,"kb_used_data":164684,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93756260,"statfs":{"total":96636764160,"available":96006410240,"internally_reserved":0,"allocated":168636416,"data_stored":168349423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":461696685},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953587,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540148,"kb_used_data":193372,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93831692,"statfs":{"total":96636764160,"available":96083652608,"internally_reserved":0,"allocated":198012928,"data_stored":197721863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166801891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102989824,"data_stored":102924331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":197357568,"data_stored":197349883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:31.696 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:32.632 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:32.634 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:32.659 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-10T16:01:32.617122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128348686,"num_objects":31339,"num_object_clones":0,"num_object_copies":62678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31339,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261715,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15214,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160695,"ondisk_log_size":160695,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1680608,"kb_used_data":440360,"kb_used_omap":12,"kb_used_meta":1240179,"kb_avail":281434912,"statfs":{"total":289910292480,"available":288189349888,"internally_reserved":0,"allocated":450928640,"data_stored":450001709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1269943301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,6,8,12,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30232576,"num_objects":-7381,"num_object_clones":0,"num_object_copies":-14762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7381,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003999"},"pg_stats":[{"pgid":"2.f","version":"50'18821","reported_seq":21399,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.219526+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:31.219526+0000","last_peered":"2024-09-10T16:01:31.219526+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:31.219526+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7983104,"num_objects":1949,"num_object_clones":0,"num_object_copies":3898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2728,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18821,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-779}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19100","reported_seq":19217,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:30.964431+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:30.964431+0000","last_peered":"2024-09-10T16:01:30.964431+0000","last_clean":"2024-09-10T16:01:30.964431+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:30.964431+0000","last_undegraded":"2024-09-10T16:01:30.964431+0000","last_fullsized":"2024-09-10T16:01:30.964431+0000","mapping_epoch":36,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8085504,"num_objects":1974,"num_object_clones":0,"num_object_copies":3948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17436,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18637","reported_seq":19116,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:30.949660+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:30.949660+0000","last_peered":"2024-09-10T16:01:30.949660+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:30.949660+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8159256,"num_objects":1993,"num_object_clones":0,"num_object_copies":3986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1993,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16905,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1636}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18710","reported_seq":18800,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:30.888184+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:30.888184+0000","last_peered":"2024-09-10T16:01:30.888184+0000","last_clean":"2024-09-10T16:01:30.888184+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:30.888184+0000","last_undegraded":"2024-09-10T16:01:30.888184+0000","last_fullsized":"2024-09-10T16:01:30.888184+0000","mapping_epoch":36,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8077312,"num_objects":1972,"num_object_clones":0,"num_object_copies":3944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18710,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18404","reported_seq":18541,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.209745+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:31.209745+0000","last_peered":"2024-09-10T16:01:31.209745+0000","last_clean":"2024-09-10T16:01:31.209745+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:31.209745+0000","last_undegraded":"2024-09-10T16:01:31.209745+0000","last_fullsized":"2024-09-10T16:01:31.209745+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7892992,"num_objects":1927,"num_object_clones":0,"num_object_copies":3854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15475,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18932","reported_seq":26830,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.619876+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:31.619876+0000","last_peered":"2024-09-10T16:01:31.619876+0000","last_clean":"2024-09-10T16:01:31.619876+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:31.619876+0000","last_undegraded":"2024-09-10T16:01:31.619876+0000","last_fullsized":"2024-09-10T16:01:31.619876+0000","mapping_epoch":47,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7925760,"num_objects":1935,"num_object_clones":0,"num_object_copies":3870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15650,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18363","reported_seq":19133,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.214759+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:31.214759+0000","last_peered":"2024-09-10T16:01:31.214759+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:31.214759+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7876608,"num_objects":1923,"num_object_clones":0,"num_object_copies":3846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15013,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1601}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18947","reported_seq":19088,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.157656+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:31.157656+0000","last_peered":"2024-09-10T16:01:31.157656+0000","last_clean":"2024-09-10T16:01:31.157656+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:31.157656+0000","last_undegraded":"2024-09-10T16:01:31.157656+0000","last_fullsized":"2024-09-10T16:01:31.157656+0000","mapping_epoch":35,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8056832,"num_objects":1967,"num_object_clones":0,"num_object_copies":3934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15706,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18810","reported_seq":26947,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.721394+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:31.721394+0000","last_peered":"2024-09-10T16:01:31.721394+0000","last_clean":"2024-09-10T16:01:31.721394+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:31.721394+0000","last_undegraded":"2024-09-10T16:01:31.721394+0000","last_fullsized":"2024-09-10T16:01:31.721394+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7819264,"num_objects":1909,"num_object_clones":0,"num_object_copies":3818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18615","reported_seq":19440,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.165660+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:31.165660+0000","last_peered":"2024-09-10T16:01:31.165660+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:31.165660+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8040448,"num_objects":1963,"num_object_clones":0,"num_object_copies":3926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15479,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1556}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18781","reported_seq":26591,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:30.925006+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:30.925006+0000","last_peered":"2024-09-10T16:01:30.925006+0000","last_clean":"2024-09-10T16:01:30.925006+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:30.925006+0000","last_undegraded":"2024-09-10T16:01:30.925006+0000","last_fullsized":"2024-09-10T16:01:30.925006+0000","mapping_epoch":49,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8208384,"num_objects":2004,"num_object_clones":0,"num_object_copies":4008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16755,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18503","reported_seq":18926,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:30.925613+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:30.925613+0000","last_peered":"2024-09-10T16:01:30.925613+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:30.925613+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7999488,"num_objects":1953,"num_object_clones":0,"num_object_copies":3906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16372,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1733}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18887","reported_seq":19733,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.204043+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:31.204043+0000","last_peered":"2024-09-10T16:01:31.204043+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:31.204043+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8056832,"num_objects":1967,"num_object_clones":0,"num_object_copies":3934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16079,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1511}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18777","reported_seq":19533,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.202131+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:31.202131+0000","last_peered":"2024-09-10T16:01:31.202131+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:31.202131+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7950336,"num_objects":1941,"num_object_clones":0,"num_object_copies":3882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15195,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1685}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18754","reported_seq":19586,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.234836+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:31.234836+0000","last_peered":"2024-09-10T16:01:31.234836+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:31.234836+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8241152,"num_objects":2012,"num_object_clones":0,"num_object_copies":4024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15360,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19151","reported_seq":19918,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.136029+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:31.136029+0000","last_peered":"2024-09-10T16:01:31.136029+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:31.136029+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7975395,"num_objects":1948,"num_object_clones":0,"num_object_copies":3896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15970,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":128348667,"num_objects":31337,"num_object_clones":0,"num_object_copies":62674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261713,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15212,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446902272,"data_stored":446825481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160692,"ondisk_log_size":160692,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626535,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545492,"kb_used_data":94592,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93826348,"statfs":{"total":96636764160,"available":96078180352,"internally_reserved":0,"allocated":96862208,"data_stored":96513335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461696686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,1,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149817,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612580,"kb_used_data":157524,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93759260,"statfs":{"total":96636764160,"available":96009482240,"internally_reserved":0,"allocated":161304576,"data_stored":161017583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":465956525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953588,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522536,"kb_used_data":188244,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93849304,"statfs":{"total":96636764160,"available":96101687296,"internally_reserved":0,"allocated":192761856,"data_stored":192470791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,3,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":159367168,"data_stored":159363555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95416320,"data_stored":95350827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192118784,"data_stored":192111099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:32.661 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:33.216 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:33.217 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:33.235 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-10T16:01:32.617122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128348686,"num_objects":31339,"num_object_clones":0,"num_object_copies":62678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31339,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261715,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15214,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160695,"ondisk_log_size":160695,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1680608,"kb_used_data":440360,"kb_used_omap":12,"kb_used_meta":1240179,"kb_avail":281434912,"statfs":{"total":289910292480,"available":288189349888,"internally_reserved":0,"allocated":450928640,"data_stored":450001709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1269943301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,6,8,12,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30232576,"num_objects":-7381,"num_object_clones":0,"num_object_copies":-14762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7381,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003999"},"pg_stats":[{"pgid":"2.f","version":"50'18821","reported_seq":21399,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.219526+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:31.219526+0000","last_peered":"2024-09-10T16:01:31.219526+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:31.219526+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7983104,"num_objects":1949,"num_object_clones":0,"num_object_copies":3898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2728,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18821,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-779}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19100","reported_seq":19217,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:30.964431+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:30.964431+0000","last_peered":"2024-09-10T16:01:30.964431+0000","last_clean":"2024-09-10T16:01:30.964431+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:30.964431+0000","last_undegraded":"2024-09-10T16:01:30.964431+0000","last_fullsized":"2024-09-10T16:01:30.964431+0000","mapping_epoch":36,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8085504,"num_objects":1974,"num_object_clones":0,"num_object_copies":3948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17436,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18637","reported_seq":19116,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:30.949660+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:30.949660+0000","last_peered":"2024-09-10T16:01:30.949660+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:30.949660+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8159256,"num_objects":1993,"num_object_clones":0,"num_object_copies":3986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1993,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16905,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1636}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18710","reported_seq":18800,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:30.888184+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:30.888184+0000","last_peered":"2024-09-10T16:01:30.888184+0000","last_clean":"2024-09-10T16:01:30.888184+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:30.888184+0000","last_undegraded":"2024-09-10T16:01:30.888184+0000","last_fullsized":"2024-09-10T16:01:30.888184+0000","mapping_epoch":36,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8077312,"num_objects":1972,"num_object_clones":0,"num_object_copies":3944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18710,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18404","reported_seq":18541,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.209745+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:31.209745+0000","last_peered":"2024-09-10T16:01:31.209745+0000","last_clean":"2024-09-10T16:01:31.209745+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:31.209745+0000","last_undegraded":"2024-09-10T16:01:31.209745+0000","last_fullsized":"2024-09-10T16:01:31.209745+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7892992,"num_objects":1927,"num_object_clones":0,"num_object_copies":3854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15475,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18932","reported_seq":26830,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.619876+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:31.619876+0000","last_peered":"2024-09-10T16:01:31.619876+0000","last_clean":"2024-09-10T16:01:31.619876+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:31.619876+0000","last_undegraded":"2024-09-10T16:01:31.619876+0000","last_fullsized":"2024-09-10T16:01:31.619876+0000","mapping_epoch":47,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7925760,"num_objects":1935,"num_object_clones":0,"num_object_copies":3870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15650,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18363","reported_seq":19133,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.214759+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:31.214759+0000","last_peered":"2024-09-10T16:01:31.214759+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:31.214759+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7876608,"num_objects":1923,"num_object_clones":0,"num_object_copies":3846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15013,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1601}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18947","reported_seq":19088,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.157656+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:31.157656+0000","last_peered":"2024-09-10T16:01:31.157656+0000","last_clean":"2024-09-10T16:01:31.157656+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:31.157656+0000","last_undegraded":"2024-09-10T16:01:31.157656+0000","last_fullsized":"2024-09-10T16:01:31.157656+0000","mapping_epoch":35,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8056832,"num_objects":1967,"num_object_clones":0,"num_object_copies":3934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15706,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18810","reported_seq":26947,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.721394+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:31.721394+0000","last_peered":"2024-09-10T16:01:31.721394+0000","last_clean":"2024-09-10T16:01:31.721394+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:31.721394+0000","last_undegraded":"2024-09-10T16:01:31.721394+0000","last_fullsized":"2024-09-10T16:01:31.721394+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7819264,"num_objects":1909,"num_object_clones":0,"num_object_copies":3818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18615","reported_seq":19440,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.165660+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:31.165660+0000","last_peered":"2024-09-10T16:01:31.165660+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:31.165660+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8040448,"num_objects":1963,"num_object_clones":0,"num_object_copies":3926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15479,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1556}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18781","reported_seq":26591,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:30.925006+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:30.925006+0000","last_peered":"2024-09-10T16:01:30.925006+0000","last_clean":"2024-09-10T16:01:30.925006+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:30.925006+0000","last_undegraded":"2024-09-10T16:01:30.925006+0000","last_fullsized":"2024-09-10T16:01:30.925006+0000","mapping_epoch":49,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8208384,"num_objects":2004,"num_object_clones":0,"num_object_copies":4008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16755,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18503","reported_seq":18926,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:30.925613+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:30.925613+0000","last_peered":"2024-09-10T16:01:30.925613+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:30.925613+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7999488,"num_objects":1953,"num_object_clones":0,"num_object_copies":3906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16372,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1733}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18887","reported_seq":19733,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.204043+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:31.204043+0000","last_peered":"2024-09-10T16:01:31.204043+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:31.204043+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8056832,"num_objects":1967,"num_object_clones":0,"num_object_copies":3934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16079,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1511}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18777","reported_seq":19533,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.202131+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:31.202131+0000","last_peered":"2024-09-10T16:01:31.202131+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:31.202131+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7950336,"num_objects":1941,"num_object_clones":0,"num_object_copies":3882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15195,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1685}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18754","reported_seq":19586,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.234836+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:31.234836+0000","last_peered":"2024-09-10T16:01:31.234836+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:31.234836+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8241152,"num_objects":2012,"num_object_clones":0,"num_object_copies":4024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15360,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19151","reported_seq":19918,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.136029+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:31.136029+0000","last_peered":"2024-09-10T16:01:31.136029+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:31.136029+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7975395,"num_objects":1948,"num_object_clones":0,"num_object_copies":3896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15970,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":128348667,"num_objects":31337,"num_object_clones":0,"num_object_copies":62674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261713,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15212,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446902272,"data_stored":446825481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160692,"ondisk_log_size":160692,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626535,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545492,"kb_used_data":94592,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93826348,"statfs":{"total":96636764160,"available":96078180352,"internally_reserved":0,"allocated":96862208,"data_stored":96513335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461696686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,1,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149817,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612580,"kb_used_data":157524,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93759260,"statfs":{"total":96636764160,"available":96009482240,"internally_reserved":0,"allocated":161304576,"data_stored":161017583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":465956525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953588,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522536,"kb_used_data":188244,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93849304,"statfs":{"total":96636764160,"available":96101687296,"internally_reserved":0,"allocated":192761856,"data_stored":192470791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,3,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":159367168,"data_stored":159363555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95416320,"data_stored":95350827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192118784,"data_stored":192111099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:33.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:33.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'18821", 'reported_seq': 21399, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:31.219526+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:01:31.219526+0000', 'last_peered': '2024-09-10T16:01:31.219526+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:01:31.219526+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'8800", 'ondisk_log_start': "46'8800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7983104, 'num_objects': 1949, 'num_object_clones': 0, 'num_object_copies': 3898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2728, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1949, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18821, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1245, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -779}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:33.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:33.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'18637", 'reported_seq': 19116, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:30.949660+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:01:30.949660+0000', 'last_peered': '2024-09-10T16:01:30.949660+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:01:30.949660+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'8600", 'ondisk_log_start': "46'8600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8159256, 'num_objects': 1993, 'num_object_clones': 0, 'num_object_copies': 3986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1993, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16905, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1636}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:33.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:33.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'18363", 'reported_seq': 19133, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:31.214759+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:01:31.214759+0000', 'last_peered': '2024-09-10T16:01:31.214759+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:01:31.214759+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7876608, 'num_objects': 1923, 'num_object_clones': 0, 'num_object_copies': 3846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1923, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15013, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1601}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:33.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:33.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'18615", 'reported_seq': 19440, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:31.165660+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:01:31.165660+0000', 'last_peered': '2024-09-10T16:01:31.165660+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:01:31.165660+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'8600", 'ondisk_log_start': "46'8600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8040448, 'num_objects': 1963, 'num_object_clones': 0, 'num_object_copies': 3926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1963, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15479, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1556}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:33.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:33.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'18503", 'reported_seq': 18926, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:30.925613+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:01:30.925613+0000', 'last_peered': '2024-09-10T16:01:30.925613+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:01:30.925613+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7999488, 'num_objects': 1953, 'num_object_clones': 0, 'num_object_copies': 3906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1953, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16372, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1733}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:33.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:33.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'18887", 'reported_seq': 19733, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:31.204043+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:01:31.204043+0000', 'last_peered': '2024-09-10T16:01:31.204043+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:01:31.204043+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'8800", 'ondisk_log_start': "46'8800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8056832, 'num_objects': 1967, 'num_object_clones': 0, 'num_object_copies': 3934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1967, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16079, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1511}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:33.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:33.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'18777", 'reported_seq': 19533, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:31.202131+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:01:31.202131+0000', 'last_peered': '2024-09-10T16:01:31.202131+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:01:31.202131+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7950336, 'num_objects': 1941, 'num_object_clones': 0, 'num_object_copies': 3882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1941, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15195, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1685}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:33.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:33.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'18754", 'reported_seq': 19586, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:31.234836+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:01:31.234836+0000', 'last_peered': '2024-09-10T16:01:31.234836+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:01:31.234836+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8241152, 'num_objects': 2012, 'num_object_clones': 0, 'num_object_copies': 4024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2012, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15360, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:33.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:33.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'19151", 'reported_seq': 19918, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:31.136029+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:01:31.136029+0000', 'last_peered': '2024-09-10T16:01:31.136029+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:01:31.136029+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'9100", 'ondisk_log_start': "46'9100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7975395, 'num_objects': 1948, 'num_object_clones': 0, 'num_object_copies': 3896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1948, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15970, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1746}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:33.238 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:33.926 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:33.926 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31436/62678 objects degraded (50.155%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":619,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":31339,"data_bytes":128348686,"bytes_used":1720942592,"bytes_avail":288189349888,"bytes_total":289910292480,"degraded_objects":31436,"degraded_total":62678,"degraded_ratio":0.50154759245668334,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":614},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:33.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:33.954 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:34.530 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:34.531 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:34.558 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-10T16:01:32.617122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128348686,"num_objects":31339,"num_object_clones":0,"num_object_copies":62678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31339,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261715,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15214,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160695,"ondisk_log_size":160695,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1680608,"kb_used_data":440360,"kb_used_omap":12,"kb_used_meta":1240179,"kb_avail":281434912,"statfs":{"total":289910292480,"available":288189349888,"internally_reserved":0,"allocated":450928640,"data_stored":450001709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1269943301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,6,8,12,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30232576,"num_objects":-7381,"num_object_clones":0,"num_object_copies":-14762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7381,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003999"},"pg_stats":[{"pgid":"2.f","version":"50'18821","reported_seq":21399,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.219526+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:31.219526+0000","last_peered":"2024-09-10T16:01:31.219526+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:31.219526+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7983104,"num_objects":1949,"num_object_clones":0,"num_object_copies":3898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2728,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18821,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-779}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19100","reported_seq":19217,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:30.964431+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:30.964431+0000","last_peered":"2024-09-10T16:01:30.964431+0000","last_clean":"2024-09-10T16:01:30.964431+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:30.964431+0000","last_undegraded":"2024-09-10T16:01:30.964431+0000","last_fullsized":"2024-09-10T16:01:30.964431+0000","mapping_epoch":36,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8085504,"num_objects":1974,"num_object_clones":0,"num_object_copies":3948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17436,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18637","reported_seq":19116,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:30.949660+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:30.949660+0000","last_peered":"2024-09-10T16:01:30.949660+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:30.949660+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8159256,"num_objects":1993,"num_object_clones":0,"num_object_copies":3986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1993,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16905,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1636}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18710","reported_seq":18800,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:30.888184+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:30.888184+0000","last_peered":"2024-09-10T16:01:30.888184+0000","last_clean":"2024-09-10T16:01:30.888184+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:30.888184+0000","last_undegraded":"2024-09-10T16:01:30.888184+0000","last_fullsized":"2024-09-10T16:01:30.888184+0000","mapping_epoch":36,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8077312,"num_objects":1972,"num_object_clones":0,"num_object_copies":3944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18710,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18404","reported_seq":18541,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.209745+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:31.209745+0000","last_peered":"2024-09-10T16:01:31.209745+0000","last_clean":"2024-09-10T16:01:31.209745+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:31.209745+0000","last_undegraded":"2024-09-10T16:01:31.209745+0000","last_fullsized":"2024-09-10T16:01:31.209745+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7892992,"num_objects":1927,"num_object_clones":0,"num_object_copies":3854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15475,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18932","reported_seq":26830,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.619876+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:31.619876+0000","last_peered":"2024-09-10T16:01:31.619876+0000","last_clean":"2024-09-10T16:01:31.619876+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:31.619876+0000","last_undegraded":"2024-09-10T16:01:31.619876+0000","last_fullsized":"2024-09-10T16:01:31.619876+0000","mapping_epoch":47,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7925760,"num_objects":1935,"num_object_clones":0,"num_object_copies":3870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15650,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18363","reported_seq":19133,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.214759+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:31.214759+0000","last_peered":"2024-09-10T16:01:31.214759+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:31.214759+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7876608,"num_objects":1923,"num_object_clones":0,"num_object_copies":3846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15013,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1601}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'18947","reported_seq":19088,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.157656+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:31.157656+0000","last_peered":"2024-09-10T16:01:31.157656+0000","last_clean":"2024-09-10T16:01:31.157656+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:31.157656+0000","last_undegraded":"2024-09-10T16:01:31.157656+0000","last_fullsized":"2024-09-10T16:01:31.157656+0000","mapping_epoch":35,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8056832,"num_objects":1967,"num_object_clones":0,"num_object_copies":3934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15706,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18810","reported_seq":26947,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.721394+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:31.721394+0000","last_peered":"2024-09-10T16:01:31.721394+0000","last_clean":"2024-09-10T16:01:31.721394+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:31.721394+0000","last_undegraded":"2024-09-10T16:01:31.721394+0000","last_fullsized":"2024-09-10T16:01:31.721394+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7819264,"num_objects":1909,"num_object_clones":0,"num_object_copies":3818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18615","reported_seq":19440,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.165660+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:31.165660+0000","last_peered":"2024-09-10T16:01:31.165660+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:31.165660+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8040448,"num_objects":1963,"num_object_clones":0,"num_object_copies":3926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15479,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1556}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18781","reported_seq":26591,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:30.925006+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:30.925006+0000","last_peered":"2024-09-10T16:01:30.925006+0000","last_clean":"2024-09-10T16:01:30.925006+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:30.925006+0000","last_undegraded":"2024-09-10T16:01:30.925006+0000","last_fullsized":"2024-09-10T16:01:30.925006+0000","mapping_epoch":49,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8208384,"num_objects":2004,"num_object_clones":0,"num_object_copies":4008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16755,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18503","reported_seq":18926,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:30.925613+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:30.925613+0000","last_peered":"2024-09-10T16:01:30.925613+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:30.925613+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7999488,"num_objects":1953,"num_object_clones":0,"num_object_copies":3906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16372,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1733}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'18887","reported_seq":19733,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.204043+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:31.204043+0000","last_peered":"2024-09-10T16:01:31.204043+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:31.204043+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8056832,"num_objects":1967,"num_object_clones":0,"num_object_copies":3934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16079,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1511}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18777","reported_seq":19533,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.202131+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:31.202131+0000","last_peered":"2024-09-10T16:01:31.202131+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:31.202131+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7950336,"num_objects":1941,"num_object_clones":0,"num_object_copies":3882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15195,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1685}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18754","reported_seq":19586,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.234836+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:31.234836+0000","last_peered":"2024-09-10T16:01:31.234836+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:31.234836+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8241152,"num_objects":2012,"num_object_clones":0,"num_object_copies":4024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15360,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19151","reported_seq":19918,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:31.136029+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:31.136029+0000","last_peered":"2024-09-10T16:01:31.136029+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:31.136029+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7975395,"num_objects":1948,"num_object_clones":0,"num_object_copies":3896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15970,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":128348667,"num_objects":31337,"num_object_clones":0,"num_object_copies":62674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261713,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15212,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446902272,"data_stored":446825481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160692,"ondisk_log_size":160692,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626535,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545492,"kb_used_data":94592,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93826348,"statfs":{"total":96636764160,"available":96078180352,"internally_reserved":0,"allocated":96862208,"data_stored":96513335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461696686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,1,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149817,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612580,"kb_used_data":157524,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93759260,"statfs":{"total":96636764160,"available":96009482240,"internally_reserved":0,"allocated":161304576,"data_stored":161017583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":465956525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953588,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522536,"kb_used_data":188244,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93849304,"statfs":{"total":96636764160,"available":96101687296,"internally_reserved":0,"allocated":192761856,"data_stored":192470791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,3,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":159367168,"data_stored":159363555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95416320,"data_stored":95350827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192118784,"data_stored":192111099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:37.560 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:38.098 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:38.099 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:38.124 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-09-10T16:01:36.621139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":119730702,"num_objects":29235,"num_object_clones":0,"num_object_copies":58470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29235,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263819,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15296,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1663928,"kb_used_data":427840,"kb_used_omap":12,"kb_used_meta":1236019,"kb_avail":281451592,"statfs":{"total":289910292480,"available":288206430208,"internally_reserved":0,"allocated":438108160,"data_stored":437181229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1265683461},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7,3,10,13,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28401664,"num_objects":-6934,"num_object_clones":0,"num_object_copies":-13868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6934,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006712"},"pg_stats":[{"pgid":"2.f","version":"50'18979","reported_seq":21721,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.287396+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:36.287396+0000","last_peered":"2024-09-10T16:01:36.287396+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:36.287396+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7335936,"num_objects":1791,"num_object_clones":0,"num_object_copies":3582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2646,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18979,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-855}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19257","reported_seq":19374,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:35.959642+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:35.959642+0000","last_peered":"2024-09-10T16:01:35.959642+0000","last_clean":"2024-09-10T16:01:35.959642+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:35.959642+0000","last_undegraded":"2024-09-10T16:01:35.959642+0000","last_fullsized":"2024-09-10T16:01:35.959642+0000","mapping_epoch":36,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7442432,"num_objects":1817,"num_object_clones":0,"num_object_copies":3634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17593,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18798","reported_seq":19279,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:35.948668+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:35.948668+0000","last_peered":"2024-09-10T16:01:35.948668+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:35.948668+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7499800,"num_objects":1832,"num_object_clones":0,"num_object_copies":3664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17066,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1797}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18856","reported_seq":18946,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:35.916977+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:35.916977+0000","last_peered":"2024-09-10T16:01:35.916977+0000","last_clean":"2024-09-10T16:01:35.916977+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:35.916977+0000","last_undegraded":"2024-09-10T16:01:35.916977+0000","last_fullsized":"2024-09-10T16:01:35.916977+0000","mapping_epoch":36,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7479296,"num_objects":1826,"num_object_clones":0,"num_object_copies":3652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18856,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18542","reported_seq":18679,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:36.234364+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:36.234364+0000","last_peered":"2024-09-10T16:01:36.234364+0000","last_clean":"2024-09-10T16:01:36.234364+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:36.234364+0000","last_undegraded":"2024-09-10T16:01:36.234364+0000","last_fullsized":"2024-09-10T16:01:36.234364+0000","mapping_epoch":35,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327744,"num_objects":1789,"num_object_clones":0,"num_object_copies":3578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15613,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'18932","reported_seq":26830,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.619876+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:31.619876+0000","last_peered":"2024-09-10T16:01:31.619876+0000","last_clean":"2024-09-10T16:01:31.619876+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:31.619876+0000","last_undegraded":"2024-09-10T16:01:31.619876+0000","last_fullsized":"2024-09-10T16:01:31.619876+0000","mapping_epoch":47,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7925760,"num_objects":1935,"num_object_clones":0,"num_object_copies":3870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15650,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18504","reported_seq":19275,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.151183+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:36.151183+0000","last_peered":"2024-09-10T16:01:36.151183+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:36.151183+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7299072,"num_objects":1782,"num_object_clones":0,"num_object_copies":3564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15154,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1742}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19084","reported_seq":19223,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:36.260355+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:36.260355+0000","last_peered":"2024-09-10T16:01:36.260355+0000","last_clean":"2024-09-10T16:01:36.260355+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:36.260355+0000","last_undegraded":"2024-09-10T16:01:36.260355+0000","last_fullsized":"2024-09-10T16:01:36.260355+0000","mapping_epoch":35,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7495680,"num_objects":1830,"num_object_clones":0,"num_object_copies":3660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15843,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18810","reported_seq":26947,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:31.721394+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:31.721394+0000","last_peered":"2024-09-10T16:01:31.721394+0000","last_clean":"2024-09-10T16:01:31.721394+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:31.721394+0000","last_undegraded":"2024-09-10T16:01:31.721394+0000","last_fullsized":"2024-09-10T16:01:31.721394+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7819264,"num_objects":1909,"num_object_clones":0,"num_object_copies":3818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18779","reported_seq":19603,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.270845+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:36.270845+0000","last_peered":"2024-09-10T16:01:36.270845+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:36.270845+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7368704,"num_objects":1799,"num_object_clones":0,"num_object_copies":3598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15643,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1720}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18974","reported_seq":26784,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:35.938366+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:35.938366+0000","last_peered":"2024-09-10T16:01:35.938366+0000","last_clean":"2024-09-10T16:01:35.938366+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:35.938366+0000","last_undegraded":"2024-09-10T16:01:35.938366+0000","last_fullsized":"2024-09-10T16:01:35.938366+0000","mapping_epoch":49,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7417856,"num_objects":1811,"num_object_clones":0,"num_object_copies":3622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16948,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18661","reported_seq":19084,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:35.950506+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:35.950506+0000","last_peered":"2024-09-10T16:01:35.950506+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:35.950506+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7352320,"num_objects":1795,"num_object_clones":0,"num_object_copies":3590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16530,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1891}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19011","reported_seq":19858,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.287967+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:36.287967+0000","last_peered":"2024-09-10T16:01:36.287967+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:36.287967+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7548928,"num_objects":1843,"num_object_clones":0,"num_object_copies":3686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16203,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18906","reported_seq":19662,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:35.888587+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:35.888587+0000","last_peered":"2024-09-10T16:01:35.888587+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:35.888587+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7421952,"num_objects":1812,"num_object_clones":0,"num_object_copies":3624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15324,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1814}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18900","reported_seq":19732,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.145122+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:36.145122+0000","last_peered":"2024-09-10T16:01:36.145122+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:36.145122+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7643136,"num_objects":1866,"num_object_clones":0,"num_object_copies":3732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15506,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1670}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19303","reported_seq":20070,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.193822+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:36.193822+0000","last_peered":"2024-09-10T16:01:36.193822+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:36.193822+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7352803,"num_objects":1796,"num_object_clones":0,"num_object_copies":3592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16122,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1898}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":119730683,"num_objects":29233,"num_object_clones":0,"num_object_copies":58466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29233,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263817,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15294,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434688000,"data_stored":434611209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626535,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545492,"kb_used_data":94592,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93826348,"statfs":{"total":96636764160,"available":96078180352,"internally_reserved":0,"allocated":96862208,"data_stored":96513335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461696686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,1,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149818,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596868,"kb_used_data":150128,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93774972,"statfs":{"total":96636764160,"available":96025571328,"internally_reserved":0,"allocated":153731072,"data_stored":153444079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":457436845},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953589,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521568,"kb_used_data":183120,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93850272,"statfs":{"total":96636764160,"available":96102678528,"internally_reserved":0,"allocated":187514880,"data_stored":187223815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,5,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":152281088,"data_stored":152277475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95416320,"data_stored":95350827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":186990592,"data_stored":186982907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:38.126 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:38.678 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:38.680 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:38.703 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-10T16:01:38.621732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":118432270,"num_objects":28918,"num_object_clones":0,"num_object_copies":57836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":264136,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15296,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1660764,"kb_used_data":420520,"kb_used_omap":12,"kb_used_meta":1240179,"kb_avail":281454756,"statfs":{"total":289910292480,"available":288209670144,"internally_reserved":0,"allocated":430612480,"data_stored":429685549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1269943301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,3,12,11,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21127168,"num_objects":-5158,"num_object_clones":0,"num_object_copies":-10316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5158,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006756"},"pg_stats":[{"pgid":"2.f","version":"50'18979","reported_seq":21721,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.287396+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:36.287396+0000","last_peered":"2024-09-10T16:01:36.287396+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:36.287396+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7335936,"num_objects":1791,"num_object_clones":0,"num_object_copies":3582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2646,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18979,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-855}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19257","reported_seq":19374,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:35.959642+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:35.959642+0000","last_peered":"2024-09-10T16:01:35.959642+0000","last_clean":"2024-09-10T16:01:35.959642+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:35.959642+0000","last_undegraded":"2024-09-10T16:01:35.959642+0000","last_fullsized":"2024-09-10T16:01:35.959642+0000","mapping_epoch":36,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7442432,"num_objects":1817,"num_object_clones":0,"num_object_copies":3634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17593,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18798","reported_seq":19279,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:35.948668+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:35.948668+0000","last_peered":"2024-09-10T16:01:35.948668+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:35.948668+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7499800,"num_objects":1832,"num_object_clones":0,"num_object_copies":3664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17066,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1797}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18856","reported_seq":18946,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:35.916977+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:35.916977+0000","last_peered":"2024-09-10T16:01:35.916977+0000","last_clean":"2024-09-10T16:01:35.916977+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:35.916977+0000","last_undegraded":"2024-09-10T16:01:35.916977+0000","last_fullsized":"2024-09-10T16:01:35.916977+0000","mapping_epoch":36,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7479296,"num_objects":1826,"num_object_clones":0,"num_object_copies":3652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18856,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18542","reported_seq":18679,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:36.234364+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:36.234364+0000","last_peered":"2024-09-10T16:01:36.234364+0000","last_clean":"2024-09-10T16:01:36.234364+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:36.234364+0000","last_undegraded":"2024-09-10T16:01:36.234364+0000","last_fullsized":"2024-09-10T16:01:36.234364+0000","mapping_epoch":35,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327744,"num_objects":1789,"num_object_clones":0,"num_object_copies":3578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15613,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19092","reported_seq":26990,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:36.811170+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:36.811170+0000","last_peered":"2024-09-10T16:01:36.811170+0000","last_clean":"2024-09-10T16:01:36.811170+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:36.811170+0000","last_undegraded":"2024-09-10T16:01:36.811170+0000","last_fullsized":"2024-09-10T16:01:36.811170+0000","mapping_epoch":47,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7270400,"num_objects":1775,"num_object_clones":0,"num_object_copies":3550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15810,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18504","reported_seq":19275,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.151183+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:36.151183+0000","last_peered":"2024-09-10T16:01:36.151183+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:36.151183+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7299072,"num_objects":1782,"num_object_clones":0,"num_object_copies":3564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15154,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1742}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19084","reported_seq":19223,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:36.260355+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:36.260355+0000","last_peered":"2024-09-10T16:01:36.260355+0000","last_clean":"2024-09-10T16:01:36.260355+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:36.260355+0000","last_undegraded":"2024-09-10T16:01:36.260355+0000","last_fullsized":"2024-09-10T16:01:36.260355+0000","mapping_epoch":35,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7495680,"num_objects":1830,"num_object_clones":0,"num_object_copies":3660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15843,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18967","reported_seq":27104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:36.723007+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:36.723007+0000","last_peered":"2024-09-10T16:01:36.723007+0000","last_clean":"2024-09-10T16:01:36.723007+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:36.723007+0000","last_undegraded":"2024-09-10T16:01:36.723007+0000","last_fullsized":"2024-09-10T16:01:36.723007+0000","mapping_epoch":45,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7176192,"num_objects":1752,"num_object_clones":0,"num_object_copies":3504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16944,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18779","reported_seq":19603,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.270845+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:36.270845+0000","last_peered":"2024-09-10T16:01:36.270845+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:36.270845+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7368704,"num_objects":1799,"num_object_clones":0,"num_object_copies":3598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15643,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1720}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18974","reported_seq":26784,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:35.938366+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:35.938366+0000","last_peered":"2024-09-10T16:01:35.938366+0000","last_clean":"2024-09-10T16:01:35.938366+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:35.938366+0000","last_undegraded":"2024-09-10T16:01:35.938366+0000","last_fullsized":"2024-09-10T16:01:35.938366+0000","mapping_epoch":49,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7417856,"num_objects":1811,"num_object_clones":0,"num_object_copies":3622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16948,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18661","reported_seq":19084,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:35.950506+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:35.950506+0000","last_peered":"2024-09-10T16:01:35.950506+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:35.950506+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7352320,"num_objects":1795,"num_object_clones":0,"num_object_copies":3590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16530,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1891}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19011","reported_seq":19858,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.287967+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:36.287967+0000","last_peered":"2024-09-10T16:01:36.287967+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:36.287967+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7548928,"num_objects":1843,"num_object_clones":0,"num_object_copies":3686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16203,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18906","reported_seq":19662,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:35.888587+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:35.888587+0000","last_peered":"2024-09-10T16:01:35.888587+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:35.888587+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7421952,"num_objects":1812,"num_object_clones":0,"num_object_copies":3624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15324,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1814}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18900","reported_seq":19732,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.145122+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:36.145122+0000","last_peered":"2024-09-10T16:01:36.145122+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:36.145122+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7643136,"num_objects":1866,"num_object_clones":0,"num_object_copies":3732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15506,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1670}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19303","reported_seq":20070,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.193822+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:36.193822+0000","last_peered":"2024-09-10T16:01:36.193822+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:36.193822+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7352803,"num_objects":1796,"num_object_clones":0,"num_object_copies":3592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16122,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1898}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":118432251,"num_objects":28916,"num_object_clones":0,"num_object_copies":57832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":264134,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15294,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":427286528,"data_stored":427209737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626536,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542328,"kb_used_data":87272,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93829512,"statfs":{"total":96636764160,"available":96081420288,"internally_reserved":0,"allocated":89366528,"data_stored":89017655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":465956526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149818,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596868,"kb_used_data":150128,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93774972,"statfs":{"total":96636764160,"available":96025571328,"internally_reserved":0,"allocated":153731072,"data_stored":153444079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":457436845},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953589,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521568,"kb_used_data":183120,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93850272,"statfs":{"total":96636764160,"available":96102678528,"internally_reserved":0,"allocated":187514880,"data_stored":187223815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,5,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":152281088,"data_stored":152277475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88014848,"data_stored":87949355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":186990592,"data_stored":186982907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:38.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:38.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'18979", 'reported_seq': 21721, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:36.287396+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:01:36.287396+0000', 'last_peered': '2024-09-10T16:01:36.287396+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:01:36.287396+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7335936, 'num_objects': 1791, 'num_object_clones': 0, 'num_object_copies': 3582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2646, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1791, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18979, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -855}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:38.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:38.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'18798", 'reported_seq': 19279, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:35.948668+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:01:35.948668+0000', 'last_peered': '2024-09-10T16:01:35.948668+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:01:35.948668+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7499800, 'num_objects': 1832, 'num_object_clones': 0, 'num_object_copies': 3664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1832, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17066, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1797}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:38.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:38.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'18504", 'reported_seq': 19275, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:36.151183+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:01:36.151183+0000', 'last_peered': '2024-09-10T16:01:36.151183+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:01:36.151183+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7299072, 'num_objects': 1782, 'num_object_clones': 0, 'num_object_copies': 3564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1782, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15154, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1742}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:38.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:38.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'18779", 'reported_seq': 19603, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:36.270845+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:01:36.270845+0000', 'last_peered': '2024-09-10T16:01:36.270845+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:01:36.270845+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7368704, 'num_objects': 1799, 'num_object_clones': 0, 'num_object_copies': 3598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1799, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15643, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1720}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:38.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:38.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'18661", 'reported_seq': 19084, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:35.950506+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:01:35.950506+0000', 'last_peered': '2024-09-10T16:01:35.950506+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:01:35.950506+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'8600", 'ondisk_log_start': "46'8600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7352320, 'num_objects': 1795, 'num_object_clones': 0, 'num_object_copies': 3590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1795, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16530, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1891}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:38.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:38.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'19011", 'reported_seq': 19858, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:36.287967+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:01:36.287967+0000', 'last_peered': '2024-09-10T16:01:36.287967+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:01:36.287967+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'9000", 'ondisk_log_start': "46'9000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7548928, 'num_objects': 1843, 'num_object_clones': 0, 'num_object_copies': 3686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16203, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1635}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:38.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:38.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'18906", 'reported_seq': 19662, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:35.888587+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:01:35.888587+0000', 'last_peered': '2024-09-10T16:01:35.888587+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:01:35.888587+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7421952, 'num_objects': 1812, 'num_object_clones': 0, 'num_object_copies': 3624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1812, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15324, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1814}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:38.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:38.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'18900", 'reported_seq': 19732, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:36.145122+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:01:36.145122+0000', 'last_peered': '2024-09-10T16:01:36.145122+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:01:36.145122+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'8800", 'ondisk_log_start': "46'8800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7643136, 'num_objects': 1866, 'num_object_clones': 0, 'num_object_copies': 3732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1866, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15506, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1670}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:38.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:38.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'19303", 'reported_seq': 20070, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:36.193822+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:01:36.193822+0000', 'last_peered': '2024-09-10T16:01:36.193822+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:01:36.193822+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'9300", 'ondisk_log_start': "46'9300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7352803, 'num_objects': 1796, 'num_object_clones': 0, 'num_object_copies': 3592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1796, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16122, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1898}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:38.706 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:39.397 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:39.397 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31354/57836 objects degraded (54.212%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":625,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":28918,"data_bytes":118432270,"bytes_used":1700622336,"bytes_avail":288209670144,"bytes_total":289910292480,"degraded_objects":31354,"degraded_total":57836,"degraded_ratio":0.54211909537312397,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":429},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:39.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:39.426 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:39.975 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:39.975 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:39.999 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-10T16:01:38.621732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":118432270,"num_objects":28918,"num_object_clones":0,"num_object_copies":57836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":264136,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15296,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1660764,"kb_used_data":420520,"kb_used_omap":12,"kb_used_meta":1240179,"kb_avail":281454756,"statfs":{"total":289910292480,"available":288209670144,"internally_reserved":0,"allocated":430612480,"data_stored":429685549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1269943301},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,3,12,11,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21127168,"num_objects":-5158,"num_object_clones":0,"num_object_copies":-10316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5158,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006756"},"pg_stats":[{"pgid":"2.f","version":"50'18979","reported_seq":21721,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.287396+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:36.287396+0000","last_peered":"2024-09-10T16:01:36.287396+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:36.287396+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7335936,"num_objects":1791,"num_object_clones":0,"num_object_copies":3582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2646,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18979,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-855}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19257","reported_seq":19374,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:35.959642+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:35.959642+0000","last_peered":"2024-09-10T16:01:35.959642+0000","last_clean":"2024-09-10T16:01:35.959642+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:35.959642+0000","last_undegraded":"2024-09-10T16:01:35.959642+0000","last_fullsized":"2024-09-10T16:01:35.959642+0000","mapping_epoch":36,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7442432,"num_objects":1817,"num_object_clones":0,"num_object_copies":3634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17593,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18798","reported_seq":19279,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:35.948668+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:35.948668+0000","last_peered":"2024-09-10T16:01:35.948668+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:35.948668+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7499800,"num_objects":1832,"num_object_clones":0,"num_object_copies":3664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17066,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1797}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18856","reported_seq":18946,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:35.916977+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:35.916977+0000","last_peered":"2024-09-10T16:01:35.916977+0000","last_clean":"2024-09-10T16:01:35.916977+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:35.916977+0000","last_undegraded":"2024-09-10T16:01:35.916977+0000","last_fullsized":"2024-09-10T16:01:35.916977+0000","mapping_epoch":36,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7479296,"num_objects":1826,"num_object_clones":0,"num_object_copies":3652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18856,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18542","reported_seq":18679,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:36.234364+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:36.234364+0000","last_peered":"2024-09-10T16:01:36.234364+0000","last_clean":"2024-09-10T16:01:36.234364+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:36.234364+0000","last_undegraded":"2024-09-10T16:01:36.234364+0000","last_fullsized":"2024-09-10T16:01:36.234364+0000","mapping_epoch":35,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327744,"num_objects":1789,"num_object_clones":0,"num_object_copies":3578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15613,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19092","reported_seq":26990,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:36.811170+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:36.811170+0000","last_peered":"2024-09-10T16:01:36.811170+0000","last_clean":"2024-09-10T16:01:36.811170+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:36.811170+0000","last_undegraded":"2024-09-10T16:01:36.811170+0000","last_fullsized":"2024-09-10T16:01:36.811170+0000","mapping_epoch":47,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7270400,"num_objects":1775,"num_object_clones":0,"num_object_copies":3550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15810,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18504","reported_seq":19275,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.151183+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:36.151183+0000","last_peered":"2024-09-10T16:01:36.151183+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:36.151183+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7299072,"num_objects":1782,"num_object_clones":0,"num_object_copies":3564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15154,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1742}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19084","reported_seq":19223,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:36.260355+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:36.260355+0000","last_peered":"2024-09-10T16:01:36.260355+0000","last_clean":"2024-09-10T16:01:36.260355+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:36.260355+0000","last_undegraded":"2024-09-10T16:01:36.260355+0000","last_fullsized":"2024-09-10T16:01:36.260355+0000","mapping_epoch":35,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7495680,"num_objects":1830,"num_object_clones":0,"num_object_copies":3660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15843,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18967","reported_seq":27104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:36.723007+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:36.723007+0000","last_peered":"2024-09-10T16:01:36.723007+0000","last_clean":"2024-09-10T16:01:36.723007+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:36.723007+0000","last_undegraded":"2024-09-10T16:01:36.723007+0000","last_fullsized":"2024-09-10T16:01:36.723007+0000","mapping_epoch":45,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7176192,"num_objects":1752,"num_object_clones":0,"num_object_copies":3504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16944,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18779","reported_seq":19603,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.270845+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:36.270845+0000","last_peered":"2024-09-10T16:01:36.270845+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:36.270845+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7368704,"num_objects":1799,"num_object_clones":0,"num_object_copies":3598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15643,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1720}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'18974","reported_seq":26784,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:35.938366+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:35.938366+0000","last_peered":"2024-09-10T16:01:35.938366+0000","last_clean":"2024-09-10T16:01:35.938366+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:35.938366+0000","last_undegraded":"2024-09-10T16:01:35.938366+0000","last_fullsized":"2024-09-10T16:01:35.938366+0000","mapping_epoch":49,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7417856,"num_objects":1811,"num_object_clones":0,"num_object_copies":3622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16948,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952593+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952593+0000","last_peered":"2024-09-10T16:00:38.952593+0000","last_clean":"2024-09-10T16:00:38.952593+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952593+0000","last_undegraded":"2024-09-10T16:00:38.952593+0000","last_fullsized":"2024-09-10T16:00:38.952593+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18661","reported_seq":19084,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:35.950506+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:35.950506+0000","last_peered":"2024-09-10T16:01:35.950506+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:35.950506+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7352320,"num_objects":1795,"num_object_clones":0,"num_object_copies":3590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16530,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1891}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19011","reported_seq":19858,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.287967+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:36.287967+0000","last_peered":"2024-09-10T16:01:36.287967+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:36.287967+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7548928,"num_objects":1843,"num_object_clones":0,"num_object_copies":3686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16203,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952674+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952674+0000","last_peered":"2024-09-10T16:00:38.952674+0000","last_clean":"2024-09-10T16:00:38.952674+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952674+0000","last_undegraded":"2024-09-10T16:00:38.952674+0000","last_fullsized":"2024-09-10T16:00:38.952674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'18906","reported_seq":19662,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:35.888587+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:35.888587+0000","last_peered":"2024-09-10T16:01:35.888587+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:35.888587+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7421952,"num_objects":1812,"num_object_clones":0,"num_object_copies":3624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15324,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1814}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'18900","reported_seq":19732,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.145122+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:36.145122+0000","last_peered":"2024-09-10T16:01:36.145122+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:36.145122+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7643136,"num_objects":1866,"num_object_clones":0,"num_object_copies":3732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15506,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1670}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19303","reported_seq":20070,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:36.193822+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:36.193822+0000","last_peered":"2024-09-10T16:01:36.193822+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:36.193822+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7352803,"num_objects":1796,"num_object_clones":0,"num_object_copies":3592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16122,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1898}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":118432251,"num_objects":28916,"num_object_clones":0,"num_object_copies":57832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":264134,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15294,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":427286528,"data_stored":427209737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626536,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542328,"kb_used_data":87272,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93829512,"statfs":{"total":96636764160,"available":96081420288,"internally_reserved":0,"allocated":89366528,"data_stored":89017655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":465956526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149818,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596868,"kb_used_data":150128,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93774972,"statfs":{"total":96636764160,"available":96025571328,"internally_reserved":0,"allocated":153731072,"data_stored":153444079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":457436845},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953589,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521568,"kb_used_data":183120,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93850272,"statfs":{"total":96636764160,"available":96102678528,"internally_reserved":0,"allocated":187514880,"data_stored":187223815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,5,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":152281088,"data_stored":152277475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88014848,"data_stored":87949355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":186990592,"data_stored":186982907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:43.001 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:43.565 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:43.566 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:43.579 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-10T16:01:42.623719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":108429838,"num_objects":26476,"num_object_clones":0,"num_object_copies":52952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266578,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15377,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1637496,"kb_used_data":401416,"kb_used_omap":12,"kb_used_meta":1236019,"kb_avail":281478024,"statfs":{"total":289910292480,"available":288233496576,"internally_reserved":0,"allocated":411049984,"data_stored":410123053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1265683461},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,3,12,9,11,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29966336,"num_objects":-7316,"num_object_clones":0,"num_object_copies":-14632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7316,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007382"},"pg_stats":[{"pgid":"2.f","version":"50'19145","reported_seq":22049,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.305195+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:41.305195+0000","last_peered":"2024-09-10T16:01:41.305195+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:41.305195+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6656000,"num_objects":1625,"num_object_clones":0,"num_object_copies":3250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19145,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1408,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-940}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19397","reported_seq":19514,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:40.924195+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:40.924195+0000","last_peered":"2024-09-10T16:01:40.924195+0000","last_clean":"2024-09-10T16:01:40.924195+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:40.924195+0000","last_undegraded":"2024-09-10T16:01:40.924195+0000","last_fullsized":"2024-09-10T16:01:40.924195+0000","mapping_epoch":36,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6868992,"num_objects":1677,"num_object_clones":0,"num_object_copies":3354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17733,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18954","reported_seq":19435,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:40.904784+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:40.904784+0000","last_peered":"2024-09-10T16:01:40.904784+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:40.904784+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6860824,"num_objects":1676,"num_object_clones":0,"num_object_copies":3352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1676,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17222,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1953}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18991","reported_seq":19081,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:40.869234+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:40.869234+0000","last_peered":"2024-09-10T16:01:40.869234+0000","last_clean":"2024-09-10T16:01:40.869234+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:40.869234+0000","last_undegraded":"2024-09-10T16:01:40.869234+0000","last_fullsized":"2024-09-10T16:01:40.869234+0000","mapping_epoch":36,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6926336,"num_objects":1691,"num_object_clones":0,"num_object_copies":3382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18991,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18717","reported_seq":18854,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.311791+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:41.311791+0000","last_peered":"2024-09-10T16:01:41.311791+0000","last_clean":"2024-09-10T16:01:41.311791+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:41.311791+0000","last_undegraded":"2024-09-10T16:01:41.311791+0000","last_fullsized":"2024-09-10T16:01:41.311791+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6610944,"num_objects":1614,"num_object_clones":0,"num_object_copies":3228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15788,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19251","reported_seq":27149,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.875814+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:41.875814+0000","last_peered":"2024-09-10T16:01:41.875814+0000","last_clean":"2024-09-10T16:01:41.875814+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:41.875814+0000","last_undegraded":"2024-09-10T16:01:41.875814+0000","last_fullsized":"2024-09-10T16:01:41.875814+0000","mapping_epoch":47,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":1616,"num_object_clones":0,"num_object_copies":3232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15969,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18652","reported_seq":19423,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.180324+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:41.180324+0000","last_peered":"2024-09-10T16:01:41.180324+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:41.180324+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6692864,"num_objects":1634,"num_object_clones":0,"num_object_copies":3268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15302,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1890}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19236","reported_seq":19377,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.227043+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:41.227043+0000","last_peered":"2024-09-10T16:01:41.227043+0000","last_clean":"2024-09-10T16:01:41.227043+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:41.227043+0000","last_undegraded":"2024-09-10T16:01:41.227043+0000","last_fullsized":"2024-09-10T16:01:41.227043+0000","mapping_epoch":35,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873088,"num_objects":1678,"num_object_clones":0,"num_object_copies":3356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15995,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19132","reported_seq":27269,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.872825+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:41.872825+0000","last_peered":"2024-09-10T16:01:41.872825+0000","last_clean":"2024-09-10T16:01:41.872825+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:41.872825+0000","last_undegraded":"2024-09-10T16:01:41.872825+0000","last_fullsized":"2024-09-10T16:01:41.872825+0000","mapping_epoch":45,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6500352,"num_objects":1587,"num_object_clones":0,"num_object_copies":3174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17109,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18924","reported_seq":19748,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.258167+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:41.258167+0000","last_peered":"2024-09-10T16:01:41.258167+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:41.258167+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6774784,"num_objects":1654,"num_object_clones":0,"num_object_copies":3308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15788,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1865}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19107","reported_seq":26917,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:40.909831+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:40.909831+0000","last_peered":"2024-09-10T16:01:40.909831+0000","last_clean":"2024-09-10T16:01:40.909831+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:40.909831+0000","last_undegraded":"2024-09-10T16:01:40.909831+0000","last_fullsized":"2024-09-10T16:01:40.909831+0000","mapping_epoch":49,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873088,"num_objects":1678,"num_object_clones":0,"num_object_copies":3356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17081,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18808","reported_seq":19231,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:40.940850+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:40.940850+0000","last_peered":"2024-09-10T16:01:40.940850+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:40.940850+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":1648,"num_object_clones":0,"num_object_copies":3296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16677,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2038}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19161","reported_seq":20009,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.299400+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:41.299400+0000","last_peered":"2024-09-10T16:01:41.299400+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:41.299400+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6934528,"num_objects":1693,"num_object_clones":0,"num_object_copies":3386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16353,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19059","reported_seq":19815,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.328036+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:41.328036+0000","last_peered":"2024-09-10T16:01:41.328036+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:41.328036+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6795264,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15477,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1967}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19072","reported_seq":19904,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.295652+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:41.295652+0000","last_peered":"2024-09-10T16:01:41.295652+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:41.295652+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6938624,"num_objects":1694,"num_object_clones":0,"num_object_copies":3388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15678,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19449","reported_seq":20216,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.233585+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:41.233585+0000","last_peered":"2024-09-10T16:01:41.233585+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:41.233585+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6754787,"num_objects":1650,"num_object_clones":0,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16268,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2044}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":108429819,"num_objects":26474,"num_object_clones":0,"num_object_copies":52948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266576,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15375,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":407097344,"data_stored":407020553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626537,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522504,"kb_used_data":79928,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93849336,"statfs":{"total":96636764160,"available":96101720064,"internally_reserved":0,"allocated":81846272,"data_stored":81497399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453177006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,0,5,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149819,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594056,"kb_used_data":143160,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93777784,"statfs":{"total":96636764160,"available":96028450816,"internally_reserved":0,"allocated":146595840,"data_stored":146308847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":461696685},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,9,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953590,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520936,"kb_used_data":178328,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93850904,"statfs":{"total":96636764160,"available":96103325696,"internally_reserved":0,"allocated":182607872,"data_stored":182316807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,2,1,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":144617472,"data_stored":144613859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80338944,"data_stored":80273451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":182140928,"data_stored":182133243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:43.580 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:44.096 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:44.098 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:44.125 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-10T16:01:42.623719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":108429838,"num_objects":26476,"num_object_clones":0,"num_object_copies":52952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266578,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15377,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1637496,"kb_used_data":401416,"kb_used_omap":12,"kb_used_meta":1236019,"kb_avail":281478024,"statfs":{"total":289910292480,"available":288233496576,"internally_reserved":0,"allocated":411049984,"data_stored":410123053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1265683461},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,3,12,9,11,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29966336,"num_objects":-7316,"num_object_clones":0,"num_object_copies":-14632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7316,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007382"},"pg_stats":[{"pgid":"2.f","version":"50'19145","reported_seq":22049,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.305195+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:41.305195+0000","last_peered":"2024-09-10T16:01:41.305195+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:41.305195+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6656000,"num_objects":1625,"num_object_clones":0,"num_object_copies":3250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19145,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1408,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-940}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19397","reported_seq":19514,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:40.924195+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:40.924195+0000","last_peered":"2024-09-10T16:01:40.924195+0000","last_clean":"2024-09-10T16:01:40.924195+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:40.924195+0000","last_undegraded":"2024-09-10T16:01:40.924195+0000","last_fullsized":"2024-09-10T16:01:40.924195+0000","mapping_epoch":36,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6868992,"num_objects":1677,"num_object_clones":0,"num_object_copies":3354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17733,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18954","reported_seq":19435,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:40.904784+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:40.904784+0000","last_peered":"2024-09-10T16:01:40.904784+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:40.904784+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6860824,"num_objects":1676,"num_object_clones":0,"num_object_copies":3352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1676,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17222,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1953}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18991","reported_seq":19081,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:40.869234+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:40.869234+0000","last_peered":"2024-09-10T16:01:40.869234+0000","last_clean":"2024-09-10T16:01:40.869234+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:40.869234+0000","last_undegraded":"2024-09-10T16:01:40.869234+0000","last_fullsized":"2024-09-10T16:01:40.869234+0000","mapping_epoch":36,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6926336,"num_objects":1691,"num_object_clones":0,"num_object_copies":3382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18991,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18717","reported_seq":18854,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.311791+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:41.311791+0000","last_peered":"2024-09-10T16:01:41.311791+0000","last_clean":"2024-09-10T16:01:41.311791+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:41.311791+0000","last_undegraded":"2024-09-10T16:01:41.311791+0000","last_fullsized":"2024-09-10T16:01:41.311791+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6610944,"num_objects":1614,"num_object_clones":0,"num_object_copies":3228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15788,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19251","reported_seq":27149,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.875814+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:41.875814+0000","last_peered":"2024-09-10T16:01:41.875814+0000","last_clean":"2024-09-10T16:01:41.875814+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:41.875814+0000","last_undegraded":"2024-09-10T16:01:41.875814+0000","last_fullsized":"2024-09-10T16:01:41.875814+0000","mapping_epoch":47,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":1616,"num_object_clones":0,"num_object_copies":3232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15969,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18652","reported_seq":19423,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.180324+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:41.180324+0000","last_peered":"2024-09-10T16:01:41.180324+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:41.180324+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6692864,"num_objects":1634,"num_object_clones":0,"num_object_copies":3268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15302,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1890}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19236","reported_seq":19377,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.227043+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:41.227043+0000","last_peered":"2024-09-10T16:01:41.227043+0000","last_clean":"2024-09-10T16:01:41.227043+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:41.227043+0000","last_undegraded":"2024-09-10T16:01:41.227043+0000","last_fullsized":"2024-09-10T16:01:41.227043+0000","mapping_epoch":35,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873088,"num_objects":1678,"num_object_clones":0,"num_object_copies":3356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15995,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19132","reported_seq":27269,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.872825+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:41.872825+0000","last_peered":"2024-09-10T16:01:41.872825+0000","last_clean":"2024-09-10T16:01:41.872825+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:41.872825+0000","last_undegraded":"2024-09-10T16:01:41.872825+0000","last_fullsized":"2024-09-10T16:01:41.872825+0000","mapping_epoch":45,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6500352,"num_objects":1587,"num_object_clones":0,"num_object_copies":3174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17109,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18924","reported_seq":19748,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.258167+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:41.258167+0000","last_peered":"2024-09-10T16:01:41.258167+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:41.258167+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6774784,"num_objects":1654,"num_object_clones":0,"num_object_copies":3308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15788,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1865}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19107","reported_seq":26917,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:40.909831+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:40.909831+0000","last_peered":"2024-09-10T16:01:40.909831+0000","last_clean":"2024-09-10T16:01:40.909831+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:40.909831+0000","last_undegraded":"2024-09-10T16:01:40.909831+0000","last_fullsized":"2024-09-10T16:01:40.909831+0000","mapping_epoch":49,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873088,"num_objects":1678,"num_object_clones":0,"num_object_copies":3356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17081,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18808","reported_seq":19231,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:40.940850+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:40.940850+0000","last_peered":"2024-09-10T16:01:40.940850+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:40.940850+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":1648,"num_object_clones":0,"num_object_copies":3296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16677,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2038}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19161","reported_seq":20009,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.299400+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:41.299400+0000","last_peered":"2024-09-10T16:01:41.299400+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:41.299400+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6934528,"num_objects":1693,"num_object_clones":0,"num_object_copies":3386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16353,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19059","reported_seq":19815,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.328036+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:41.328036+0000","last_peered":"2024-09-10T16:01:41.328036+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:41.328036+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6795264,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15477,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1967}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19072","reported_seq":19904,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.295652+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:41.295652+0000","last_peered":"2024-09-10T16:01:41.295652+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:41.295652+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6938624,"num_objects":1694,"num_object_clones":0,"num_object_copies":3388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15678,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19449","reported_seq":20216,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.233585+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:41.233585+0000","last_peered":"2024-09-10T16:01:41.233585+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:41.233585+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6754787,"num_objects":1650,"num_object_clones":0,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16268,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2044}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":108429819,"num_objects":26474,"num_object_clones":0,"num_object_copies":52948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266576,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15375,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":407097344,"data_stored":407020553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626537,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522504,"kb_used_data":79928,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93849336,"statfs":{"total":96636764160,"available":96101720064,"internally_reserved":0,"allocated":81846272,"data_stored":81497399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453177006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,0,5,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149819,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594056,"kb_used_data":143160,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93777784,"statfs":{"total":96636764160,"available":96028450816,"internally_reserved":0,"allocated":146595840,"data_stored":146308847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":461696685},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,9,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953590,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520936,"kb_used_data":178328,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93850904,"statfs":{"total":96636764160,"available":96103325696,"internally_reserved":0,"allocated":182607872,"data_stored":182316807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,2,1,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":144617472,"data_stored":144613859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80338944,"data_stored":80273451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":182140928,"data_stored":182133243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:44.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:44.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'19145", 'reported_seq': 22049, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:41.305195+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:01:41.305195+0000', 'last_peered': '2024-09-10T16:01:41.305195+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:01:41.305195+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'9100", 'ondisk_log_start': "46'9100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6656000, 'num_objects': 1625, 'num_object_clones': 0, 'num_object_copies': 3250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2565, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1625, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19145, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1408, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -940}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:44.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:44.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'18954", 'reported_seq': 19435, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:40.904784+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:01:40.904784+0000', 'last_peered': '2024-09-10T16:01:40.904784+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:01:40.904784+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6860824, 'num_objects': 1676, 'num_object_clones': 0, 'num_object_copies': 3352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1676, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17222, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1953}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:44.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:44.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'18652", 'reported_seq': 19423, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:41.180324+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:01:41.180324+0000', 'last_peered': '2024-09-10T16:01:41.180324+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:01:41.180324+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'8600", 'ondisk_log_start': "46'8600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6692864, 'num_objects': 1634, 'num_object_clones': 0, 'num_object_copies': 3268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15302, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1890}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:44.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:44.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'18924", 'reported_seq': 19748, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:41.258167+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:01:41.258167+0000', 'last_peered': '2024-09-10T16:01:41.258167+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:01:41.258167+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6774784, 'num_objects': 1654, 'num_object_clones': 0, 'num_object_copies': 3308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1654, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15788, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1865}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:44.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:44.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'18808", 'reported_seq': 19231, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:40.940850+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:01:40.940850+0000', 'last_peered': '2024-09-10T16:01:40.940850+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:01:40.940850+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'8800", 'ondisk_log_start': "46'8800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6750208, 'num_objects': 1648, 'num_object_clones': 0, 'num_object_copies': 3296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1648, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16677, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2038}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:44.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:44.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'19161", 'reported_seq': 20009, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:41.299400+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:01:41.299400+0000', 'last_peered': '2024-09-10T16:01:41.299400+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:01:41.299400+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'9100", 'ondisk_log_start': "46'9100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6934528, 'num_objects': 1693, 'num_object_clones': 0, 'num_object_copies': 3386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1693, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16353, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1785}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:44.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:44.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'19059", 'reported_seq': 19815, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:41.328036+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:01:41.328036+0000', 'last_peered': '2024-09-10T16:01:41.328036+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:01:41.328036+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'9000", 'ondisk_log_start': "46'9000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6795264, 'num_objects': 1659, 'num_object_clones': 0, 'num_object_copies': 3318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1659, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15477, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1967}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:44.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:44.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'19072", 'reported_seq': 19904, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:41.295652+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:01:41.295652+0000', 'last_peered': '2024-09-10T16:01:41.295652+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:01:41.295652+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'9000", 'ondisk_log_start': "46'9000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6938624, 'num_objects': 1694, 'num_object_clones': 0, 'num_object_copies': 3388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1694, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15678, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1842}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:44.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:44.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'19449", 'reported_seq': 20216, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:41.233585+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:01:41.233585+0000', 'last_peered': '2024-09-10T16:01:41.233585+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:01:41.233585+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'9400", 'ondisk_log_start': "46'9400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6754787, 'num_objects': 1650, 'num_object_clones': 0, 'num_object_copies': 3300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16268, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2044}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:44.128 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:44.850 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:44.851 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31273/52952 objects degraded (59.059%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":630,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":26476,"data_bytes":108429838,"bytes_used":1676795904,"bytes_avail":288233496576,"bytes_total":289910292480,"degraded_objects":31273,"degraded_total":52952,"degraded_ratio":0.590591479075389,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":609},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:44.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:44.874 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:45.435 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:45.435 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:45.455 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-10T16:01:44.624089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":108429838,"num_objects":26476,"num_object_clones":0,"num_object_copies":52952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266578,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15377,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1637496,"kb_used_data":401416,"kb_used_omap":12,"kb_used_meta":1236019,"kb_avail":281478024,"statfs":{"total":289910292480,"available":288233496576,"internally_reserved":0,"allocated":411049984,"data_stored":410123053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1265683461},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,3,12,9,11,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19918848,"num_objects":-4863,"num_object_clones":0,"num_object_copies":-9726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4863,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006966"},"pg_stats":[{"pgid":"2.f","version":"50'19145","reported_seq":22049,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.305195+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:41.305195+0000","last_peered":"2024-09-10T16:01:41.305195+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:41.305195+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6656000,"num_objects":1625,"num_object_clones":0,"num_object_copies":3250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19145,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1408,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-940}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19397","reported_seq":19514,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:40.924195+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:40.924195+0000","last_peered":"2024-09-10T16:01:40.924195+0000","last_clean":"2024-09-10T16:01:40.924195+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:40.924195+0000","last_undegraded":"2024-09-10T16:01:40.924195+0000","last_fullsized":"2024-09-10T16:01:40.924195+0000","mapping_epoch":36,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6868992,"num_objects":1677,"num_object_clones":0,"num_object_copies":3354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17733,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'18954","reported_seq":19435,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:40.904784+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:40.904784+0000","last_peered":"2024-09-10T16:01:40.904784+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:40.904784+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6860824,"num_objects":1676,"num_object_clones":0,"num_object_copies":3352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1676,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17222,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1953}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'18991","reported_seq":19081,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:40.869234+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:40.869234+0000","last_peered":"2024-09-10T16:01:40.869234+0000","last_clean":"2024-09-10T16:01:40.869234+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:40.869234+0000","last_undegraded":"2024-09-10T16:01:40.869234+0000","last_fullsized":"2024-09-10T16:01:40.869234+0000","mapping_epoch":36,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6926336,"num_objects":1691,"num_object_clones":0,"num_object_copies":3382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18991,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18717","reported_seq":18854,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.311791+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:41.311791+0000","last_peered":"2024-09-10T16:01:41.311791+0000","last_clean":"2024-09-10T16:01:41.311791+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:41.311791+0000","last_undegraded":"2024-09-10T16:01:41.311791+0000","last_fullsized":"2024-09-10T16:01:41.311791+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6610944,"num_objects":1614,"num_object_clones":0,"num_object_copies":3228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15788,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19251","reported_seq":27149,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.875814+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:41.875814+0000","last_peered":"2024-09-10T16:01:41.875814+0000","last_clean":"2024-09-10T16:01:41.875814+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:41.875814+0000","last_undegraded":"2024-09-10T16:01:41.875814+0000","last_fullsized":"2024-09-10T16:01:41.875814+0000","mapping_epoch":47,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":1616,"num_object_clones":0,"num_object_copies":3232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15969,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18652","reported_seq":19423,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.180324+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:41.180324+0000","last_peered":"2024-09-10T16:01:41.180324+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:41.180324+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6692864,"num_objects":1634,"num_object_clones":0,"num_object_copies":3268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15302,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1890}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19236","reported_seq":19377,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.227043+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:41.227043+0000","last_peered":"2024-09-10T16:01:41.227043+0000","last_clean":"2024-09-10T16:01:41.227043+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:41.227043+0000","last_undegraded":"2024-09-10T16:01:41.227043+0000","last_fullsized":"2024-09-10T16:01:41.227043+0000","mapping_epoch":35,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873088,"num_objects":1678,"num_object_clones":0,"num_object_copies":3356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15995,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19132","reported_seq":27269,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:41.872825+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:41.872825+0000","last_peered":"2024-09-10T16:01:41.872825+0000","last_clean":"2024-09-10T16:01:41.872825+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:41.872825+0000","last_undegraded":"2024-09-10T16:01:41.872825+0000","last_fullsized":"2024-09-10T16:01:41.872825+0000","mapping_epoch":45,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6500352,"num_objects":1587,"num_object_clones":0,"num_object_copies":3174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17109,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'18924","reported_seq":19748,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.258167+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:41.258167+0000","last_peered":"2024-09-10T16:01:41.258167+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:41.258167+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6774784,"num_objects":1654,"num_object_clones":0,"num_object_copies":3308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15788,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1865}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19107","reported_seq":26917,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:40.909831+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:40.909831+0000","last_peered":"2024-09-10T16:01:40.909831+0000","last_clean":"2024-09-10T16:01:40.909831+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:40.909831+0000","last_undegraded":"2024-09-10T16:01:40.909831+0000","last_fullsized":"2024-09-10T16:01:40.909831+0000","mapping_epoch":49,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873088,"num_objects":1678,"num_object_clones":0,"num_object_copies":3356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17081,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18808","reported_seq":19231,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:40.940850+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:40.940850+0000","last_peered":"2024-09-10T16:01:40.940850+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:40.940850+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":1648,"num_object_clones":0,"num_object_copies":3296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16677,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2038}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19161","reported_seq":20009,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.299400+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:41.299400+0000","last_peered":"2024-09-10T16:01:41.299400+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:41.299400+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6934528,"num_objects":1693,"num_object_clones":0,"num_object_copies":3386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16353,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19059","reported_seq":19815,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.328036+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:41.328036+0000","last_peered":"2024-09-10T16:01:41.328036+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:41.328036+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6795264,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15477,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1967}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19072","reported_seq":19904,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.295652+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:41.295652+0000","last_peered":"2024-09-10T16:01:41.295652+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:41.295652+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6938624,"num_objects":1694,"num_object_clones":0,"num_object_copies":3388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15678,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952678+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952678+0000","last_peered":"2024-09-10T16:00:38.952678+0000","last_clean":"2024-09-10T16:00:38.952678+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952678+0000","last_undegraded":"2024-09-10T16:00:38.952678+0000","last_fullsized":"2024-09-10T16:00:38.952678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19449","reported_seq":20216,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:41.233585+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:41.233585+0000","last_peered":"2024-09-10T16:01:41.233585+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:41.233585+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6754787,"num_objects":1650,"num_object_clones":0,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16268,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2044}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":108429819,"num_objects":26474,"num_object_clones":0,"num_object_copies":52948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266576,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15375,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":407097344,"data_stored":407020553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626537,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522504,"kb_used_data":79928,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93849336,"statfs":{"total":96636764160,"available":96101720064,"internally_reserved":0,"allocated":81846272,"data_stored":81497399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453177006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,0,5,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149819,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594056,"kb_used_data":143160,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93777784,"statfs":{"total":96636764160,"available":96028450816,"internally_reserved":0,"allocated":146595840,"data_stored":146308847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":461696685},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,9,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953590,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520936,"kb_used_data":178328,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93850904,"statfs":{"total":96636764160,"available":96103325696,"internally_reserved":0,"allocated":182607872,"data_stored":182316807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,2,1,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":144617472,"data_stored":144613859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80338944,"data_stored":80273451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":182140928,"data_stored":182133243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:48.458 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:49.015 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:49.015 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:49.037 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-10T16:01:48.625537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":98439694,"num_objects":24037,"num_object_clones":0,"num_object_copies":48074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24037,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269017,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15473,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1652740,"kb_used_data":381964,"kb_used_omap":12,"kb_used_meta":1270707,"kb_avail":281462780,"statfs":{"total":289910292480,"available":288217886720,"internally_reserved":0,"allocated":391131136,"data_stored":390204205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1301203973},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,10,4,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21291008,"num_objects":-5198,"num_object_clones":0,"num_object_copies":-10396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5198,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004397"},"pg_stats":[{"pgid":"2.f","version":"50'19273","reported_seq":22369,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.356003+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:46.356003+0000","last_peered":"2024-09-10T16:01:46.356003+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:46.356003+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6131712,"num_objects":1497,"num_object_clones":0,"num_object_copies":2994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19273,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1504,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-972}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19569","reported_seq":19686,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:45.991380+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:45.991380+0000","last_peered":"2024-09-10T16:01:45.991380+0000","last_clean":"2024-09-10T16:01:45.991380+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:45.991380+0000","last_undegraded":"2024-09-10T16:01:45.991380+0000","last_fullsized":"2024-09-10T16:01:45.991380+0000","mapping_epoch":36,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6164480,"num_objects":1505,"num_object_clones":0,"num_object_copies":3010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17905,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19097","reported_seq":19578,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:45.979678+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:45.979678+0000","last_peered":"2024-09-10T16:01:45.979678+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:45.979678+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6275096,"num_objects":1533,"num_object_clones":0,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1533,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17365,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2096}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19147","reported_seq":19237,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:45.931138+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:45.931138+0000","last_peered":"2024-09-10T16:01:45.931138+0000","last_clean":"2024-09-10T16:01:45.931138+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:45.931138+0000","last_undegraded":"2024-09-10T16:01:45.931138+0000","last_fullsized":"2024-09-10T16:01:45.931138+0000","mapping_epoch":36,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6287360,"num_objects":1535,"num_object_clones":0,"num_object_copies":3070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19147,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18869","reported_seq":19005,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:46.311661+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:46.311661+0000","last_peered":"2024-09-10T16:01:46.311661+0000","last_clean":"2024-09-10T16:01:46.311661+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:46.311661+0000","last_undegraded":"2024-09-10T16:01:46.311661+0000","last_fullsized":"2024-09-10T16:01:46.311661+0000","mapping_epoch":35,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5988352,"num_objects":1462,"num_object_clones":0,"num_object_copies":2924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15940,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19401","reported_seq":27299,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:46.910032+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:46.910032+0000","last_peered":"2024-09-10T16:01:46.910032+0000","last_clean":"2024-09-10T16:01:46.910032+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:46.910032+0000","last_undegraded":"2024-09-10T16:01:46.910032+0000","last_fullsized":"2024-09-10T16:01:46.910032+0000","mapping_epoch":47,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6004736,"num_objects":1466,"num_object_clones":0,"num_object_copies":2932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16119,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18795","reported_seq":19566,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.250091+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:46.250091+0000","last_peered":"2024-09-10T16:01:46.250091+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:46.250091+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6107136,"num_objects":1491,"num_object_clones":0,"num_object_copies":2982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15445,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2033}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19401","reported_seq":19542,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:46.303306+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:46.303306+0000","last_peered":"2024-09-10T16:01:46.303306+0000","last_clean":"2024-09-10T16:01:46.303306+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:46.303306+0000","last_undegraded":"2024-09-10T16:01:46.303306+0000","last_fullsized":"2024-09-10T16:01:46.303306+0000","mapping_epoch":35,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6197248,"num_objects":1513,"num_object_clones":0,"num_object_copies":3026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16160,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19286","reported_seq":27424,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:46.896393+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:46.896393+0000","last_peered":"2024-09-10T16:01:46.896393+0000","last_clean":"2024-09-10T16:01:46.896393+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:46.896393+0000","last_undegraded":"2024-09-10T16:01:46.896393+0000","last_fullsized":"2024-09-10T16:01:46.896393+0000","mapping_epoch":45,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5869568,"num_objects":1433,"num_object_clones":0,"num_object_copies":2866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17263,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19055","reported_seq":19879,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.309023+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:46.309023+0000","last_peered":"2024-09-10T16:01:46.309023+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:46.309023+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6238208,"num_objects":1523,"num_object_clones":0,"num_object_copies":3046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15919,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1996}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19259","reported_seq":27069,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:45.834074+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:45.834074+0000","last_peered":"2024-09-10T16:01:45.834074+0000","last_clean":"2024-09-10T16:01:45.834074+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:45.834074+0000","last_undegraded":"2024-09-10T16:01:45.834074+0000","last_fullsized":"2024-09-10T16:01:45.834074+0000","mapping_epoch":49,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6250496,"num_objects":1526,"num_object_clones":0,"num_object_copies":3052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17233,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18971","reported_seq":19394,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:45.968676+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:45.968676+0000","last_peered":"2024-09-10T16:01:45.968676+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:45.968676+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6082560,"num_objects":1485,"num_object_clones":0,"num_object_copies":2970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16840,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19310","reported_seq":20158,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.354433+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:46.354433+0000","last_peered":"2024-09-10T16:01:46.354433+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:46.354433+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6324224,"num_objects":1544,"num_object_clones":0,"num_object_copies":3088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16502,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19203","reported_seq":19959,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.285825+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:46.285825+0000","last_peered":"2024-09-10T16:01:46.285825+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:46.285825+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6205440,"num_objects":1515,"num_object_clones":0,"num_object_copies":3030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15621,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19237","reported_seq":20069,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.249054+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:46.249054+0000","last_peered":"2024-09-10T16:01:46.249054+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:46.249054+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6262784,"num_objects":1529,"num_object_clones":0,"num_object_copies":3058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15843,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2007}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19621","reported_seq":20388,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.332955+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:46.332955+0000","last_peered":"2024-09-10T16:01:46.332955+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:46.332955+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050275,"num_objects":1478,"num_object_clones":0,"num_object_copies":2956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16440,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2216}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":98439675,"num_objects":24035,"num_object_clones":0,"num_object_copies":48070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24035,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269015,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15471,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":387129344,"data_stored":387052553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626538,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523696,"kb_used_data":72800,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93848144,"statfs":{"total":96636764160,"available":96100499456,"internally_reserved":0,"allocated":74547200,"data_stored":74198327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461696686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,2,0,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149820,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608872,"kb_used_data":135760,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93762968,"statfs":{"total":96636764160,"available":96013279232,"internally_reserved":0,"allocated":139018240,"data_stored":138731247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":484437677},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,6,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953591,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520172,"kb_used_data":173404,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93851668,"statfs":{"total":96636764160,"available":96104108032,"internally_reserved":0,"allocated":177565696,"data_stored":177274631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":137158656,"data_stored":137155043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72929280,"data_stored":72863787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177041408,"data_stored":177033723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:49.039 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:49.598 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:49.599 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:49.629 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-10T16:01:48.625537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":98439694,"num_objects":24037,"num_object_clones":0,"num_object_copies":48074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24037,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269017,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15473,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1652740,"kb_used_data":381964,"kb_used_omap":12,"kb_used_meta":1270707,"kb_avail":281462780,"statfs":{"total":289910292480,"available":288217886720,"internally_reserved":0,"allocated":391131136,"data_stored":390204205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1301203973},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,10,4,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21291008,"num_objects":-5198,"num_object_clones":0,"num_object_copies":-10396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5198,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004397"},"pg_stats":[{"pgid":"2.f","version":"50'19273","reported_seq":22369,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.356003+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:46.356003+0000","last_peered":"2024-09-10T16:01:46.356003+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:46.356003+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6131712,"num_objects":1497,"num_object_clones":0,"num_object_copies":2994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19273,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1504,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-972}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19569","reported_seq":19686,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:45.991380+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:45.991380+0000","last_peered":"2024-09-10T16:01:45.991380+0000","last_clean":"2024-09-10T16:01:45.991380+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:45.991380+0000","last_undegraded":"2024-09-10T16:01:45.991380+0000","last_fullsized":"2024-09-10T16:01:45.991380+0000","mapping_epoch":36,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6164480,"num_objects":1505,"num_object_clones":0,"num_object_copies":3010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17905,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19097","reported_seq":19578,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:45.979678+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:45.979678+0000","last_peered":"2024-09-10T16:01:45.979678+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:45.979678+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6275096,"num_objects":1533,"num_object_clones":0,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1533,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17365,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2096}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19147","reported_seq":19237,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:45.931138+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:45.931138+0000","last_peered":"2024-09-10T16:01:45.931138+0000","last_clean":"2024-09-10T16:01:45.931138+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:45.931138+0000","last_undegraded":"2024-09-10T16:01:45.931138+0000","last_fullsized":"2024-09-10T16:01:45.931138+0000","mapping_epoch":36,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6287360,"num_objects":1535,"num_object_clones":0,"num_object_copies":3070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19147,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18869","reported_seq":19005,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:46.311661+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:46.311661+0000","last_peered":"2024-09-10T16:01:46.311661+0000","last_clean":"2024-09-10T16:01:46.311661+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:46.311661+0000","last_undegraded":"2024-09-10T16:01:46.311661+0000","last_fullsized":"2024-09-10T16:01:46.311661+0000","mapping_epoch":35,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5988352,"num_objects":1462,"num_object_clones":0,"num_object_copies":2924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15940,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19401","reported_seq":27299,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:46.910032+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:46.910032+0000","last_peered":"2024-09-10T16:01:46.910032+0000","last_clean":"2024-09-10T16:01:46.910032+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:46.910032+0000","last_undegraded":"2024-09-10T16:01:46.910032+0000","last_fullsized":"2024-09-10T16:01:46.910032+0000","mapping_epoch":47,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6004736,"num_objects":1466,"num_object_clones":0,"num_object_copies":2932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16119,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18795","reported_seq":19566,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.250091+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:46.250091+0000","last_peered":"2024-09-10T16:01:46.250091+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:46.250091+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6107136,"num_objects":1491,"num_object_clones":0,"num_object_copies":2982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15445,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2033}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19401","reported_seq":19542,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:46.303306+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:46.303306+0000","last_peered":"2024-09-10T16:01:46.303306+0000","last_clean":"2024-09-10T16:01:46.303306+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:46.303306+0000","last_undegraded":"2024-09-10T16:01:46.303306+0000","last_fullsized":"2024-09-10T16:01:46.303306+0000","mapping_epoch":35,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6197248,"num_objects":1513,"num_object_clones":0,"num_object_copies":3026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16160,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19286","reported_seq":27424,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:46.896393+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:46.896393+0000","last_peered":"2024-09-10T16:01:46.896393+0000","last_clean":"2024-09-10T16:01:46.896393+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:46.896393+0000","last_undegraded":"2024-09-10T16:01:46.896393+0000","last_fullsized":"2024-09-10T16:01:46.896393+0000","mapping_epoch":45,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5869568,"num_objects":1433,"num_object_clones":0,"num_object_copies":2866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17263,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19055","reported_seq":19879,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.309023+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:46.309023+0000","last_peered":"2024-09-10T16:01:46.309023+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:46.309023+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6238208,"num_objects":1523,"num_object_clones":0,"num_object_copies":3046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15919,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1996}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19259","reported_seq":27069,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:45.834074+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:45.834074+0000","last_peered":"2024-09-10T16:01:45.834074+0000","last_clean":"2024-09-10T16:01:45.834074+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:45.834074+0000","last_undegraded":"2024-09-10T16:01:45.834074+0000","last_fullsized":"2024-09-10T16:01:45.834074+0000","mapping_epoch":49,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6250496,"num_objects":1526,"num_object_clones":0,"num_object_copies":3052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17233,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18971","reported_seq":19394,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:45.968676+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:45.968676+0000","last_peered":"2024-09-10T16:01:45.968676+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:45.968676+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6082560,"num_objects":1485,"num_object_clones":0,"num_object_copies":2970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16840,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19310","reported_seq":20158,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.354433+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:46.354433+0000","last_peered":"2024-09-10T16:01:46.354433+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:46.354433+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6324224,"num_objects":1544,"num_object_clones":0,"num_object_copies":3088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16502,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19203","reported_seq":19959,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.285825+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:46.285825+0000","last_peered":"2024-09-10T16:01:46.285825+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:46.285825+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6205440,"num_objects":1515,"num_object_clones":0,"num_object_copies":3030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15621,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19237","reported_seq":20069,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.249054+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:46.249054+0000","last_peered":"2024-09-10T16:01:46.249054+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:46.249054+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6262784,"num_objects":1529,"num_object_clones":0,"num_object_copies":3058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15843,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2007}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19621","reported_seq":20388,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.332955+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:46.332955+0000","last_peered":"2024-09-10T16:01:46.332955+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:46.332955+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050275,"num_objects":1478,"num_object_clones":0,"num_object_copies":2956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16440,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2216}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":98439675,"num_objects":24035,"num_object_clones":0,"num_object_copies":48070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24035,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269015,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15471,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":387129344,"data_stored":387052553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626538,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523696,"kb_used_data":72800,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93848144,"statfs":{"total":96636764160,"available":96100499456,"internally_reserved":0,"allocated":74547200,"data_stored":74198327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461696686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,2,0,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149820,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608872,"kb_used_data":135760,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93762968,"statfs":{"total":96636764160,"available":96013279232,"internally_reserved":0,"allocated":139018240,"data_stored":138731247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":484437677},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,6,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953591,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520172,"kb_used_data":173404,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93851668,"statfs":{"total":96636764160,"available":96104108032,"internally_reserved":0,"allocated":177565696,"data_stored":177274631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":137158656,"data_stored":137155043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72929280,"data_stored":72863787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177041408,"data_stored":177033723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:49.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:49.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'19273", 'reported_seq': 22369, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:46.356003+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:01:46.356003+0000', 'last_peered': '2024-09-10T16:01:46.356003+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:01:46.356003+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'9200", 'ondisk_log_start': "46'9200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6131712, 'num_objects': 1497, 'num_object_clones': 0, 'num_object_copies': 2994, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2469, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1497, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19273, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1504, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -972}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:49.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:49.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'19097", 'reported_seq': 19578, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:45.979678+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:01:45.979678+0000', 'last_peered': '2024-09-10T16:01:45.979678+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:01:45.979678+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'9000", 'ondisk_log_start': "46'9000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6275096, 'num_objects': 1533, 'num_object_clones': 0, 'num_object_copies': 3066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1533, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17365, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2096}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:49.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:49.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'18795", 'reported_seq': 19566, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:46.250091+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:01:46.250091+0000', 'last_peered': '2024-09-10T16:01:46.250091+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:01:46.250091+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6107136, 'num_objects': 1491, 'num_object_clones': 0, 'num_object_copies': 2982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1491, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15445, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2033}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:49.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:49.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'19055", 'reported_seq': 19879, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:46.309023+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:01:46.309023+0000', 'last_peered': '2024-09-10T16:01:46.309023+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:01:46.309023+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'9000", 'ondisk_log_start': "46'9000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6238208, 'num_objects': 1523, 'num_object_clones': 0, 'num_object_copies': 3046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1523, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15919, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1996}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:49.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:49.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'18971", 'reported_seq': 19394, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:45.968676+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:01:45.968676+0000', 'last_peered': '2024-09-10T16:01:45.968676+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:01:45.968676+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6082560, 'num_objects': 1485, 'num_object_clones': 0, 'num_object_copies': 2970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1485, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16840, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2201}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:49.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:49.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'19310", 'reported_seq': 20158, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:46.354433+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:01:46.354433+0000', 'last_peered': '2024-09-10T16:01:46.354433+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:01:46.354433+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'9300", 'ondisk_log_start': "46'9300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6324224, 'num_objects': 1544, 'num_object_clones': 0, 'num_object_copies': 3088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16502, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1934}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:49.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:49.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'19203", 'reported_seq': 19959, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:46.285825+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:01:46.285825+0000', 'last_peered': '2024-09-10T16:01:46.285825+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:01:46.285825+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'9200", 'ondisk_log_start': "46'9200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6205440, 'num_objects': 1515, 'num_object_clones': 0, 'num_object_copies': 3030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1515, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15621, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2111}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:49.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:49.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'19237", 'reported_seq': 20069, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:46.249054+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:01:46.249054+0000', 'last_peered': '2024-09-10T16:01:46.249054+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:01:46.249054+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'9200", 'ondisk_log_start': "46'9200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6262784, 'num_objects': 1529, 'num_object_clones': 0, 'num_object_copies': 3058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1529, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15843, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2007}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:49.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:49.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'19621", 'reported_seq': 20388, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:46.332955+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:01:46.332955+0000', 'last_peered': '2024-09-10T16:01:46.332955+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:01:46.332955+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'9600", 'ondisk_log_start': "46'9600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6050275, 'num_objects': 1478, 'num_object_clones': 0, 'num_object_copies': 2956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1478, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16440, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2216}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:49.633 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:50.325 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:50.325 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31177/48074 objects degraded (64.852%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":636,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":24037,"data_bytes":98439694,"bytes_used":1692405760,"bytes_avail":288217886720,"bytes_total":289910292480,"degraded_objects":31177,"degraded_total":48074,"degraded_ratio":0.64852103007862882,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":177,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":433},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:50.352 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:50.352 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:50.931 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:50.932 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:50.956 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-10T16:01:50.626301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":98439694,"num_objects":24037,"num_object_clones":0,"num_object_copies":48074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24037,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269017,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15473,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1652740,"kb_used_data":381964,"kb_used_omap":12,"kb_used_meta":1270707,"kb_avail":281462780,"statfs":{"total":289910292480,"available":288217886720,"internally_reserved":0,"allocated":391131136,"data_stored":390204205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1301203973},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,10,4,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19992576,"num_objects":-4881,"num_object_clones":0,"num_object_copies":-9762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4881,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004569"},"pg_stats":[{"pgid":"2.f","version":"50'19273","reported_seq":22369,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.356003+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:46.356003+0000","last_peered":"2024-09-10T16:01:46.356003+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:46.356003+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6131712,"num_objects":1497,"num_object_clones":0,"num_object_copies":2994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2469,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19273,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1504,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-972}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19569","reported_seq":19686,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:45.991380+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:45.991380+0000","last_peered":"2024-09-10T16:01:45.991380+0000","last_clean":"2024-09-10T16:01:45.991380+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:45.991380+0000","last_undegraded":"2024-09-10T16:01:45.991380+0000","last_fullsized":"2024-09-10T16:01:45.991380+0000","mapping_epoch":36,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6164480,"num_objects":1505,"num_object_clones":0,"num_object_copies":3010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17905,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19097","reported_seq":19578,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:45.979678+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:45.979678+0000","last_peered":"2024-09-10T16:01:45.979678+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:45.979678+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6275096,"num_objects":1533,"num_object_clones":0,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1533,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17365,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2096}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19147","reported_seq":19237,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:45.931138+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:45.931138+0000","last_peered":"2024-09-10T16:01:45.931138+0000","last_clean":"2024-09-10T16:01:45.931138+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:45.931138+0000","last_undegraded":"2024-09-10T16:01:45.931138+0000","last_fullsized":"2024-09-10T16:01:45.931138+0000","mapping_epoch":36,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6287360,"num_objects":1535,"num_object_clones":0,"num_object_copies":3070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19147,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'18869","reported_seq":19005,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:46.311661+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:46.311661+0000","last_peered":"2024-09-10T16:01:46.311661+0000","last_clean":"2024-09-10T16:01:46.311661+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:46.311661+0000","last_undegraded":"2024-09-10T16:01:46.311661+0000","last_fullsized":"2024-09-10T16:01:46.311661+0000","mapping_epoch":35,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5988352,"num_objects":1462,"num_object_clones":0,"num_object_copies":2924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15940,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19401","reported_seq":27299,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:46.910032+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:46.910032+0000","last_peered":"2024-09-10T16:01:46.910032+0000","last_clean":"2024-09-10T16:01:46.910032+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:46.910032+0000","last_undegraded":"2024-09-10T16:01:46.910032+0000","last_fullsized":"2024-09-10T16:01:46.910032+0000","mapping_epoch":47,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6004736,"num_objects":1466,"num_object_clones":0,"num_object_copies":2932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16119,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18795","reported_seq":19566,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.250091+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:46.250091+0000","last_peered":"2024-09-10T16:01:46.250091+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:46.250091+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6107136,"num_objects":1491,"num_object_clones":0,"num_object_copies":2982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15445,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2033}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19401","reported_seq":19542,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:46.303306+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:46.303306+0000","last_peered":"2024-09-10T16:01:46.303306+0000","last_clean":"2024-09-10T16:01:46.303306+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:46.303306+0000","last_undegraded":"2024-09-10T16:01:46.303306+0000","last_fullsized":"2024-09-10T16:01:46.303306+0000","mapping_epoch":35,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6197248,"num_objects":1513,"num_object_clones":0,"num_object_copies":3026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16160,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19286","reported_seq":27424,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:46.896393+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:46.896393+0000","last_peered":"2024-09-10T16:01:46.896393+0000","last_clean":"2024-09-10T16:01:46.896393+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:46.896393+0000","last_undegraded":"2024-09-10T16:01:46.896393+0000","last_fullsized":"2024-09-10T16:01:46.896393+0000","mapping_epoch":45,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5869568,"num_objects":1433,"num_object_clones":0,"num_object_copies":2866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17263,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19055","reported_seq":19879,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.309023+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:46.309023+0000","last_peered":"2024-09-10T16:01:46.309023+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:46.309023+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6238208,"num_objects":1523,"num_object_clones":0,"num_object_copies":3046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15919,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1996}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19259","reported_seq":27069,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:45.834074+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:45.834074+0000","last_peered":"2024-09-10T16:01:45.834074+0000","last_clean":"2024-09-10T16:01:45.834074+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:45.834074+0000","last_undegraded":"2024-09-10T16:01:45.834074+0000","last_fullsized":"2024-09-10T16:01:45.834074+0000","mapping_epoch":49,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6250496,"num_objects":1526,"num_object_clones":0,"num_object_copies":3052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17233,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'18971","reported_seq":19394,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:45.968676+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:45.968676+0000","last_peered":"2024-09-10T16:01:45.968676+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:45.968676+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6082560,"num_objects":1485,"num_object_clones":0,"num_object_copies":2970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16840,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19310","reported_seq":20158,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.354433+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:46.354433+0000","last_peered":"2024-09-10T16:01:46.354433+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:46.354433+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6324224,"num_objects":1544,"num_object_clones":0,"num_object_copies":3088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16502,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19203","reported_seq":19959,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.285825+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:46.285825+0000","last_peered":"2024-09-10T16:01:46.285825+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:46.285825+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6205440,"num_objects":1515,"num_object_clones":0,"num_object_copies":3030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15621,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2111}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19237","reported_seq":20069,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.249054+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:46.249054+0000","last_peered":"2024-09-10T16:01:46.249054+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:46.249054+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6262784,"num_objects":1529,"num_object_clones":0,"num_object_copies":3058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15843,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2007}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19621","reported_seq":20388,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:46.332955+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:46.332955+0000","last_peered":"2024-09-10T16:01:46.332955+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:46.332955+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050275,"num_objects":1478,"num_object_clones":0,"num_object_copies":2956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16440,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2216}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":98439675,"num_objects":24035,"num_object_clones":0,"num_object_copies":48070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24035,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269015,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15471,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":387129344,"data_stored":387052553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626538,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523696,"kb_used_data":72800,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93848144,"statfs":{"total":96636764160,"available":96100499456,"internally_reserved":0,"allocated":74547200,"data_stored":74198327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461696686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,2,0,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149820,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608872,"kb_used_data":135760,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93762968,"statfs":{"total":96636764160,"available":96013279232,"internally_reserved":0,"allocated":139018240,"data_stored":138731247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":484437677},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,6,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953591,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520172,"kb_used_data":173404,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93851668,"statfs":{"total":96636764160,"available":96104108032,"internally_reserved":0,"allocated":177565696,"data_stored":177274631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":137158656,"data_stored":137155043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72929280,"data_stored":72863787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177041408,"data_stored":177033723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:53.959 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:54.522 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:54.524 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:54.556 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-10T16:01:52.626887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88310286,"num_objects":21564,"num_object_clones":0,"num_object_copies":43128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":271490,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15550,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1616128,"kb_used_data":361996,"kb_used_omap":12,"kb_used_meta":1254067,"kb_avail":281499392,"statfs":{"total":289910292480,"available":288255377408,"internally_reserved":0,"allocated":370683904,"data_stored":369756973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1284164613},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,11,10,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30121984,"num_objects":-7354,"num_object_clones":0,"num_object_copies":-14708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7354,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003757"},"pg_stats":[{"pgid":"2.f","version":"50'19439","reported_seq":22689,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.366729+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:51.366729+0000","last_peered":"2024-09-10T16:01:51.366729+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:51.366729+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5451776,"num_objects":1331,"num_object_clones":0,"num_object_copies":2662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2392,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19439,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1581,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1061}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19722","reported_seq":19840,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.033730+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:51.033730+0000","last_peered":"2024-09-10T16:01:51.033730+0000","last_clean":"2024-09-10T16:01:51.033730+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:51.033730+0000","last_undegraded":"2024-09-10T16:01:51.033730+0000","last_fullsized":"2024-09-10T16:01:51.033730+0000","mapping_epoch":36,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5537792,"num_objects":1352,"num_object_clones":0,"num_object_copies":2704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18058,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19235","reported_seq":19716,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:50.902907+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:50.902907+0000","last_peered":"2024-09-10T16:01:50.902907+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:50.902907+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5709848,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17503,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2234}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19286","reported_seq":19376,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:50.984962+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:50.984962+0000","last_peered":"2024-09-10T16:01:50.984962+0000","last_clean":"2024-09-10T16:01:50.984962+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:50.984962+0000","last_undegraded":"2024-09-10T16:01:50.984962+0000","last_fullsized":"2024-09-10T16:01:50.984962+0000","mapping_epoch":36,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5718016,"num_objects":1396,"num_object_clones":0,"num_object_copies":2792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19286,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19005","reported_seq":19142,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.237418+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:51.237418+0000","last_peered":"2024-09-10T16:01:51.237418+0000","last_clean":"2024-09-10T16:01:51.237418+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:51.237418+0000","last_undegraded":"2024-09-10T16:01:51.237418+0000","last_fullsized":"2024-09-10T16:01:51.237418+0000","mapping_epoch":35,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5431296,"num_objects":1326,"num_object_clones":0,"num_object_copies":2652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19558","reported_seq":27456,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.946246+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:51.946246+0000","last_peered":"2024-09-10T16:01:51.946246+0000","last_clean":"2024-09-10T16:01:51.946246+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:51.946246+0000","last_undegraded":"2024-09-10T16:01:51.946246+0000","last_fullsized":"2024-09-10T16:01:51.946246+0000","mapping_epoch":47,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5361664,"num_objects":1309,"num_object_clones":0,"num_object_copies":2618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16276,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18941","reported_seq":19712,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.305809+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:51.305809+0000","last_peered":"2024-09-10T16:01:51.305809+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:51.305809+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5509120,"num_objects":1345,"num_object_clones":0,"num_object_copies":2690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15591,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19552","reported_seq":19694,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.379739+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:51.379739+0000","last_peered":"2024-09-10T16:01:51.379739+0000","last_clean":"2024-09-10T16:01:51.379739+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:51.379739+0000","last_undegraded":"2024-09-10T16:01:51.379739+0000","last_fullsized":"2024-09-10T16:01:51.379739+0000","mapping_epoch":35,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5578752,"num_objects":1362,"num_object_clones":0,"num_object_copies":2724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16311,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19439","reported_seq":27577,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.863640+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:51.863640+0000","last_peered":"2024-09-10T16:01:51.863640+0000","last_clean":"2024-09-10T16:01:51.863640+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:51.863640+0000","last_undegraded":"2024-09-10T16:01:51.863640+0000","last_fullsized":"2024-09-10T16:01:51.863640+0000","mapping_epoch":45,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1280,"num_object_clones":0,"num_object_copies":2560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17416,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19224","reported_seq":20049,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.343806+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:51.343806+0000","last_peered":"2024-09-10T16:01:51.343806+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:51.343806+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5545984,"num_objects":1354,"num_object_clones":0,"num_object_copies":2708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16088,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19427","reported_seq":27237,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:50.979238+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:50.979238+0000","last_peered":"2024-09-10T16:01:50.979238+0000","last_clean":"2024-09-10T16:01:50.979238+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:50.979238+0000","last_undegraded":"2024-09-10T16:01:50.979238+0000","last_fullsized":"2024-09-10T16:01:50.979238+0000","mapping_epoch":49,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5562368,"num_objects":1358,"num_object_clones":0,"num_object_copies":2716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17401,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19109","reported_seq":19532,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:50.932664+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:50.932664+0000","last_peered":"2024-09-10T16:01:50.932664+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:50.932664+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5517312,"num_objects":1347,"num_object_clones":0,"num_object_copies":2694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16978,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2339}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19475","reported_seq":20321,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.349751+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:51.349751+0000","last_peered":"2024-09-10T16:01:51.349751+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:51.349751+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5648384,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16667,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19375","reported_seq":20130,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.373301+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:51.373301+0000","last_peered":"2024-09-10T16:01:51.373301+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:51.373301+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5500928,"num_objects":1343,"num_object_clones":0,"num_object_copies":2686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15793,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19409","reported_seq":20241,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.355245+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:51.355245+0000","last_peered":"2024-09-10T16:01:51.355245+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:51.355245+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5558272,"num_objects":1357,"num_object_clones":0,"num_object_copies":2714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16015,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19771","reported_seq":20538,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.264857+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:51.264857+0000","last_peered":"2024-09-10T16:01:51.264857+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:51.264857+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5435875,"num_objects":1328,"num_object_clones":0,"num_object_copies":2656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16590,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":88310267,"num_objects":21562,"num_object_clones":0,"num_object_copies":43124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21562,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":271488,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15548,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":366817280,"data_stored":366740489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626539,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520164,"kb_used_data":65108,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93851676,"statfs":{"total":96636764160,"available":96104116224,"internally_reserved":0,"allocated":66670592,"data_stored":66321719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":465956526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149821,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592980,"kb_used_data":128192,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93778860,"statfs":{"total":96636764160,"available":96029552640,"internally_reserved":0,"allocated":131268608,"data_stored":130981615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953592,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502984,"kb_used_data":168696,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93868856,"statfs":{"total":96636764160,"available":96121708544,"internally_reserved":0,"allocated":172744704,"data_stored":172453639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129523712,"data_stored":129520099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65146880,"data_stored":65081387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172146688,"data_stored":172139003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:54.557 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:55.104 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:55.106 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:55.124 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-10T16:01:54.627523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88310286,"num_objects":21564,"num_object_clones":0,"num_object_copies":43128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":271490,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15550,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1616128,"kb_used_data":361996,"kb_used_omap":12,"kb_used_meta":1254067,"kb_avail":281499392,"statfs":{"total":289910292480,"available":288255377408,"internally_reserved":0,"allocated":370683904,"data_stored":369756973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1284164613},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,11,10,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20119552,"num_objects":-4912,"num_object_clones":0,"num_object_copies":-9824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4912,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003804"},"pg_stats":[{"pgid":"2.f","version":"50'19439","reported_seq":22689,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.366729+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:51.366729+0000","last_peered":"2024-09-10T16:01:51.366729+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:51.366729+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5451776,"num_objects":1331,"num_object_clones":0,"num_object_copies":2662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2392,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19439,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1581,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1061}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19722","reported_seq":19840,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.033730+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:51.033730+0000","last_peered":"2024-09-10T16:01:51.033730+0000","last_clean":"2024-09-10T16:01:51.033730+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:51.033730+0000","last_undegraded":"2024-09-10T16:01:51.033730+0000","last_fullsized":"2024-09-10T16:01:51.033730+0000","mapping_epoch":36,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5537792,"num_objects":1352,"num_object_clones":0,"num_object_copies":2704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18058,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19235","reported_seq":19716,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:50.902907+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:50.902907+0000","last_peered":"2024-09-10T16:01:50.902907+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:50.902907+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5709848,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17503,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2234}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19286","reported_seq":19376,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:50.984962+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:50.984962+0000","last_peered":"2024-09-10T16:01:50.984962+0000","last_clean":"2024-09-10T16:01:50.984962+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:50.984962+0000","last_undegraded":"2024-09-10T16:01:50.984962+0000","last_fullsized":"2024-09-10T16:01:50.984962+0000","mapping_epoch":36,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5718016,"num_objects":1396,"num_object_clones":0,"num_object_copies":2792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19286,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19005","reported_seq":19142,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.237418+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:51.237418+0000","last_peered":"2024-09-10T16:01:51.237418+0000","last_clean":"2024-09-10T16:01:51.237418+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:51.237418+0000","last_undegraded":"2024-09-10T16:01:51.237418+0000","last_fullsized":"2024-09-10T16:01:51.237418+0000","mapping_epoch":35,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5431296,"num_objects":1326,"num_object_clones":0,"num_object_copies":2652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19558","reported_seq":27456,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.946246+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:51.946246+0000","last_peered":"2024-09-10T16:01:51.946246+0000","last_clean":"2024-09-10T16:01:51.946246+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:51.946246+0000","last_undegraded":"2024-09-10T16:01:51.946246+0000","last_fullsized":"2024-09-10T16:01:51.946246+0000","mapping_epoch":47,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5361664,"num_objects":1309,"num_object_clones":0,"num_object_copies":2618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16276,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18941","reported_seq":19712,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.305809+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:51.305809+0000","last_peered":"2024-09-10T16:01:51.305809+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:51.305809+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5509120,"num_objects":1345,"num_object_clones":0,"num_object_copies":2690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15591,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19552","reported_seq":19694,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.379739+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:51.379739+0000","last_peered":"2024-09-10T16:01:51.379739+0000","last_clean":"2024-09-10T16:01:51.379739+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:51.379739+0000","last_undegraded":"2024-09-10T16:01:51.379739+0000","last_fullsized":"2024-09-10T16:01:51.379739+0000","mapping_epoch":35,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5578752,"num_objects":1362,"num_object_clones":0,"num_object_copies":2724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16311,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19439","reported_seq":27577,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.863640+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:51.863640+0000","last_peered":"2024-09-10T16:01:51.863640+0000","last_clean":"2024-09-10T16:01:51.863640+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:51.863640+0000","last_undegraded":"2024-09-10T16:01:51.863640+0000","last_fullsized":"2024-09-10T16:01:51.863640+0000","mapping_epoch":45,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1280,"num_object_clones":0,"num_object_copies":2560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17416,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19224","reported_seq":20049,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.343806+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:51.343806+0000","last_peered":"2024-09-10T16:01:51.343806+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:51.343806+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5545984,"num_objects":1354,"num_object_clones":0,"num_object_copies":2708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16088,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19427","reported_seq":27237,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:50.979238+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:50.979238+0000","last_peered":"2024-09-10T16:01:50.979238+0000","last_clean":"2024-09-10T16:01:50.979238+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:50.979238+0000","last_undegraded":"2024-09-10T16:01:50.979238+0000","last_fullsized":"2024-09-10T16:01:50.979238+0000","mapping_epoch":49,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5562368,"num_objects":1358,"num_object_clones":0,"num_object_copies":2716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17401,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19109","reported_seq":19532,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:50.932664+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:50.932664+0000","last_peered":"2024-09-10T16:01:50.932664+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:50.932664+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5517312,"num_objects":1347,"num_object_clones":0,"num_object_copies":2694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16978,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2339}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19475","reported_seq":20321,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.349751+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:51.349751+0000","last_peered":"2024-09-10T16:01:51.349751+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:51.349751+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5648384,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16667,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19375","reported_seq":20130,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.373301+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:51.373301+0000","last_peered":"2024-09-10T16:01:51.373301+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:51.373301+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5500928,"num_objects":1343,"num_object_clones":0,"num_object_copies":2686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15793,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19409","reported_seq":20241,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.355245+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:51.355245+0000","last_peered":"2024-09-10T16:01:51.355245+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:51.355245+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5558272,"num_objects":1357,"num_object_clones":0,"num_object_copies":2714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16015,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19771","reported_seq":20538,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.264857+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:51.264857+0000","last_peered":"2024-09-10T16:01:51.264857+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:51.264857+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5435875,"num_objects":1328,"num_object_clones":0,"num_object_copies":2656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16590,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":88310267,"num_objects":21562,"num_object_clones":0,"num_object_copies":43124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21562,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":271488,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15548,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":366817280,"data_stored":366740489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626539,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520164,"kb_used_data":65108,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93851676,"statfs":{"total":96636764160,"available":96104116224,"internally_reserved":0,"allocated":66670592,"data_stored":66321719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":465956526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149821,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592980,"kb_used_data":128192,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93778860,"statfs":{"total":96636764160,"available":96029552640,"internally_reserved":0,"allocated":131268608,"data_stored":130981615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953592,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502984,"kb_used_data":168696,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93868856,"statfs":{"total":96636764160,"available":96121708544,"internally_reserved":0,"allocated":172744704,"data_stored":172453639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129523712,"data_stored":129520099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65146880,"data_stored":65081387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172146688,"data_stored":172139003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:55.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:01:55.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'19439", 'reported_seq': 22689, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:51.366729+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:01:51.366729+0000', 'last_peered': '2024-09-10T16:01:51.366729+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:01:51.366729+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'9400", 'ondisk_log_start': "46'9400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5451776, 'num_objects': 1331, 'num_object_clones': 0, 'num_object_copies': 2662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2392, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1331, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19439, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1581, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1061}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:55.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:01:55.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'19235", 'reported_seq': 19716, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:50.902907+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:01:50.902907+0000', 'last_peered': '2024-09-10T16:01:50.902907+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:01:50.902907+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'9200", 'ondisk_log_start': "46'9200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5709848, 'num_objects': 1395, 'num_object_clones': 0, 'num_object_copies': 2790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1395, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17503, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2234}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:55.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:01:55.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'18941", 'reported_seq': 19712, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:51.305809+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:01:51.305809+0000', 'last_peered': '2024-09-10T16:01:51.305809+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:01:51.305809+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5509120, 'num_objects': 1345, 'num_object_clones': 0, 'num_object_copies': 2690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1345, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15591, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2179}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:55.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:01:55.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'19224", 'reported_seq': 20049, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:51.343806+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:01:51.343806+0000', 'last_peered': '2024-09-10T16:01:51.343806+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:01:51.343806+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'9200", 'ondisk_log_start': "46'9200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5545984, 'num_objects': 1354, 'num_object_clones': 0, 'num_object_copies': 2708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1354, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16088, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2165}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:55.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:01:55.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'19109", 'reported_seq': 19532, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:50.932664+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:01:50.932664+0000', 'last_peered': '2024-09-10T16:01:50.932664+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:01:50.932664+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'9100", 'ondisk_log_start': "46'9100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5517312, 'num_objects': 1347, 'num_object_clones': 0, 'num_object_copies': 2694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1347, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16978, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2339}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:01:55.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:01:55.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'19475", 'reported_seq': 20321, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:51.349751+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:01:51.349751+0000', 'last_peered': '2024-09-10T16:01:51.349751+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:01:51.349751+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'9400", 'ondisk_log_start': "46'9400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5648384, 'num_objects': 1379, 'num_object_clones': 0, 'num_object_copies': 2758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1379, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16667, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2099}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:55.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:01:55.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'19375", 'reported_seq': 20130, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:51.373301+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:01:51.373301+0000', 'last_peered': '2024-09-10T16:01:51.373301+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:01:51.373301+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'9300", 'ondisk_log_start': "46'9300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5500928, 'num_objects': 1343, 'num_object_clones': 0, 'num_object_copies': 2686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1343, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15793, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:55.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:01:55.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'19409", 'reported_seq': 20241, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:51.355245+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:01:51.355245+0000', 'last_peered': '2024-09-10T16:01:51.355245+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:01:51.355245+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'9400", 'ondisk_log_start': "46'9400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5558272, 'num_objects': 1357, 'num_object_clones': 0, 'num_object_copies': 2714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1357, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16015, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2179}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:55.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:01:55.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'19771", 'reported_seq': 20538, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:51.264857+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:01:51.264857+0000', 'last_peered': '2024-09-10T16:01:51.264857+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:01:51.264857+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'9700", 'ondisk_log_start': "46'9700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5435875, 'num_objects': 1328, 'num_object_clones': 0, 'num_object_copies': 2656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1328, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16590, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2366}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:01:55.127 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:01:55.840 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:55.840 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31100/43128 objects degraded (72.111%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":641,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":21564,"data_bytes":88310286,"bytes_used":1654915072,"bytes_avail":288255377408,"bytes_total":289910292480,"degraded_objects":31100,"degraded_total":43128,"degraded_ratio":0.72110925616768684,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":409},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:01:55.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:01:55.862 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:56.414 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:56.414 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:01:56.436 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-10T16:01:54.627523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88310286,"num_objects":21564,"num_object_clones":0,"num_object_copies":43128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":271490,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15550,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1616128,"kb_used_data":361996,"kb_used_omap":12,"kb_used_meta":1254067,"kb_avail":281499392,"statfs":{"total":289910292480,"available":288255377408,"internally_reserved":0,"allocated":370683904,"data_stored":369756973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1284164613},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,11,10,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20119552,"num_objects":-4912,"num_object_clones":0,"num_object_copies":-9824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4912,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003804"},"pg_stats":[{"pgid":"2.f","version":"50'19439","reported_seq":22689,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.366729+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:51.366729+0000","last_peered":"2024-09-10T16:01:51.366729+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:51.366729+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5451776,"num_objects":1331,"num_object_clones":0,"num_object_copies":2662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2392,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19439,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1581,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1061}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19722","reported_seq":19840,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.033730+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:51.033730+0000","last_peered":"2024-09-10T16:01:51.033730+0000","last_clean":"2024-09-10T16:01:51.033730+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:51.033730+0000","last_undegraded":"2024-09-10T16:01:51.033730+0000","last_fullsized":"2024-09-10T16:01:51.033730+0000","mapping_epoch":36,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5537792,"num_objects":1352,"num_object_clones":0,"num_object_copies":2704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18058,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19235","reported_seq":19716,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:50.902907+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:50.902907+0000","last_peered":"2024-09-10T16:01:50.902907+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:50.902907+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5709848,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17503,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2234}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19286","reported_seq":19376,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:50.984962+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:50.984962+0000","last_peered":"2024-09-10T16:01:50.984962+0000","last_clean":"2024-09-10T16:01:50.984962+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:50.984962+0000","last_undegraded":"2024-09-10T16:01:50.984962+0000","last_fullsized":"2024-09-10T16:01:50.984962+0000","mapping_epoch":36,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5718016,"num_objects":1396,"num_object_clones":0,"num_object_copies":2792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19286,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19005","reported_seq":19142,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.237418+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:51.237418+0000","last_peered":"2024-09-10T16:01:51.237418+0000","last_clean":"2024-09-10T16:01:51.237418+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:51.237418+0000","last_undegraded":"2024-09-10T16:01:51.237418+0000","last_fullsized":"2024-09-10T16:01:51.237418+0000","mapping_epoch":35,"log_start":"46'9000","ondisk_log_start":"46'9000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5431296,"num_objects":1326,"num_object_clones":0,"num_object_copies":2652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16076,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19558","reported_seq":27456,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.946246+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:51.946246+0000","last_peered":"2024-09-10T16:01:51.946246+0000","last_clean":"2024-09-10T16:01:51.946246+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:51.946246+0000","last_undegraded":"2024-09-10T16:01:51.946246+0000","last_fullsized":"2024-09-10T16:01:51.946246+0000","mapping_epoch":47,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5361664,"num_objects":1309,"num_object_clones":0,"num_object_copies":2618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16276,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'18941","reported_seq":19712,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.305809+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:51.305809+0000","last_peered":"2024-09-10T16:01:51.305809+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:51.305809+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5509120,"num_objects":1345,"num_object_clones":0,"num_object_copies":2690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15591,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19552","reported_seq":19694,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.379739+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:51.379739+0000","last_peered":"2024-09-10T16:01:51.379739+0000","last_clean":"2024-09-10T16:01:51.379739+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:51.379739+0000","last_undegraded":"2024-09-10T16:01:51.379739+0000","last_fullsized":"2024-09-10T16:01:51.379739+0000","mapping_epoch":35,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5578752,"num_objects":1362,"num_object_clones":0,"num_object_copies":2724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16311,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19439","reported_seq":27577,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:51.863640+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:51.863640+0000","last_peered":"2024-09-10T16:01:51.863640+0000","last_clean":"2024-09-10T16:01:51.863640+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:51.863640+0000","last_undegraded":"2024-09-10T16:01:51.863640+0000","last_fullsized":"2024-09-10T16:01:51.863640+0000","mapping_epoch":45,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1280,"num_object_clones":0,"num_object_copies":2560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17416,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19224","reported_seq":20049,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.343806+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:51.343806+0000","last_peered":"2024-09-10T16:01:51.343806+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:51.343806+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5545984,"num_objects":1354,"num_object_clones":0,"num_object_copies":2708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16088,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19427","reported_seq":27237,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:50.979238+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:50.979238+0000","last_peered":"2024-09-10T16:01:50.979238+0000","last_clean":"2024-09-10T16:01:50.979238+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:50.979238+0000","last_undegraded":"2024-09-10T16:01:50.979238+0000","last_fullsized":"2024-09-10T16:01:50.979238+0000","mapping_epoch":49,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5562368,"num_objects":1358,"num_object_clones":0,"num_object_copies":2716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17401,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19109","reported_seq":19532,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:50.932664+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:50.932664+0000","last_peered":"2024-09-10T16:01:50.932664+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:50.932664+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5517312,"num_objects":1347,"num_object_clones":0,"num_object_copies":2694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16978,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2339}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19475","reported_seq":20321,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.349751+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:51.349751+0000","last_peered":"2024-09-10T16:01:51.349751+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:51.349751+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5648384,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16667,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19375","reported_seq":20130,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.373301+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:51.373301+0000","last_peered":"2024-09-10T16:01:51.373301+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:51.373301+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5500928,"num_objects":1343,"num_object_clones":0,"num_object_copies":2686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15793,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19409","reported_seq":20241,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.355245+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:51.355245+0000","last_peered":"2024-09-10T16:01:51.355245+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:51.355245+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5558272,"num_objects":1357,"num_object_clones":0,"num_object_copies":2714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16015,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952736+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952736+0000","last_peered":"2024-09-10T16:00:38.952736+0000","last_clean":"2024-09-10T16:00:38.952736+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952736+0000","last_undegraded":"2024-09-10T16:00:38.952736+0000","last_fullsized":"2024-09-10T16:00:38.952736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19771","reported_seq":20538,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:51.264857+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:51.264857+0000","last_peered":"2024-09-10T16:01:51.264857+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:51.264857+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5435875,"num_objects":1328,"num_object_clones":0,"num_object_copies":2656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16590,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":88310267,"num_objects":21562,"num_object_clones":0,"num_object_copies":43124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21562,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":271488,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15548,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":366817280,"data_stored":366740489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626539,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520164,"kb_used_data":65108,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93851676,"statfs":{"total":96636764160,"available":96104116224,"internally_reserved":0,"allocated":66670592,"data_stored":66321719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":465956526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149821,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592980,"kb_used_data":128192,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93778860,"statfs":{"total":96636764160,"available":96029552640,"internally_reserved":0,"allocated":131268608,"data_stored":130981615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953592,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502984,"kb_used_data":168696,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93868856,"statfs":{"total":96636764160,"available":96121708544,"internally_reserved":0,"allocated":172744704,"data_stored":172453639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129523712,"data_stored":129520099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65146880,"data_stored":65081387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172146688,"data_stored":172139003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:01:59.438 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:01:59.989 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:01:59.992 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:00.017 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-10T16:01:58.628587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":78139918,"num_objects":19081,"num_object_clones":0,"num_object_copies":38162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":273973,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15646,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1614140,"kb_used_data":341952,"kb_used_omap":12,"kb_used_meta":1272115,"kb_avail":281501380,"statfs":{"total":289910292480,"available":288257413120,"internally_reserved":0,"allocated":350158848,"data_stored":349231917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1302645765},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,7,10,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21544960,"num_objects":-5260,"num_object_clones":0,"num_object_copies":-10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5260,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003919"},"pg_stats":[{"pgid":"2.f","version":"50'19573","reported_seq":23015,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.433009+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:56.433009+0000","last_peered":"2024-09-10T16:01:56.433009+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:56.433009+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4902912,"num_objects":1197,"num_object_clones":0,"num_object_copies":2394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19573,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1677,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19878","reported_seq":19996,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.048939+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:56.048939+0000","last_peered":"2024-09-10T16:01:56.048939+0000","last_clean":"2024-09-10T16:01:56.048939+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:56.048939+0000","last_undegraded":"2024-09-10T16:01:56.048939+0000","last_fullsized":"2024-09-10T16:01:56.048939+0000","mapping_epoch":36,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4898816,"num_objects":1196,"num_object_clones":0,"num_object_copies":2392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18214,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19429","reported_seq":19909,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.012239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:56.012239+0000","last_peered":"2024-09-10T16:01:56.012239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:56.012239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915224,"num_objects":1201,"num_object_clones":0,"num_object_copies":2402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17697,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2428}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19438","reported_seq":19528,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.012253+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:56.012253+0000","last_peered":"2024-09-10T16:01:56.012253+0000","last_clean":"2024-09-10T16:01:56.012253+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:56.012253+0000","last_undegraded":"2024-09-10T16:01:56.012253+0000","last_fullsized":"2024-09-10T16:01:56.012253+0000","mapping_epoch":36,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5095424,"num_objects":1244,"num_object_clones":0,"num_object_copies":2488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19438,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19164","reported_seq":19301,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.387924+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:56.387924+0000","last_peered":"2024-09-10T16:01:56.387924+0000","last_clean":"2024-09-10T16:01:56.387924+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:56.387924+0000","last_undegraded":"2024-09-10T16:01:56.387924+0000","last_fullsized":"2024-09-10T16:01:56.387924+0000","mapping_epoch":35,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4780032,"num_objects":1167,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16235,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19707","reported_seq":27605,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.996140+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:56.996140+0000","last_peered":"2024-09-10T16:01:56.996140+0000","last_clean":"2024-09-10T16:01:56.996140+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:56.996140+0000","last_undegraded":"2024-09-10T16:01:56.996140+0000","last_fullsized":"2024-09-10T16:01:56.996140+0000","mapping_epoch":47,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4751360,"num_objects":1160,"num_object_clones":0,"num_object_copies":2320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19105","reported_seq":19874,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.423708+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:56.423708+0000","last_peered":"2024-09-10T16:01:56.423708+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:56.423708+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4837376,"num_objects":1181,"num_object_clones":0,"num_object_copies":2362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15755,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19713","reported_seq":19855,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.377953+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:56.377953+0000","last_peered":"2024-09-10T16:01:56.377953+0000","last_clean":"2024-09-10T16:01:56.377953+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:56.377953+0000","last_undegraded":"2024-09-10T16:01:56.377953+0000","last_fullsized":"2024-09-10T16:01:56.377953+0000","mapping_epoch":35,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4919296,"num_objects":1201,"num_object_clones":0,"num_object_copies":2402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16472,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19569","reported_seq":27707,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.984912+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:56.984912+0000","last_peered":"2024-09-10T16:01:56.984912+0000","last_clean":"2024-09-10T16:01:56.984912+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:56.984912+0000","last_undegraded":"2024-09-10T16:01:56.984912+0000","last_fullsized":"2024-09-10T16:01:56.984912+0000","mapping_epoch":45,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4710400,"num_objects":1150,"num_object_clones":0,"num_object_copies":2300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17546,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19357","reported_seq":20182,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.360163+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:56.360163+0000","last_peered":"2024-09-10T16:01:56.360163+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:56.360163+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5001216,"num_objects":1221,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16221,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19597","reported_seq":27407,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.051014+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:56.051014+0000","last_peered":"2024-09-10T16:01:56.051014+0000","last_clean":"2024-09-10T16:01:56.051014+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:56.051014+0000","last_undegraded":"2024-09-10T16:01:56.051014+0000","last_fullsized":"2024-09-10T16:01:56.051014+0000","mapping_epoch":49,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4866048,"num_objects":1188,"num_object_clones":0,"num_object_copies":2376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17571,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19281","reported_seq":19704,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.012769+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:56.012769+0000","last_peered":"2024-09-10T16:01:56.012769+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:56.012769+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4812800,"num_objects":1175,"num_object_clones":0,"num_object_copies":2350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17150,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2511}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19619","reported_seq":20467,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.397402+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:56.397402+0000","last_peered":"2024-09-10T16:01:56.397402+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:56.397402+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5058560,"num_objects":1235,"num_object_clones":0,"num_object_copies":2470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16811,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19535","reported_seq":20291,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.275178+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:56.275178+0000","last_peered":"2024-09-10T16:01:56.275178+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:56.275178+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4845568,"num_objects":1183,"num_object_clones":0,"num_object_copies":2366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15953,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19563","reported_seq":20395,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.331238+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:56.331238+0000","last_peered":"2024-09-10T16:01:56.331238+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:56.331238+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4927488,"num_objects":1203,"num_object_clones":0,"num_object_copies":2406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16169,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19922","reported_seq":20689,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.380592+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:56.380592+0000","last_peered":"2024-09-10T16:01:56.380592+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:56.380592+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4817379,"num_objects":1177,"num_object_clones":0,"num_object_copies":2354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16741,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2517}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":78139899,"num_objects":19079,"num_object_clones":0,"num_object_copies":38158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19079,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":273971,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15644,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":346537984,"data_stored":346461193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626540,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504408,"kb_used_data":57660,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93867432,"statfs":{"total":96636764160,"available":96120250368,"internally_reserved":0,"allocated":59043840,"data_stored":58694967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457436846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149822,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589688,"kb_used_data":120744,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93782152,"statfs":{"total":96636764160,"available":96032923648,"internally_reserved":0,"allocated":123641856,"data_stored":123354863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":480177837},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953593,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520044,"kb_used_data":163548,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93851796,"statfs":{"total":96636764160,"available":96104239104,"internally_reserved":0,"allocated":167473152,"data_stored":167182087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122105856,"data_stored":122102243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57552896,"data_stored":57487403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166879232,"data_stored":166871547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:00.019 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:00.594 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:00.595 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:00.615 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-10T16:01:58.628587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":78139918,"num_objects":19081,"num_object_clones":0,"num_object_copies":38162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":273973,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15646,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1614140,"kb_used_data":341952,"kb_used_omap":12,"kb_used_meta":1272115,"kb_avail":281501380,"statfs":{"total":289910292480,"available":288257413120,"internally_reserved":0,"allocated":350158848,"data_stored":349231917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1302645765},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,7,10,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21544960,"num_objects":-5260,"num_object_clones":0,"num_object_copies":-10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5260,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003919"},"pg_stats":[{"pgid":"2.f","version":"50'19573","reported_seq":23015,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.433009+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:56.433009+0000","last_peered":"2024-09-10T16:01:56.433009+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:56.433009+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4902912,"num_objects":1197,"num_object_clones":0,"num_object_copies":2394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19573,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1677,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19878","reported_seq":19996,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.048939+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:56.048939+0000","last_peered":"2024-09-10T16:01:56.048939+0000","last_clean":"2024-09-10T16:01:56.048939+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:56.048939+0000","last_undegraded":"2024-09-10T16:01:56.048939+0000","last_fullsized":"2024-09-10T16:01:56.048939+0000","mapping_epoch":36,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4898816,"num_objects":1196,"num_object_clones":0,"num_object_copies":2392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18214,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19429","reported_seq":19909,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.012239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:56.012239+0000","last_peered":"2024-09-10T16:01:56.012239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:56.012239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915224,"num_objects":1201,"num_object_clones":0,"num_object_copies":2402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17697,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2428}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19438","reported_seq":19528,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.012253+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:56.012253+0000","last_peered":"2024-09-10T16:01:56.012253+0000","last_clean":"2024-09-10T16:01:56.012253+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:56.012253+0000","last_undegraded":"2024-09-10T16:01:56.012253+0000","last_fullsized":"2024-09-10T16:01:56.012253+0000","mapping_epoch":36,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5095424,"num_objects":1244,"num_object_clones":0,"num_object_copies":2488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19438,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19164","reported_seq":19301,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.387924+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:56.387924+0000","last_peered":"2024-09-10T16:01:56.387924+0000","last_clean":"2024-09-10T16:01:56.387924+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:56.387924+0000","last_undegraded":"2024-09-10T16:01:56.387924+0000","last_fullsized":"2024-09-10T16:01:56.387924+0000","mapping_epoch":35,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4780032,"num_objects":1167,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16235,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19707","reported_seq":27605,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.996140+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:56.996140+0000","last_peered":"2024-09-10T16:01:56.996140+0000","last_clean":"2024-09-10T16:01:56.996140+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:56.996140+0000","last_undegraded":"2024-09-10T16:01:56.996140+0000","last_fullsized":"2024-09-10T16:01:56.996140+0000","mapping_epoch":47,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4751360,"num_objects":1160,"num_object_clones":0,"num_object_copies":2320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19105","reported_seq":19874,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.423708+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:56.423708+0000","last_peered":"2024-09-10T16:01:56.423708+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:56.423708+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4837376,"num_objects":1181,"num_object_clones":0,"num_object_copies":2362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15755,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19713","reported_seq":19855,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.377953+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:56.377953+0000","last_peered":"2024-09-10T16:01:56.377953+0000","last_clean":"2024-09-10T16:01:56.377953+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:56.377953+0000","last_undegraded":"2024-09-10T16:01:56.377953+0000","last_fullsized":"2024-09-10T16:01:56.377953+0000","mapping_epoch":35,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4919296,"num_objects":1201,"num_object_clones":0,"num_object_copies":2402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16472,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19569","reported_seq":27707,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.984912+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:56.984912+0000","last_peered":"2024-09-10T16:01:56.984912+0000","last_clean":"2024-09-10T16:01:56.984912+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:56.984912+0000","last_undegraded":"2024-09-10T16:01:56.984912+0000","last_fullsized":"2024-09-10T16:01:56.984912+0000","mapping_epoch":45,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4710400,"num_objects":1150,"num_object_clones":0,"num_object_copies":2300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17546,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19357","reported_seq":20182,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.360163+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:56.360163+0000","last_peered":"2024-09-10T16:01:56.360163+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:56.360163+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5001216,"num_objects":1221,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16221,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19597","reported_seq":27407,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.051014+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:56.051014+0000","last_peered":"2024-09-10T16:01:56.051014+0000","last_clean":"2024-09-10T16:01:56.051014+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:56.051014+0000","last_undegraded":"2024-09-10T16:01:56.051014+0000","last_fullsized":"2024-09-10T16:01:56.051014+0000","mapping_epoch":49,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4866048,"num_objects":1188,"num_object_clones":0,"num_object_copies":2376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17571,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19281","reported_seq":19704,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.012769+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:56.012769+0000","last_peered":"2024-09-10T16:01:56.012769+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:56.012769+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4812800,"num_objects":1175,"num_object_clones":0,"num_object_copies":2350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17150,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2511}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19619","reported_seq":20467,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.397402+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:56.397402+0000","last_peered":"2024-09-10T16:01:56.397402+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:56.397402+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5058560,"num_objects":1235,"num_object_clones":0,"num_object_copies":2470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16811,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19535","reported_seq":20291,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.275178+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:56.275178+0000","last_peered":"2024-09-10T16:01:56.275178+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:56.275178+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4845568,"num_objects":1183,"num_object_clones":0,"num_object_copies":2366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15953,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19563","reported_seq":20395,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.331238+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:56.331238+0000","last_peered":"2024-09-10T16:01:56.331238+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:56.331238+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4927488,"num_objects":1203,"num_object_clones":0,"num_object_copies":2406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16169,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19922","reported_seq":20689,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.380592+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:56.380592+0000","last_peered":"2024-09-10T16:01:56.380592+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:56.380592+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4817379,"num_objects":1177,"num_object_clones":0,"num_object_copies":2354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16741,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2517}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":78139899,"num_objects":19079,"num_object_clones":0,"num_object_copies":38158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19079,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":273971,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15644,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":346537984,"data_stored":346461193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626540,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504408,"kb_used_data":57660,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93867432,"statfs":{"total":96636764160,"available":96120250368,"internally_reserved":0,"allocated":59043840,"data_stored":58694967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457436846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149822,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589688,"kb_used_data":120744,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93782152,"statfs":{"total":96636764160,"available":96032923648,"internally_reserved":0,"allocated":123641856,"data_stored":123354863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":480177837},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953593,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520044,"kb_used_data":163548,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93851796,"statfs":{"total":96636764160,"available":96104239104,"internally_reserved":0,"allocated":167473152,"data_stored":167182087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122105856,"data_stored":122102243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57552896,"data_stored":57487403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166879232,"data_stored":166871547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:00.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:00.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'19573", 'reported_seq': 23015, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:56.433009+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:01:56.433009+0000', 'last_peered': '2024-09-10T16:01:56.433009+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:01:56.433009+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'9500", 'ondisk_log_start': "46'9500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4902912, 'num_objects': 1197, 'num_object_clones': 0, 'num_object_copies': 2394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2296, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1197, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19573, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1677, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1099}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:00.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:00.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'19429", 'reported_seq': 19909, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:56.012239+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:01:56.012239+0000', 'last_peered': '2024-09-10T16:01:56.012239+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:01:56.012239+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'9400", 'ondisk_log_start': "46'9400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4915224, 'num_objects': 1201, 'num_object_clones': 0, 'num_object_copies': 2402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1201, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17697, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2428}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:00.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:00.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'19105", 'reported_seq': 19874, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:56.423708+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:01:56.423708+0000', 'last_peered': '2024-09-10T16:01:56.423708+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:01:56.423708+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'9100", 'ondisk_log_start': "46'9100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4837376, 'num_objects': 1181, 'num_object_clones': 0, 'num_object_copies': 2362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1181, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15755, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2343}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:00.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:00.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'19357", 'reported_seq': 20182, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:56.360163+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:01:56.360163+0000', 'last_peered': '2024-09-10T16:01:56.360163+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:01:56.360163+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'9300", 'ondisk_log_start': "46'9300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5001216, 'num_objects': 1221, 'num_object_clones': 0, 'num_object_copies': 2442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1221, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16221, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2298}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:00.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:00.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'19281", 'reported_seq': 19704, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:56.012769+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:01:56.012769+0000', 'last_peered': '2024-09-10T16:01:56.012769+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:01:56.012769+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'9200", 'ondisk_log_start': "46'9200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4812800, 'num_objects': 1175, 'num_object_clones': 0, 'num_object_copies': 2350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1175, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17150, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2511}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:00.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:00.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'19619", 'reported_seq': 20467, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:56.397402+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:01:56.397402+0000', 'last_peered': '2024-09-10T16:01:56.397402+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:01:56.397402+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'9600", 'ondisk_log_start': "46'9600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5058560, 'num_objects': 1235, 'num_object_clones': 0, 'num_object_copies': 2470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16811, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2243}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:00.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:00.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'19535", 'reported_seq': 20291, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:56.275178+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:01:56.275178+0000', 'last_peered': '2024-09-10T16:01:56.275178+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:01:56.275178+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'9500", 'ondisk_log_start': "46'9500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4845568, 'num_objects': 1183, 'num_object_clones': 0, 'num_object_copies': 2366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1183, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15953, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2443}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:00.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:00.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'19563", 'reported_seq': 20395, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:56.331238+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:01:56.331238+0000', 'last_peered': '2024-09-10T16:01:56.331238+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:01:56.331238+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'9500", 'ondisk_log_start': "46'9500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4927488, 'num_objects': 1203, 'num_object_clones': 0, 'num_object_copies': 2406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1203, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16169, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2333}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:00.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:00.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'19922", 'reported_seq': 20689, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:01:56.380592+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:01:56.380592+0000', 'last_peered': '2024-09-10T16:01:56.380592+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:01:56.380592+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'9900", 'ondisk_log_start': "46'9900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4817379, 'num_objects': 1177, 'num_object_clones': 0, 'num_object_copies': 2354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16741, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2517}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:00.619 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:01.296 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:01.296 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31004/38162 objects degraded (81.243%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":647,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":19081,"data_bytes":78139918,"bytes_used":1652879360,"bytes_avail":288257413120,"bytes_total":289910292480,"degraded_objects":31004,"degraded_total":38162,"degraded_ratio":0.81243121429694465,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":412},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:01.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:01.315 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:01.876 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:01.879 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:01.908 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-10T16:02:00.629319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":78139918,"num_objects":19081,"num_object_clones":0,"num_object_copies":38162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":273973,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15646,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1614140,"kb_used_data":341952,"kb_used_omap":12,"kb_used_meta":1272115,"kb_avail":281501380,"statfs":{"total":289910292480,"available":288257413120,"internally_reserved":0,"allocated":350158848,"data_stored":349231917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1302645765},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,7,10,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20299776,"num_objects":-4956,"num_object_clones":0,"num_object_copies":-9912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4956,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003781"},"pg_stats":[{"pgid":"2.f","version":"50'19573","reported_seq":23015,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.433009+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:01:56.433009+0000","last_peered":"2024-09-10T16:01:56.433009+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:01:56.433009+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4902912,"num_objects":1197,"num_object_clones":0,"num_object_copies":2394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19573,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1677,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'19878","reported_seq":19996,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.048939+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:01:56.048939+0000","last_peered":"2024-09-10T16:01:56.048939+0000","last_clean":"2024-09-10T16:01:56.048939+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:01:56.048939+0000","last_undegraded":"2024-09-10T16:01:56.048939+0000","last_fullsized":"2024-09-10T16:01:56.048939+0000","mapping_epoch":36,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4898816,"num_objects":1196,"num_object_clones":0,"num_object_copies":2392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18214,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19429","reported_seq":19909,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.012239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:01:56.012239+0000","last_peered":"2024-09-10T16:01:56.012239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:01:56.012239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915224,"num_objects":1201,"num_object_clones":0,"num_object_copies":2402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17697,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2428}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19438","reported_seq":19528,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.012253+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:01:56.012253+0000","last_peered":"2024-09-10T16:01:56.012253+0000","last_clean":"2024-09-10T16:01:56.012253+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:01:56.012253+0000","last_undegraded":"2024-09-10T16:01:56.012253+0000","last_fullsized":"2024-09-10T16:01:56.012253+0000","mapping_epoch":36,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5095424,"num_objects":1244,"num_object_clones":0,"num_object_copies":2488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19438,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19164","reported_seq":19301,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.387924+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:01:56.387924+0000","last_peered":"2024-09-10T16:01:56.387924+0000","last_clean":"2024-09-10T16:01:56.387924+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:01:56.387924+0000","last_undegraded":"2024-09-10T16:01:56.387924+0000","last_fullsized":"2024-09-10T16:01:56.387924+0000","mapping_epoch":35,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4780032,"num_objects":1167,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16235,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19707","reported_seq":27605,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.996140+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:01:56.996140+0000","last_peered":"2024-09-10T16:01:56.996140+0000","last_clean":"2024-09-10T16:01:56.996140+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:01:56.996140+0000","last_undegraded":"2024-09-10T16:01:56.996140+0000","last_fullsized":"2024-09-10T16:01:56.996140+0000","mapping_epoch":47,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4751360,"num_objects":1160,"num_object_clones":0,"num_object_copies":2320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19105","reported_seq":19874,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.423708+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:01:56.423708+0000","last_peered":"2024-09-10T16:01:56.423708+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:01:56.423708+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9100","ondisk_log_start":"46'9100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4837376,"num_objects":1181,"num_object_clones":0,"num_object_copies":2362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15755,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19713","reported_seq":19855,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.377953+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:01:56.377953+0000","last_peered":"2024-09-10T16:01:56.377953+0000","last_clean":"2024-09-10T16:01:56.377953+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:01:56.377953+0000","last_undegraded":"2024-09-10T16:01:56.377953+0000","last_fullsized":"2024-09-10T16:01:56.377953+0000","mapping_epoch":35,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4919296,"num_objects":1201,"num_object_clones":0,"num_object_copies":2402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16472,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19569","reported_seq":27707,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.984912+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:01:56.984912+0000","last_peered":"2024-09-10T16:01:56.984912+0000","last_clean":"2024-09-10T16:01:56.984912+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:01:56.984912+0000","last_undegraded":"2024-09-10T16:01:56.984912+0000","last_fullsized":"2024-09-10T16:01:56.984912+0000","mapping_epoch":45,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4710400,"num_objects":1150,"num_object_clones":0,"num_object_copies":2300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17546,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19357","reported_seq":20182,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.360163+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:01:56.360163+0000","last_peered":"2024-09-10T16:01:56.360163+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:01:56.360163+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5001216,"num_objects":1221,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16221,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19597","reported_seq":27407,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:01:56.051014+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:01:56.051014+0000","last_peered":"2024-09-10T16:01:56.051014+0000","last_clean":"2024-09-10T16:01:56.051014+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:01:56.051014+0000","last_undegraded":"2024-09-10T16:01:56.051014+0000","last_fullsized":"2024-09-10T16:01:56.051014+0000","mapping_epoch":49,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4866048,"num_objects":1188,"num_object_clones":0,"num_object_copies":2376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17571,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19281","reported_seq":19704,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.012769+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:01:56.012769+0000","last_peered":"2024-09-10T16:01:56.012769+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:01:56.012769+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4812800,"num_objects":1175,"num_object_clones":0,"num_object_copies":2350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17150,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2511}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19619","reported_seq":20467,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.397402+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:01:56.397402+0000","last_peered":"2024-09-10T16:01:56.397402+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:01:56.397402+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5058560,"num_objects":1235,"num_object_clones":0,"num_object_copies":2470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16811,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19535","reported_seq":20291,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.275178+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:01:56.275178+0000","last_peered":"2024-09-10T16:01:56.275178+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:01:56.275178+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4845568,"num_objects":1183,"num_object_clones":0,"num_object_copies":2366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15953,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19563","reported_seq":20395,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.331238+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:01:56.331238+0000","last_peered":"2024-09-10T16:01:56.331238+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:01:56.331238+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4927488,"num_objects":1203,"num_object_clones":0,"num_object_copies":2406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16169,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'19922","reported_seq":20689,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:01:56.380592+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:01:56.380592+0000","last_peered":"2024-09-10T16:01:56.380592+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:01:56.380592+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4817379,"num_objects":1177,"num_object_clones":0,"num_object_copies":2354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16741,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2517}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":78139899,"num_objects":19079,"num_object_clones":0,"num_object_copies":38158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19079,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":273971,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15644,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":346537984,"data_stored":346461193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626540,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504408,"kb_used_data":57660,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93867432,"statfs":{"total":96636764160,"available":96120250368,"internally_reserved":0,"allocated":59043840,"data_stored":58694967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457436846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149822,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589688,"kb_used_data":120744,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93782152,"statfs":{"total":96636764160,"available":96032923648,"internally_reserved":0,"allocated":123641856,"data_stored":123354863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":480177837},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953593,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520044,"kb_used_data":163548,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93851796,"statfs":{"total":96636764160,"available":96104239104,"internally_reserved":0,"allocated":167473152,"data_stored":167182087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122105856,"data_stored":122102243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57552896,"data_stored":57487403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166879232,"data_stored":166871547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:04.910 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:05.530 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:05.559 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-10T16:02:04.630866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67838478,"num_objects":16566,"num_object_clones":0,"num_object_copies":33132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16566,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":276488,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15719,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1628692,"kb_used_data":321820,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281486828,"statfs":{"total":289910292480,"available":288242511872,"internally_reserved":0,"allocated":329543680,"data_stored":328616749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1338166277},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,9,11,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20471808,"num_objects":-4998,"num_object_clones":0,"num_object_copies":-9996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4998,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003978"},"pg_stats":[{"pgid":"2.f","version":"50'19737","reported_seq":23324,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.500650+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:01.500650+0000","last_peered":"2024-09-10T16:02:01.500650+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:01.500650+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4231168,"num_objects":1033,"num_object_clones":0,"num_object_copies":2066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19737,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1750,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1190}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20024","reported_seq":20141,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:00.978008+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:00.978008+0000","last_peered":"2024-09-10T16:02:00.978008+0000","last_clean":"2024-09-10T16:02:00.978008+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:00.978008+0000","last_undegraded":"2024-09-10T16:02:00.978008+0000","last_fullsized":"2024-09-10T16:02:00.978008+0000","mapping_epoch":36,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4300800,"num_objects":1050,"num_object_clones":0,"num_object_copies":2100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18360,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19575","reported_seq":20056,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:00.954963+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:00.954963+0000","last_peered":"2024-09-10T16:02:00.954963+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:00.954963+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4317208,"num_objects":1055,"num_object_clones":0,"num_object_copies":2110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1055,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17843,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19597","reported_seq":19687,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:00.941934+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:00.941934+0000","last_peered":"2024-09-10T16:02:00.941934+0000","last_clean":"2024-09-10T16:02:00.941934+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:00.941934+0000","last_undegraded":"2024-09-10T16:02:00.941934+0000","last_fullsized":"2024-09-10T16:02:00.941934+0000","mapping_epoch":36,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4444160,"num_objects":1085,"num_object_clones":0,"num_object_copies":2170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19597,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19314","reported_seq":19451,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:01.399029+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:01.399029+0000","last_peered":"2024-09-10T16:02:01.399029+0000","last_clean":"2024-09-10T16:02:01.399029+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:01.399029+0000","last_undegraded":"2024-09-10T16:02:01.399029+0000","last_fullsized":"2024-09-10T16:02:01.399029+0000","mapping_epoch":35,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4165632,"num_objects":1017,"num_object_clones":0,"num_object_copies":2034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16385,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19868","reported_seq":27766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:02.020504+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:02.020504+0000","last_peered":"2024-09-10T16:02:02.020504+0000","last_clean":"2024-09-10T16:02:02.020504+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:02.020504+0000","last_undegraded":"2024-09-10T16:02:02.020504+0000","last_fullsized":"2024-09-10T16:02:02.020504+0000","mapping_epoch":47,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4091904,"num_objects":999,"num_object_clones":0,"num_object_copies":1998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16586,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19235","reported_seq":20006,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.318450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:01.318450+0000","last_peered":"2024-09-10T16:02:01.318450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:01.318450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304896,"num_objects":1051,"num_object_clones":0,"num_object_copies":2102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15885,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2473}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19850","reported_seq":19991,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:01.504064+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:01.504064+0000","last_peered":"2024-09-10T16:02:01.504064+0000","last_clean":"2024-09-10T16:02:01.504064+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:01.504064+0000","last_undegraded":"2024-09-10T16:02:01.504064+0000","last_fullsized":"2024-09-10T16:02:01.504064+0000","mapping_epoch":35,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4358144,"num_objects":1064,"num_object_clones":0,"num_object_copies":2128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16609,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19734","reported_seq":27872,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:02.020683+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:02.020683+0000","last_peered":"2024-09-10T16:02:02.020683+0000","last_clean":"2024-09-10T16:02:02.020683+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:02.020683+0000","last_undegraded":"2024-09-10T16:02:02.020683+0000","last_fullsized":"2024-09-10T16:02:02.020683+0000","mapping_epoch":45,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4034560,"num_objects":985,"num_object_clones":0,"num_object_copies":1970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17711,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19510","reported_seq":20335,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.358910+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:01.358910+0000","last_peered":"2024-09-10T16:02:01.358910+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:01.358910+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4374528,"num_objects":1068,"num_object_clones":0,"num_object_copies":2136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16374,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2451}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19762","reported_seq":27572,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:00.984863+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:00.984863+0000","last_peered":"2024-09-10T16:02:00.984863+0000","last_clean":"2024-09-10T16:02:00.984863+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:00.984863+0000","last_undegraded":"2024-09-10T16:02:00.984863+0000","last_fullsized":"2024-09-10T16:02:00.984863+0000","mapping_epoch":49,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1023,"num_object_clones":0,"num_object_copies":2046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17736,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19430","reported_seq":19852,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.033634+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:01.033634+0000","last_peered":"2024-09-10T16:02:01.033634+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:01.033634+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4202496,"num_objects":1026,"num_object_clones":0,"num_object_copies":2052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17299,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2660}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19809","reported_seq":20657,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.472075+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:01.472075+0000","last_peered":"2024-09-10T16:02:01.472075+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:01.472075+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4280320,"num_objects":1045,"num_object_clones":0,"num_object_copies":2090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19699","reported_seq":20455,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.484676+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:01.484676+0000","last_peered":"2024-09-10T16:02:01.484676+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:01.484676+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4173824,"num_objects":1019,"num_object_clones":0,"num_object_copies":2038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16117,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2607}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19744","reported_seq":20576,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.506223+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:01.506223+0000","last_peered":"2024-09-10T16:02:01.506223+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:01.506223+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4186112,"num_objects":1022,"num_object_clones":0,"num_object_copies":2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16350,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20077","reported_seq":20844,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.377580+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:01.377580+0000","last_peered":"2024-09-10T16:02:01.377580+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:01.377580+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4182499,"num_objects":1022,"num_object_clones":0,"num_object_copies":2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16896,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":67838459,"num_objects":16564,"num_object_clones":0,"num_object_copies":33128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":276486,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15717,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":325980160,"data_stored":325903369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626541,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518844,"kb_used_data":49900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93852996,"statfs":{"total":96636764160,"available":96105467904,"internally_reserved":0,"allocated":51097600,"data_stored":50748727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480177838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,2,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149823,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590532,"kb_used_data":113264,"kb_used_omap":4,"kb_used_meta":477243,"kb_avail":93781308,"statfs":{"total":96636764160,"available":96032059392,"internally_reserved":0,"allocated":115982336,"data_stored":115695343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":488697517},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,4,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953594,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519316,"kb_used_data":158656,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93852524,"statfs":{"total":96636764160,"available":96104984576,"internally_reserved":0,"allocated":162463744,"data_stored":162172679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114257920,"data_stored":114254307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49807360,"data_stored":49741867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":161914880,"data_stored":161907195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:05.561 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:06.147 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:06.148 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:06.169 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-10T16:02:04.630866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67838478,"num_objects":16566,"num_object_clones":0,"num_object_copies":33132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16566,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":276488,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15719,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1628692,"kb_used_data":321820,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281486828,"statfs":{"total":289910292480,"available":288242511872,"internally_reserved":0,"allocated":329543680,"data_stored":328616749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1338166277},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,9,11,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20471808,"num_objects":-4998,"num_object_clones":0,"num_object_copies":-9996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4998,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003978"},"pg_stats":[{"pgid":"2.f","version":"50'19737","reported_seq":23324,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.500650+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:01.500650+0000","last_peered":"2024-09-10T16:02:01.500650+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:01.500650+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4231168,"num_objects":1033,"num_object_clones":0,"num_object_copies":2066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19737,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1750,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1190}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20024","reported_seq":20141,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:00.978008+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:00.978008+0000","last_peered":"2024-09-10T16:02:00.978008+0000","last_clean":"2024-09-10T16:02:00.978008+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:00.978008+0000","last_undegraded":"2024-09-10T16:02:00.978008+0000","last_fullsized":"2024-09-10T16:02:00.978008+0000","mapping_epoch":36,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4300800,"num_objects":1050,"num_object_clones":0,"num_object_copies":2100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18360,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19575","reported_seq":20056,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:00.954963+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:00.954963+0000","last_peered":"2024-09-10T16:02:00.954963+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:00.954963+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4317208,"num_objects":1055,"num_object_clones":0,"num_object_copies":2110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1055,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17843,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19597","reported_seq":19687,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:00.941934+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:00.941934+0000","last_peered":"2024-09-10T16:02:00.941934+0000","last_clean":"2024-09-10T16:02:00.941934+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:00.941934+0000","last_undegraded":"2024-09-10T16:02:00.941934+0000","last_fullsized":"2024-09-10T16:02:00.941934+0000","mapping_epoch":36,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4444160,"num_objects":1085,"num_object_clones":0,"num_object_copies":2170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19597,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19314","reported_seq":19451,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:01.399029+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:01.399029+0000","last_peered":"2024-09-10T16:02:01.399029+0000","last_clean":"2024-09-10T16:02:01.399029+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:01.399029+0000","last_undegraded":"2024-09-10T16:02:01.399029+0000","last_fullsized":"2024-09-10T16:02:01.399029+0000","mapping_epoch":35,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4165632,"num_objects":1017,"num_object_clones":0,"num_object_copies":2034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16385,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19868","reported_seq":27766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:02.020504+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:02.020504+0000","last_peered":"2024-09-10T16:02:02.020504+0000","last_clean":"2024-09-10T16:02:02.020504+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:02.020504+0000","last_undegraded":"2024-09-10T16:02:02.020504+0000","last_fullsized":"2024-09-10T16:02:02.020504+0000","mapping_epoch":47,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4091904,"num_objects":999,"num_object_clones":0,"num_object_copies":1998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16586,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19235","reported_seq":20006,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.318450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:01.318450+0000","last_peered":"2024-09-10T16:02:01.318450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:01.318450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9200","ondisk_log_start":"46'9200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304896,"num_objects":1051,"num_object_clones":0,"num_object_copies":2102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15885,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2473}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'19850","reported_seq":19991,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:01.504064+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:01.504064+0000","last_peered":"2024-09-10T16:02:01.504064+0000","last_clean":"2024-09-10T16:02:01.504064+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:01.504064+0000","last_undegraded":"2024-09-10T16:02:01.504064+0000","last_fullsized":"2024-09-10T16:02:01.504064+0000","mapping_epoch":35,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4358144,"num_objects":1064,"num_object_clones":0,"num_object_copies":2128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16609,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19734","reported_seq":27872,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:02.020683+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:02.020683+0000","last_peered":"2024-09-10T16:02:02.020683+0000","last_clean":"2024-09-10T16:02:02.020683+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:02.020683+0000","last_undegraded":"2024-09-10T16:02:02.020683+0000","last_fullsized":"2024-09-10T16:02:02.020683+0000","mapping_epoch":45,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4034560,"num_objects":985,"num_object_clones":0,"num_object_copies":1970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17711,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19510","reported_seq":20335,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.358910+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:01.358910+0000","last_peered":"2024-09-10T16:02:01.358910+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:01.358910+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4374528,"num_objects":1068,"num_object_clones":0,"num_object_copies":2136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16374,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2451}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19762","reported_seq":27572,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:00.984863+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:00.984863+0000","last_peered":"2024-09-10T16:02:00.984863+0000","last_clean":"2024-09-10T16:02:00.984863+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:00.984863+0000","last_undegraded":"2024-09-10T16:02:00.984863+0000","last_fullsized":"2024-09-10T16:02:00.984863+0000","mapping_epoch":49,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1023,"num_object_clones":0,"num_object_copies":2046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17736,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19430","reported_seq":19852,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.033634+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:01.033634+0000","last_peered":"2024-09-10T16:02:01.033634+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:01.033634+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4202496,"num_objects":1026,"num_object_clones":0,"num_object_copies":2052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17299,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2660}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19809","reported_seq":20657,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.472075+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:01.472075+0000","last_peered":"2024-09-10T16:02:01.472075+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:01.472075+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4280320,"num_objects":1045,"num_object_clones":0,"num_object_copies":2090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19699","reported_seq":20455,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.484676+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:01.484676+0000","last_peered":"2024-09-10T16:02:01.484676+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:01.484676+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4173824,"num_objects":1019,"num_object_clones":0,"num_object_copies":2038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16117,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2607}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19744","reported_seq":20576,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.506223+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:01.506223+0000","last_peered":"2024-09-10T16:02:01.506223+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:01.506223+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4186112,"num_objects":1022,"num_object_clones":0,"num_object_copies":2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16350,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20077","reported_seq":20844,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:01.377580+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:01.377580+0000","last_peered":"2024-09-10T16:02:01.377580+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:01.377580+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4182499,"num_objects":1022,"num_object_clones":0,"num_object_copies":2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16896,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":67838459,"num_objects":16564,"num_object_clones":0,"num_object_copies":33128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":276486,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15717,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":325980160,"data_stored":325903369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626541,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518844,"kb_used_data":49900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93852996,"statfs":{"total":96636764160,"available":96105467904,"internally_reserved":0,"allocated":51097600,"data_stored":50748727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480177838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,2,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149823,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590532,"kb_used_data":113264,"kb_used_omap":4,"kb_used_meta":477243,"kb_avail":93781308,"statfs":{"total":96636764160,"available":96032059392,"internally_reserved":0,"allocated":115982336,"data_stored":115695343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":488697517},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,4,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953594,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519316,"kb_used_data":158656,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93852524,"statfs":{"total":96636764160,"available":96104984576,"internally_reserved":0,"allocated":162463744,"data_stored":162172679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114257920,"data_stored":114254307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49807360,"data_stored":49741867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":161914880,"data_stored":161907195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:06.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:06.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'19737", 'reported_seq': 23324, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:01.500650+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:02:01.500650+0000', 'last_peered': '2024-09-10T16:02:01.500650+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:02:01.500650+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'9700", 'ondisk_log_start': "46'9700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4231168, 'num_objects': 1033, 'num_object_clones': 0, 'num_object_copies': 2066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2223, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19737, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1750, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1190}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:06.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:06.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'19575", 'reported_seq': 20056, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:00.954963+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:00.954963+0000', 'last_peered': '2024-09-10T16:02:00.954963+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:00.954963+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'9500", 'ondisk_log_start': "46'9500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4317208, 'num_objects': 1055, 'num_object_clones': 0, 'num_object_copies': 2110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1055, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17843, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:06.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:06.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'19235", 'reported_seq': 20006, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:01.318450+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:01.318450+0000', 'last_peered': '2024-09-10T16:02:01.318450+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:01.318450+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'9200", 'ondisk_log_start': "46'9200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4304896, 'num_objects': 1051, 'num_object_clones': 0, 'num_object_copies': 2102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15885, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2473}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:06.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:06.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'19510", 'reported_seq': 20335, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:01.358910+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:01.358910+0000', 'last_peered': '2024-09-10T16:02:01.358910+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:01.358910+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'9500", 'ondisk_log_start': "46'9500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4374528, 'num_objects': 1068, 'num_object_clones': 0, 'num_object_copies': 2136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1068, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16374, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2451}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:06.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:06.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'19430", 'reported_seq': 19852, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:01.033634+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:02:01.033634+0000', 'last_peered': '2024-09-10T16:02:01.033634+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:02:01.033634+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'9400", 'ondisk_log_start': "46'9400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4202496, 'num_objects': 1026, 'num_object_clones': 0, 'num_object_copies': 2052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17299, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2660}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:06.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:06.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'19809", 'reported_seq': 20657, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:01.472075+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:01.472075+0000', 'last_peered': '2024-09-10T16:02:01.472075+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:01.472075+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'9800", 'ondisk_log_start': "46'9800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4280320, 'num_objects': 1045, 'num_object_clones': 0, 'num_object_copies': 2090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:06.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:06.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'19699", 'reported_seq': 20455, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:01.484676+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:01.484676+0000', 'last_peered': '2024-09-10T16:02:01.484676+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:01.484676+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'9600", 'ondisk_log_start': "46'9600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4173824, 'num_objects': 1019, 'num_object_clones': 0, 'num_object_copies': 2038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1019, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16117, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2607}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:06.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:06.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'19744", 'reported_seq': 20576, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:01.506223+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:01.506223+0000', 'last_peered': '2024-09-10T16:02:01.506223+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:01.506223+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'9700", 'ondisk_log_start': "46'9700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4186112, 'num_objects': 1022, 'num_object_clones': 0, 'num_object_copies': 2044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1022, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16350, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:06.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:06.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'20077", 'reported_seq': 20844, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:01.377580+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:01.377580+0000', 'last_peered': '2024-09-10T16:02:01.377580+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:01.377580+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'10000", 'ondisk_log_start': "46'10000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4182499, 'num_objects': 1022, 'num_object_clones': 0, 'num_object_copies': 2044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1022, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16896, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2672}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:06.172 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:06.860 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:06.861 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30857/28734 objects degraded (107.388%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":652,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":14367,"data_bytes":58831374,"bytes_used":1646555136,"bytes_avail":288263737344,"bytes_total":289910292480,"degraded_objects":30857,"degraded_total":28734,"degraded_ratio":1.0738845966450894,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":599},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:06.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:06.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:07.432 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:07.432 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:07.453 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-10T16:02:06.631612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":58831374,"num_objects":14367,"num_object_clones":0,"num_object_copies":28734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30857,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14367,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":278687,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15793,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160867,"ondisk_log_size":160867,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1607964,"kb_used_data":309388,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281507556,"statfs":{"total":289910292480,"available":288263737344,"internally_reserved":0,"allocated":316813312,"data_stored":315886381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1329646597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,7,1,6,11,10,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29478912,"num_objects":-7197,"num_object_clones":0,"num_object_copies":-14394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7197,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004088"},"pg_stats":[{"pgid":"2.f","version":"50'19889","reported_seq":23625,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.503541+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:06.503541+0000","last_peered":"2024-09-10T16:02:06.503541+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:06.503541+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3608576,"num_objects":881,"num_object_clones":0,"num_object_copies":1762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19889,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1824,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1268}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20178","reported_seq":20296,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:05.929633+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:05.929633+0000","last_peered":"2024-09-10T16:02:05.929633+0000","last_clean":"2024-09-10T16:02:05.929633+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:05.929633+0000","last_undegraded":"2024-09-10T16:02:05.929633+0000","last_fullsized":"2024-09-10T16:02:05.929633+0000","mapping_epoch":36,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":896,"num_object_clones":0,"num_object_copies":1792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18514,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19748","reported_seq":20229,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.052921+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:06.052921+0000","last_peered":"2024-09-10T16:02:06.052921+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:06.052921+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3608600,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18016,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2747}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19757","reported_seq":19847,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:06.041194+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:06.041194+0000","last_peered":"2024-09-10T16:02:06.041194+0000","last_clean":"2024-09-10T16:02:06.041194+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:06.041194+0000","last_undegraded":"2024-09-10T16:02:06.041194+0000","last_fullsized":"2024-09-10T16:02:06.041194+0000","mapping_epoch":36,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3788800,"num_objects":925,"num_object_clones":0,"num_object_copies":1850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19757,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19446","reported_seq":19583,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:06.459263+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:06.459263+0000","last_peered":"2024-09-10T16:02:06.459263+0000","last_clean":"2024-09-10T16:02:06.459263+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:06.459263+0000","last_undegraded":"2024-09-10T16:02:06.459263+0000","last_fullsized":"2024-09-10T16:02:06.459263+0000","mapping_epoch":35,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3624960,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16517,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19868","reported_seq":27766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:02.020504+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:02.020504+0000","last_peered":"2024-09-10T16:02:02.020504+0000","last_clean":"2024-09-10T16:02:02.020504+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:02.020504+0000","last_undegraded":"2024-09-10T16:02:02.020504+0000","last_fullsized":"2024-09-10T16:02:02.020504+0000","mapping_epoch":47,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4091904,"num_objects":999,"num_object_clones":0,"num_object_copies":1998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16586,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19381","reported_seq":20152,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.509741+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:06.509741+0000","last_peered":"2024-09-10T16:02:06.509741+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:06.509741+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3706880,"num_objects":905,"num_object_clones":0,"num_object_copies":1810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16031,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2619}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20020","reported_seq":20161,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:06.494371+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:06.494371+0000","last_peered":"2024-09-10T16:02:06.494371+0000","last_clean":"2024-09-10T16:02:06.494371+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:06.494371+0000","last_undegraded":"2024-09-10T16:02:06.494371+0000","last_fullsized":"2024-09-10T16:02:06.494371+0000","mapping_epoch":35,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3661824,"num_objects":894,"num_object_clones":0,"num_object_copies":1788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16779,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19734","reported_seq":27872,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:02.020683+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:02.020683+0000","last_peered":"2024-09-10T16:02:02.020683+0000","last_clean":"2024-09-10T16:02:02.020683+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:02.020683+0000","last_undegraded":"2024-09-10T16:02:02.020683+0000","last_fullsized":"2024-09-10T16:02:02.020683+0000","mapping_epoch":45,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4034560,"num_objects":985,"num_object_clones":0,"num_object_copies":1970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17711,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19655","reported_seq":20478,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.482830+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:06.482830+0000","last_peered":"2024-09-10T16:02:06.482830+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:06.482830+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3780608,"num_objects":923,"num_object_clones":0,"num_object_copies":1846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16519,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2596}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19918","reported_seq":27728,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:06.058908+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:06.058908+0000","last_peered":"2024-09-10T16:02:06.058908+0000","last_clean":"2024-09-10T16:02:06.058908+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:06.058908+0000","last_undegraded":"2024-09-10T16:02:06.058908+0000","last_fullsized":"2024-09-10T16:02:06.058908+0000","mapping_epoch":49,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3551232,"num_objects":867,"num_object_clones":0,"num_object_copies":1734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17892,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19597","reported_seq":20020,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.080108+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:06.080108+0000","last_peered":"2024-09-10T16:02:06.080108+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:06.080108+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3518464,"num_objects":859,"num_object_clones":0,"num_object_copies":1718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17466,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2827}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19980","reported_seq":20828,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.481530+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:06.481530+0000","last_peered":"2024-09-10T16:02:06.481530+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:06.481530+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3579904,"num_objects":874,"num_object_clones":0,"num_object_copies":1748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17172,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2604}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19851","reported_seq":20607,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.463041+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:06.463041+0000","last_peered":"2024-09-10T16:02:06.463041+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:06.463041+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3551232,"num_objects":867,"num_object_clones":0,"num_object_copies":1734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16269,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19908","reported_seq":20739,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.544032+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:06.544032+0000","last_peered":"2024-09-10T16:02:06.544032+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:06.544032+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3514368,"num_objects":858,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16514,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2678}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20234","reported_seq":21001,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.494155+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:06.494155+0000","last_peered":"2024-09-10T16:02:06.494155+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:06.494155+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3539427,"num_objects":865,"num_object_clones":0,"num_object_copies":1730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17053,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":58831355,"num_objects":14365,"num_object_clones":0,"num_object_copies":28730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30857,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":278685,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15791,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":313253888,"data_stored":313177097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160864,"ondisk_log_size":160864,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626541,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518844,"kb_used_data":49900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93852996,"statfs":{"total":96636764160,"available":96105467904,"internally_reserved":0,"allocated":51097600,"data_stored":50748727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480177838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,2,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149824,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570732,"kb_used_data":105928,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93801108,"statfs":{"total":96636764160,"available":96052334592,"internally_reserved":0,"allocated":108470272,"data_stored":108183279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953595,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518388,"kb_used_data":153560,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93853452,"statfs":{"total":96636764160,"available":96105934848,"internally_reserved":0,"allocated":157245440,"data_stored":156954375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,0,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106811392,"data_stored":106807779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49807360,"data_stored":49741867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":156635136,"data_stored":156627451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:10.455 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:10.987 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:10.989 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:11.007 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-10T16:02:10.633247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":57623054,"num_objects":14072,"num_object_clones":0,"num_object_copies":28144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30857,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":278982,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15793,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1592096,"kb_used_data":301840,"kb_used_omap":12,"kb_used_meta":1290163,"kb_avail":281523424,"statfs":{"total":289910292480,"available":288279986176,"internally_reserved":0,"allocated":309084160,"data_stored":308157229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1321126917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,8,0,6,12,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20516864,"num_objects":-5009,"num_object_clones":0,"num_object_copies":-10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5009,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004660"},"pg_stats":[{"pgid":"2.f","version":"50'19889","reported_seq":23625,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.503541+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:06.503541+0000","last_peered":"2024-09-10T16:02:06.503541+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:06.503541+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3608576,"num_objects":881,"num_object_clones":0,"num_object_copies":1762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19889,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1824,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1268}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20178","reported_seq":20296,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:05.929633+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:05.929633+0000","last_peered":"2024-09-10T16:02:05.929633+0000","last_clean":"2024-09-10T16:02:05.929633+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:05.929633+0000","last_undegraded":"2024-09-10T16:02:05.929633+0000","last_fullsized":"2024-09-10T16:02:05.929633+0000","mapping_epoch":36,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":896,"num_object_clones":0,"num_object_copies":1792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18514,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19748","reported_seq":20229,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.052921+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:06.052921+0000","last_peered":"2024-09-10T16:02:06.052921+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:06.052921+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3608600,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18016,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2747}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19757","reported_seq":19847,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:06.041194+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:06.041194+0000","last_peered":"2024-09-10T16:02:06.041194+0000","last_clean":"2024-09-10T16:02:06.041194+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:06.041194+0000","last_undegraded":"2024-09-10T16:02:06.041194+0000","last_fullsized":"2024-09-10T16:02:06.041194+0000","mapping_epoch":36,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3788800,"num_objects":925,"num_object_clones":0,"num_object_copies":1850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19757,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19446","reported_seq":19583,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:06.459263+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:06.459263+0000","last_peered":"2024-09-10T16:02:06.459263+0000","last_clean":"2024-09-10T16:02:06.459263+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:06.459263+0000","last_undegraded":"2024-09-10T16:02:06.459263+0000","last_fullsized":"2024-09-10T16:02:06.459263+0000","mapping_epoch":35,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3624960,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16517,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19991","reported_seq":27889,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.070015+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:07.070015+0000","last_peered":"2024-09-10T16:02:07.070015+0000","last_clean":"2024-09-10T16:02:07.070015+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:07.070015+0000","last_undegraded":"2024-09-10T16:02:07.070015+0000","last_fullsized":"2024-09-10T16:02:07.070015+0000","mapping_epoch":47,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3588096,"num_objects":876,"num_object_clones":0,"num_object_copies":1752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16709,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19381","reported_seq":20152,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.509741+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:06.509741+0000","last_peered":"2024-09-10T16:02:06.509741+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:06.509741+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3706880,"num_objects":905,"num_object_clones":0,"num_object_copies":1810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16031,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2619}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20020","reported_seq":20161,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:06.494371+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:06.494371+0000","last_peered":"2024-09-10T16:02:06.494371+0000","last_clean":"2024-09-10T16:02:06.494371+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:06.494371+0000","last_undegraded":"2024-09-10T16:02:06.494371+0000","last_fullsized":"2024-09-10T16:02:06.494371+0000","mapping_epoch":35,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3661824,"num_objects":894,"num_object_clones":0,"num_object_copies":1788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16779,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19906","reported_seq":28044,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.133077+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:07.133077+0000","last_peered":"2024-09-10T16:02:07.133077+0000","last_clean":"2024-09-10T16:02:07.133077+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:07.133077+0000","last_undegraded":"2024-09-10T16:02:07.133077+0000","last_fullsized":"2024-09-10T16:02:07.133077+0000","mapping_epoch":45,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3330048,"num_objects":813,"num_object_clones":0,"num_object_copies":1626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17883,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19655","reported_seq":20478,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.482830+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:06.482830+0000","last_peered":"2024-09-10T16:02:06.482830+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:06.482830+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3780608,"num_objects":923,"num_object_clones":0,"num_object_copies":1846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16519,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2596}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19918","reported_seq":27728,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:06.058908+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:06.058908+0000","last_peered":"2024-09-10T16:02:06.058908+0000","last_clean":"2024-09-10T16:02:06.058908+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:06.058908+0000","last_undegraded":"2024-09-10T16:02:06.058908+0000","last_fullsized":"2024-09-10T16:02:06.058908+0000","mapping_epoch":49,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3551232,"num_objects":867,"num_object_clones":0,"num_object_copies":1734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17892,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19597","reported_seq":20020,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.080108+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:06.080108+0000","last_peered":"2024-09-10T16:02:06.080108+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:06.080108+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3518464,"num_objects":859,"num_object_clones":0,"num_object_copies":1718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17466,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2827}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19980","reported_seq":20828,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.481530+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:06.481530+0000","last_peered":"2024-09-10T16:02:06.481530+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:06.481530+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3579904,"num_objects":874,"num_object_clones":0,"num_object_copies":1748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17172,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2604}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19851","reported_seq":20607,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.463041+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:06.463041+0000","last_peered":"2024-09-10T16:02:06.463041+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:06.463041+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3551232,"num_objects":867,"num_object_clones":0,"num_object_copies":1734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16269,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19908","reported_seq":20739,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.544032+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:06.544032+0000","last_peered":"2024-09-10T16:02:06.544032+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:06.544032+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3514368,"num_objects":858,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16514,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2678}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20234","reported_seq":21001,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.494155+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:06.494155+0000","last_peered":"2024-09-10T16:02:06.494155+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:06.494155+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3539427,"num_objects":865,"num_object_clones":0,"num_object_copies":1730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17053,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":57623035,"num_objects":14070,"num_object_clones":0,"num_object_copies":28140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30857,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":278980,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15791,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":305475584,"data_stored":305398793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160859,"ondisk_log_size":160859,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626543,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502976,"kb_used_data":42352,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93868864,"statfs":{"total":96636764160,"available":96121716736,"internally_reserved":0,"allocated":43368448,"data_stored":43019575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471658158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149824,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570732,"kb_used_data":105928,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93801108,"statfs":{"total":96636764160,"available":96052334592,"internally_reserved":0,"allocated":108470272,"data_stored":108183279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953595,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518388,"kb_used_data":153560,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93853452,"statfs":{"total":96636764160,"available":96105934848,"internally_reserved":0,"allocated":157245440,"data_stored":156954375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,0,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106811392,"data_stored":106807779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":42029056,"data_stored":41963563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":156635136,"data_stored":156627451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:11.009 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:11.558 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:11.560 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-10T16:02:10.633247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":57623054,"num_objects":14072,"num_object_clones":0,"num_object_copies":28144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30857,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":278982,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15793,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1592096,"kb_used_data":301840,"kb_used_omap":12,"kb_used_meta":1290163,"kb_avail":281523424,"statfs":{"total":289910292480,"available":288279986176,"internally_reserved":0,"allocated":309084160,"data_stored":308157229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1321126917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,8,0,6,12,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20516864,"num_objects":-5009,"num_object_clones":0,"num_object_copies":-10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5009,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004660"},"pg_stats":[{"pgid":"2.f","version":"50'19889","reported_seq":23625,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.503541+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:06.503541+0000","last_peered":"2024-09-10T16:02:06.503541+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:06.503541+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3608576,"num_objects":881,"num_object_clones":0,"num_object_copies":1762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19889,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1824,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1268}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20178","reported_seq":20296,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:05.929633+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:05.929633+0000","last_peered":"2024-09-10T16:02:05.929633+0000","last_clean":"2024-09-10T16:02:05.929633+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:05.929633+0000","last_undegraded":"2024-09-10T16:02:05.929633+0000","last_fullsized":"2024-09-10T16:02:05.929633+0000","mapping_epoch":36,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":896,"num_object_clones":0,"num_object_copies":1792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18514,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19748","reported_seq":20229,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.052921+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:06.052921+0000","last_peered":"2024-09-10T16:02:06.052921+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:06.052921+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3608600,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18016,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2747}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19757","reported_seq":19847,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:06.041194+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:06.041194+0000","last_peered":"2024-09-10T16:02:06.041194+0000","last_clean":"2024-09-10T16:02:06.041194+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:06.041194+0000","last_undegraded":"2024-09-10T16:02:06.041194+0000","last_fullsized":"2024-09-10T16:02:06.041194+0000","mapping_epoch":36,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3788800,"num_objects":925,"num_object_clones":0,"num_object_copies":1850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19757,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19446","reported_seq":19583,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:06.459263+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:06.459263+0000","last_peered":"2024-09-10T16:02:06.459263+0000","last_clean":"2024-09-10T16:02:06.459263+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:06.459263+0000","last_undegraded":"2024-09-10T16:02:06.459263+0000","last_fullsized":"2024-09-10T16:02:06.459263+0000","mapping_epoch":35,"log_start":"46'9400","ondisk_log_start":"46'9400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3624960,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16517,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'19991","reported_seq":27889,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.070015+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:07.070015+0000","last_peered":"2024-09-10T16:02:07.070015+0000","last_clean":"2024-09-10T16:02:07.070015+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:07.070015+0000","last_undegraded":"2024-09-10T16:02:07.070015+0000","last_fullsized":"2024-09-10T16:02:07.070015+0000","mapping_epoch":47,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3588096,"num_objects":876,"num_object_clones":0,"num_object_copies":1752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16709,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19381","reported_seq":20152,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.509741+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:06.509741+0000","last_peered":"2024-09-10T16:02:06.509741+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:06.509741+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9300","ondisk_log_start":"46'9300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3706880,"num_objects":905,"num_object_clones":0,"num_object_copies":1810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16031,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2619}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20020","reported_seq":20161,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:06.494371+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:06.494371+0000","last_peered":"2024-09-10T16:02:06.494371+0000","last_clean":"2024-09-10T16:02:06.494371+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:06.494371+0000","last_undegraded":"2024-09-10T16:02:06.494371+0000","last_fullsized":"2024-09-10T16:02:06.494371+0000","mapping_epoch":35,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3661824,"num_objects":894,"num_object_clones":0,"num_object_copies":1788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16779,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'19906","reported_seq":28044,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:07.133077+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:07.133077+0000","last_peered":"2024-09-10T16:02:07.133077+0000","last_clean":"2024-09-10T16:02:07.133077+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:07.133077+0000","last_undegraded":"2024-09-10T16:02:07.133077+0000","last_fullsized":"2024-09-10T16:02:07.133077+0000","mapping_epoch":45,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3330048,"num_objects":813,"num_object_clones":0,"num_object_copies":1626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17883,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19655","reported_seq":20478,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.482830+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:06.482830+0000","last_peered":"2024-09-10T16:02:06.482830+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:06.482830+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3780608,"num_objects":923,"num_object_clones":0,"num_object_copies":1846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16519,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2596}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'19918","reported_seq":27728,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:06.058908+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:06.058908+0000","last_peered":"2024-09-10T16:02:06.058908+0000","last_clean":"2024-09-10T16:02:06.058908+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:06.058908+0000","last_undegraded":"2024-09-10T16:02:06.058908+0000","last_fullsized":"2024-09-10T16:02:06.058908+0000","mapping_epoch":49,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3551232,"num_objects":867,"num_object_clones":0,"num_object_copies":1734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17892,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19597","reported_seq":20020,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.080108+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:06.080108+0000","last_peered":"2024-09-10T16:02:06.080108+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:06.080108+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3518464,"num_objects":859,"num_object_clones":0,"num_object_copies":1718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17466,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2827}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'19980","reported_seq":20828,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.481530+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:06.481530+0000","last_peered":"2024-09-10T16:02:06.481530+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:06.481530+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3579904,"num_objects":874,"num_object_clones":0,"num_object_copies":1748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17172,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2604}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'19851","reported_seq":20607,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.463041+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:06.463041+0000","last_peered":"2024-09-10T16:02:06.463041+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:06.463041+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3551232,"num_objects":867,"num_object_clones":0,"num_object_copies":1734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16269,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'19908","reported_seq":20739,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.544032+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:06.544032+0000","last_peered":"2024-09-10T16:02:06.544032+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:06.544032+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3514368,"num_objects":858,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16514,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2678}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20234","reported_seq":21001,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:06.494155+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:06.494155+0000","last_peered":"2024-09-10T16:02:06.494155+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:06.494155+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3539427,"num_objects":865,"num_object_clones":0,"num_object_copies":1730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17053,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":57623035,"num_objects":14070,"num_object_clones":0,"num_object_copies":28140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30857,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":278980,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15791,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":305475584,"data_stored":305398793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160859,"ondisk_log_size":160859,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626543,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502976,"kb_used_data":42352,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93868864,"statfs":{"total":96636764160,"available":96121716736,"internally_reserved":0,"allocated":43368448,"data_stored":43019575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471658158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149824,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570732,"kb_used_data":105928,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93801108,"statfs":{"total":96636764160,"available":96052334592,"internally_reserved":0,"allocated":108470272,"data_stored":108183279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953595,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518388,"kb_used_data":153560,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93853452,"statfs":{"total":96636764160,"available":96105934848,"internally_reserved":0,"allocated":157245440,"data_stored":156954375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,0,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106811392,"data_stored":106807779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":42029056,"data_stored":41963563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":156635136,"data_stored":156627451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:11.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:11.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'19889", 'reported_seq': 23625, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:06.503541+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:02:06.503541+0000', 'last_peered': '2024-09-10T16:02:06.503541+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:02:06.503541+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'9800", 'ondisk_log_start': "46'9800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3608576, 'num_objects': 881, 'num_object_clones': 0, 'num_object_copies': 1762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 881, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19889, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1824, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1268}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'19748", 'reported_seq': 20229, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:06.052921+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:06.052921+0000', 'last_peered': '2024-09-10T16:02:06.052921+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:06.052921+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'9700", 'ondisk_log_start': "46'9700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3608600, 'num_objects': 882, 'num_object_clones': 0, 'num_object_copies': 1764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 882, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18016, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2747}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'19381", 'reported_seq': 20152, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:06.509741+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:06.509741+0000', 'last_peered': '2024-09-10T16:02:06.509741+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:06.509741+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'9300", 'ondisk_log_start': "46'9300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3706880, 'num_objects': 905, 'num_object_clones': 0, 'num_object_copies': 1810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16031, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2619}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'19655", 'reported_seq': 20478, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:06.482830+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:06.482830+0000', 'last_peered': '2024-09-10T16:02:06.482830+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:06.482830+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'9600", 'ondisk_log_start': "46'9600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3780608, 'num_objects': 923, 'num_object_clones': 0, 'num_object_copies': 1846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 923, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16519, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2596}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'19597", 'reported_seq': 20020, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:06.080108+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:02:06.080108+0000', 'last_peered': '2024-09-10T16:02:06.080108+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:02:06.080108+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'9500", 'ondisk_log_start': "46'9500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3518464, 'num_objects': 859, 'num_object_clones': 0, 'num_object_copies': 1718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 859, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17466, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2827}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'19980", 'reported_seq': 20828, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:06.481530+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:06.481530+0000', 'last_peered': '2024-09-10T16:02:06.481530+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:06.481530+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'9900", 'ondisk_log_start': "46'9900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3579904, 'num_objects': 874, 'num_object_clones': 0, 'num_object_copies': 1748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 874, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17172, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2604}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:11.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:11.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'19851", 'reported_seq': 20607, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:06.463041+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:06.463041+0000', 'last_peered': '2024-09-10T16:02:06.463041+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:06.463041+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'9800", 'ondisk_log_start': "46'9800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3551232, 'num_objects': 867, 'num_object_clones': 0, 'num_object_copies': 1734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 867, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16269, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2759}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:11.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:11.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'19908", 'reported_seq': 20739, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:06.544032+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:06.544032+0000', 'last_peered': '2024-09-10T16:02:06.544032+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:06.544032+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'9900", 'ondisk_log_start': "46'9900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3514368, 'num_objects': 858, 'num_object_clones': 0, 'num_object_copies': 1716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 858, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16514, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2678}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:11.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:11.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'20234", 'reported_seq': 21001, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:06.494155+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:06.494155+0000', 'last_peered': '2024-09-10T16:02:06.494155+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:06.494155+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3539427, 'num_objects': 865, 'num_object_clones': 0, 'num_object_copies': 1730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17053, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2829}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:11.585 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:12.284 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:12.284 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30857/28144 objects degraded (109.640%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":658,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":14072,"data_bytes":57623054,"bytes_used":1630306304,"bytes_avail":288279986176,"bytes_total":289910292480,"degraded_objects":30857,"degraded_total":28144,"degraded_ratio":1.0963971006253552,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":147,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":417},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:12.308 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:12.308 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:12.861 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:12.863 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:12.882 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-10T16:02:12.633976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":47264270,"num_objects":11543,"num_object_clones":0,"num_object_copies":23086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281511,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15883,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160791,"ondisk_log_size":160791,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1571220,"kb_used_data":280996,"kb_used_omap":12,"kb_used_meta":1290163,"kb_avail":281544300,"statfs":{"total":289910292480,"available":288301363200,"internally_reserved":0,"allocated":287739904,"data_stored":286812973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1321126917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,6,5,9,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30875648,"num_objects":-7538,"num_object_clones":0,"num_object_copies":-15076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7538,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004657"},"pg_stats":[{"pgid":"2.f","version":"50'20046","reported_seq":23962,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.586084+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:11.586084+0000","last_peered":"2024-09-10T16:02:11.586084+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:11.586084+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2965504,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20046,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1914,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1335}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20332","reported_seq":20449,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.071572+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:11.071572+0000","last_peered":"2024-09-10T16:02:11.071572+0000","last_clean":"2024-09-10T16:02:11.071572+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:11.071572+0000","last_undegraded":"2024-09-10T16:02:11.071572+0000","last_fullsized":"2024-09-10T16:02:11.071572+0000","mapping_epoch":36,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3039232,"num_objects":742,"num_object_clones":0,"num_object_copies":1484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18668,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19910","reported_seq":20391,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.006491+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:11.006491+0000","last_peered":"2024-09-10T16:02:11.006491+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:11.006491+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2945048,"num_objects":720,"num_object_clones":0,"num_object_copies":1440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18178,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2909}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19921","reported_seq":20010,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.102367+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:11.102367+0000","last_peered":"2024-09-10T16:02:11.102367+0000","last_clean":"2024-09-10T16:02:11.102367+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:11.102367+0000","last_undegraded":"2024-09-10T16:02:11.102367+0000","last_fullsized":"2024-09-10T16:02:11.102367+0000","mapping_epoch":36,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3117056,"num_objects":761,"num_object_clones":0,"num_object_copies":1522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19921,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19608","reported_seq":19745,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.586861+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:11.586861+0000","last_peered":"2024-09-10T16:02:11.586861+0000","last_clean":"2024-09-10T16:02:11.586861+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:11.586861+0000","last_undegraded":"2024-09-10T16:02:11.586861+0000","last_fullsized":"2024-09-10T16:02:11.586861+0000","mapping_epoch":35,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2961408,"num_objects":723,"num_object_clones":0,"num_object_copies":1446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16679,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20137","reported_seq":28035,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.206407+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:12.206407+0000","last_peered":"2024-09-10T16:02:12.206407+0000","last_clean":"2024-09-10T16:02:12.206407+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:12.206407+0000","last_undegraded":"2024-09-10T16:02:12.206407+0000","last_fullsized":"2024-09-10T16:02:12.206407+0000","mapping_epoch":47,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2990080,"num_objects":730,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16855,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19547","reported_seq":20315,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.478105+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:11.478105+0000","last_peered":"2024-09-10T16:02:11.478105+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:11.478105+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3026944,"num_objects":739,"num_object_clones":0,"num_object_copies":1478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16197,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20182","reported_seq":20324,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.527415+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:11.527415+0000","last_peered":"2024-09-10T16:02:11.527415+0000","last_clean":"2024-09-10T16:02:11.527415+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:11.527415+0000","last_undegraded":"2024-09-10T16:02:11.527415+0000","last_fullsized":"2024-09-10T16:02:11.527415+0000","mapping_epoch":35,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2998272,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16941,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20069","reported_seq":28207,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.128416+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:12.128416+0000","last_peered":"2024-09-10T16:02:12.128416+0000","last_clean":"2024-09-10T16:02:12.128416+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:12.128416+0000","last_undegraded":"2024-09-10T16:02:12.128416+0000","last_fullsized":"2024-09-10T16:02:12.128416+0000","mapping_epoch":45,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2662400,"num_objects":650,"num_object_clones":0,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19817","reported_seq":20639,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.598889+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:11.598889+0000","last_peered":"2024-09-10T16:02:11.598889+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:11.598889+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3117056,"num_objects":761,"num_object_clones":0,"num_object_copies":1522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16681,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20080","reported_seq":27890,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.065090+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:11.065090+0000","last_peered":"2024-09-10T16:02:11.065090+0000","last_clean":"2024-09-10T16:02:11.065090+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:11.065090+0000","last_undegraded":"2024-09-10T16:02:11.065090+0000","last_fullsized":"2024-09-10T16:02:11.065090+0000","mapping_epoch":49,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2887680,"num_objects":705,"num_object_clones":0,"num_object_copies":1410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18054,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19749","reported_seq":20171,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.087362+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:11.087362+0000","last_peered":"2024-09-10T16:02:11.087362+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:11.087362+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2895872,"num_objects":707,"num_object_clones":0,"num_object_copies":1414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17618,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2979}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20133","reported_seq":20981,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.553526+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:11.553526+0000","last_peered":"2024-09-10T16:02:11.553526+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:11.553526+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2953216,"num_objects":721,"num_object_clones":0,"num_object_copies":1442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17325,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20000","reported_seq":20756,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.539647+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:11.539647+0000","last_peered":"2024-09-10T16:02:11.539647+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:11.539647+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2940928,"num_objects":718,"num_object_clones":0,"num_object_copies":1436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16418,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20074","reported_seq":20906,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.569440+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:11.569440+0000","last_peered":"2024-09-10T16:02:11.569440+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:11.569440+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2834432,"num_objects":692,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16680,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20383","reported_seq":21150,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.532339+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:11.532339+0000","last_peered":"2024-09-10T16:02:11.532339+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:11.532339+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2929123,"num_objects":716,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17202,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":47264251,"num_objects":11541,"num_object_clones":0,"num_object_copies":23082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11541,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281509,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15881,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":284688384,"data_stored":284611593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626544,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499808,"kb_used_data":35024,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93872032,"statfs":{"total":96636764160,"available":96124960768,"internally_reserved":0,"allocated":35864576,"data_stored":35515703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475917998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149825,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570476,"kb_used_data":97372,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93801364,"statfs":{"total":96636764160,"available":96052596736,"internally_reserved":0,"allocated":99708928,"data_stored":99421935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":484437677},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953596,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500936,"kb_used_data":148600,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93870904,"statfs":{"total":96636764160,"available":96123805696,"internally_reserved":0,"allocated":152166400,"data_stored":151875335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":99061760,"data_stored":99058147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":34164736,"data_stored":34099243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151461888,"data_stored":151454203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:15.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:16.449 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:16.450 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:16.475 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-09-10T16:02:14.634721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":47264270,"num_objects":11543,"num_object_clones":0,"num_object_copies":23086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281511,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15883,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160791,"ondisk_log_size":160791,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1571220,"kb_used_data":280996,"kb_used_omap":12,"kb_used_meta":1290163,"kb_avail":281544300,"statfs":{"total":289910292480,"available":288301363200,"internally_reserved":0,"allocated":287739904,"data_stored":286812973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1321126917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,6,5,9,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20574208,"num_objects":-5023,"num_object_clones":0,"num_object_copies":-10046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5023,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004633"},"pg_stats":[{"pgid":"2.f","version":"50'20046","reported_seq":23962,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.586084+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:11.586084+0000","last_peered":"2024-09-10T16:02:11.586084+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:11.586084+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2965504,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20046,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1914,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1335}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20332","reported_seq":20449,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.071572+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:11.071572+0000","last_peered":"2024-09-10T16:02:11.071572+0000","last_clean":"2024-09-10T16:02:11.071572+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:11.071572+0000","last_undegraded":"2024-09-10T16:02:11.071572+0000","last_fullsized":"2024-09-10T16:02:11.071572+0000","mapping_epoch":36,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3039232,"num_objects":742,"num_object_clones":0,"num_object_copies":1484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18668,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'19910","reported_seq":20391,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.006491+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:11.006491+0000","last_peered":"2024-09-10T16:02:11.006491+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:11.006491+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2945048,"num_objects":720,"num_object_clones":0,"num_object_copies":1440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18178,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2909}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'19921","reported_seq":20010,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.102367+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:11.102367+0000","last_peered":"2024-09-10T16:02:11.102367+0000","last_clean":"2024-09-10T16:02:11.102367+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:11.102367+0000","last_undegraded":"2024-09-10T16:02:11.102367+0000","last_fullsized":"2024-09-10T16:02:11.102367+0000","mapping_epoch":36,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3117056,"num_objects":761,"num_object_clones":0,"num_object_copies":1522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19921,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19608","reported_seq":19745,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.586861+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:11.586861+0000","last_peered":"2024-09-10T16:02:11.586861+0000","last_clean":"2024-09-10T16:02:11.586861+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:11.586861+0000","last_undegraded":"2024-09-10T16:02:11.586861+0000","last_fullsized":"2024-09-10T16:02:11.586861+0000","mapping_epoch":35,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2961408,"num_objects":723,"num_object_clones":0,"num_object_copies":1446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16679,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20137","reported_seq":28035,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.206407+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:12.206407+0000","last_peered":"2024-09-10T16:02:12.206407+0000","last_clean":"2024-09-10T16:02:12.206407+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:12.206407+0000","last_undegraded":"2024-09-10T16:02:12.206407+0000","last_fullsized":"2024-09-10T16:02:12.206407+0000","mapping_epoch":47,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2990080,"num_objects":730,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16855,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19547","reported_seq":20315,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.478105+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:11.478105+0000","last_peered":"2024-09-10T16:02:11.478105+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:11.478105+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3026944,"num_objects":739,"num_object_clones":0,"num_object_copies":1478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16197,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20182","reported_seq":20324,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.527415+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:11.527415+0000","last_peered":"2024-09-10T16:02:11.527415+0000","last_clean":"2024-09-10T16:02:11.527415+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:11.527415+0000","last_undegraded":"2024-09-10T16:02:11.527415+0000","last_fullsized":"2024-09-10T16:02:11.527415+0000","mapping_epoch":35,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2998272,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16941,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20069","reported_seq":28207,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.128416+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:12.128416+0000","last_peered":"2024-09-10T16:02:12.128416+0000","last_clean":"2024-09-10T16:02:12.128416+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:12.128416+0000","last_undegraded":"2024-09-10T16:02:12.128416+0000","last_fullsized":"2024-09-10T16:02:12.128416+0000","mapping_epoch":45,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2662400,"num_objects":650,"num_object_clones":0,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19817","reported_seq":20639,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.598889+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:11.598889+0000","last_peered":"2024-09-10T16:02:11.598889+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:11.598889+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3117056,"num_objects":761,"num_object_clones":0,"num_object_copies":1522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16681,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20080","reported_seq":27890,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.065090+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:11.065090+0000","last_peered":"2024-09-10T16:02:11.065090+0000","last_clean":"2024-09-10T16:02:11.065090+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:11.065090+0000","last_undegraded":"2024-09-10T16:02:11.065090+0000","last_fullsized":"2024-09-10T16:02:11.065090+0000","mapping_epoch":49,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2887680,"num_objects":705,"num_object_clones":0,"num_object_copies":1410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18054,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19749","reported_seq":20171,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.087362+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:11.087362+0000","last_peered":"2024-09-10T16:02:11.087362+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:11.087362+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2895872,"num_objects":707,"num_object_clones":0,"num_object_copies":1414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17618,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2979}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20133","reported_seq":20981,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.553526+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:11.553526+0000","last_peered":"2024-09-10T16:02:11.553526+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:11.553526+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2953216,"num_objects":721,"num_object_clones":0,"num_object_copies":1442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17325,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20000","reported_seq":20756,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.539647+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:11.539647+0000","last_peered":"2024-09-10T16:02:11.539647+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:11.539647+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2940928,"num_objects":718,"num_object_clones":0,"num_object_copies":1436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16418,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20074","reported_seq":20906,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.569440+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:11.569440+0000","last_peered":"2024-09-10T16:02:11.569440+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:11.569440+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2834432,"num_objects":692,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16680,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20383","reported_seq":21150,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.532339+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:11.532339+0000","last_peered":"2024-09-10T16:02:11.532339+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:11.532339+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2929123,"num_objects":716,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17202,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":47264251,"num_objects":11541,"num_object_clones":0,"num_object_copies":23082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11541,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281509,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15881,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":284688384,"data_stored":284611593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626544,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499808,"kb_used_data":35024,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93872032,"statfs":{"total":96636764160,"available":96124960768,"internally_reserved":0,"allocated":35864576,"data_stored":35515703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475917998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149825,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570476,"kb_used_data":97372,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93801364,"statfs":{"total":96636764160,"available":96052596736,"internally_reserved":0,"allocated":99708928,"data_stored":99421935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":484437677},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953596,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500936,"kb_used_data":148600,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93870904,"statfs":{"total":96636764160,"available":96123805696,"internally_reserved":0,"allocated":152166400,"data_stored":151875335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":99061760,"data_stored":99058147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":34164736,"data_stored":34099243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151461888,"data_stored":151454203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:16.476 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:17.026 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:17.029 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:17.050 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-10T16:02:16.635310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":44020238,"num_objects":10751,"num_object_clones":0,"num_object_copies":21502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10751,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282303,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15883,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160883,"ondisk_log_size":160883,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1570252,"kb_used_data":275868,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281545268,"statfs":{"total":289910292480,"available":288302354432,"internally_reserved":0,"allocated":282488832,"data_stored":281561901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1325386757},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,6,11,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23818240,"num_objects":-5815,"num_object_clones":0,"num_object_copies":-11630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5815,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004443"},"pg_stats":[{"pgid":"2.f","version":"50'20046","reported_seq":23962,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.586084+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:11.586084+0000","last_peered":"2024-09-10T16:02:11.586084+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:11.586084+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2965504,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20046,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1914,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1335}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20491","reported_seq":20609,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:16.016613+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:16.016613+0000","last_peered":"2024-09-10T16:02:16.016613+0000","last_clean":"2024-09-10T16:02:16.016613+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:16.016613+0000","last_undegraded":"2024-09-10T16:02:16.016613+0000","last_fullsized":"2024-09-10T16:02:16.016613+0000","mapping_epoch":36,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2387968,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18827,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20062","reported_seq":20543,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.031129+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:16.031129+0000","last_peered":"2024-09-10T16:02:16.031129+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:16.031129+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2322456,"num_objects":568,"num_object_clones":0,"num_object_copies":1136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":568,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18330,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3061}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20091","reported_seq":20181,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:15.983436+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:15.983436+0000","last_peered":"2024-09-10T16:02:15.983436+0000","last_clean":"2024-09-10T16:02:15.983436+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:15.983436+0000","last_undegraded":"2024-09-10T16:02:15.983436+0000","last_fullsized":"2024-09-10T16:02:15.983436+0000","mapping_epoch":36,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2420736,"num_objects":591,"num_object_clones":0,"num_object_copies":1182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20091,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19608","reported_seq":19745,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.586861+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:11.586861+0000","last_peered":"2024-09-10T16:02:11.586861+0000","last_clean":"2024-09-10T16:02:11.586861+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:11.586861+0000","last_undegraded":"2024-09-10T16:02:11.586861+0000","last_fullsized":"2024-09-10T16:02:11.586861+0000","mapping_epoch":35,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2961408,"num_objects":723,"num_object_clones":0,"num_object_copies":1446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16679,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20137","reported_seq":28035,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.206407+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:12.206407+0000","last_peered":"2024-09-10T16:02:12.206407+0000","last_clean":"2024-09-10T16:02:12.206407+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:12.206407+0000","last_undegraded":"2024-09-10T16:02:12.206407+0000","last_fullsized":"2024-09-10T16:02:12.206407+0000","mapping_epoch":47,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2990080,"num_objects":730,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16855,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19547","reported_seq":20315,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.478105+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:11.478105+0000","last_peered":"2024-09-10T16:02:11.478105+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:11.478105+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3026944,"num_objects":739,"num_object_clones":0,"num_object_copies":1478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16197,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20182","reported_seq":20324,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.527415+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:11.527415+0000","last_peered":"2024-09-10T16:02:11.527415+0000","last_clean":"2024-09-10T16:02:11.527415+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:11.527415+0000","last_undegraded":"2024-09-10T16:02:11.527415+0000","last_fullsized":"2024-09-10T16:02:11.527415+0000","mapping_epoch":35,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2998272,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16941,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20069","reported_seq":28207,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.128416+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:12.128416+0000","last_peered":"2024-09-10T16:02:12.128416+0000","last_clean":"2024-09-10T16:02:12.128416+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:12.128416+0000","last_undegraded":"2024-09-10T16:02:12.128416+0000","last_fullsized":"2024-09-10T16:02:12.128416+0000","mapping_epoch":45,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2662400,"num_objects":650,"num_object_clones":0,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19817","reported_seq":20639,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.598889+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:11.598889+0000","last_peered":"2024-09-10T16:02:11.598889+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:11.598889+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3117056,"num_objects":761,"num_object_clones":0,"num_object_copies":1522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16681,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20228","reported_seq":28038,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:16.082959+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:16.082959+0000","last_peered":"2024-09-10T16:02:16.082959+0000","last_clean":"2024-09-10T16:02:16.082959+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:16.082959+0000","last_undegraded":"2024-09-10T16:02:16.082959+0000","last_fullsized":"2024-09-10T16:02:16.082959+0000","mapping_epoch":49,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2281472,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18202,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19912","reported_seq":20334,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.091424+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:16.091424+0000","last_peered":"2024-09-10T16:02:16.091424+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:16.091424+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":544,"num_object_clones":0,"num_object_copies":1088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17781,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3142}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20133","reported_seq":20981,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.553526+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:11.553526+0000","last_peered":"2024-09-10T16:02:11.553526+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:11.553526+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2953216,"num_objects":721,"num_object_clones":0,"num_object_copies":1442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17325,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20000","reported_seq":20756,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.539647+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:11.539647+0000","last_peered":"2024-09-10T16:02:11.539647+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:11.539647+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2940928,"num_objects":718,"num_object_clones":0,"num_object_copies":1436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16418,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20074","reported_seq":20906,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.569440+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:11.569440+0000","last_peered":"2024-09-10T16:02:11.569440+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:11.569440+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2834432,"num_objects":692,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16680,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20383","reported_seq":21150,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.532339+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:11.532339+0000","last_peered":"2024-09-10T16:02:11.532339+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:11.532339+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2929123,"num_objects":716,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17202,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":44020219,"num_objects":10749,"num_object_clones":0,"num_object_copies":21498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10749,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282301,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15881,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":279408640,"data_stored":279331849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160880,"ondisk_log_size":160880,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626544,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499808,"kb_used_data":35024,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93872032,"statfs":{"total":96636764160,"available":96124960768,"internally_reserved":0,"allocated":35864576,"data_stored":35515703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475917998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149825,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570476,"kb_used_data":97372,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93801364,"statfs":{"total":96636764160,"available":96052596736,"internally_reserved":0,"allocated":99708928,"data_stored":99421935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":484437677},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953597,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499968,"kb_used_data":143472,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93871872,"statfs":{"total":96636764160,"available":96124796928,"internally_reserved":0,"allocated":146915328,"data_stored":146624263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":99061760,"data_stored":99058147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":34164736,"data_stored":34099243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146182144,"data_stored":146174459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:17.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:17.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'20046", 'reported_seq': 23962, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:11.586084+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:02:11.586084+0000', 'last_peered': '2024-09-10T16:02:11.586084+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:02:11.586084+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'10000", 'ondisk_log_start': "46'10000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2965504, 'num_objects': 724, 'num_object_clones': 0, 'num_object_copies': 1448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2059, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 724, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20046, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1914, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1335}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:17.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:17.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20062", 'reported_seq': 20543, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:16.031129+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:16.031129+0000', 'last_peered': '2024-09-10T16:02:16.031129+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:16.031129+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10000", 'ondisk_log_start': "46'10000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2322456, 'num_objects': 568, 'num_object_clones': 0, 'num_object_copies': 1136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 568, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18330, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3061}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:17.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:17.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'19547", 'reported_seq': 20315, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:11.478105+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:11.478105+0000', 'last_peered': '2024-09-10T16:02:11.478105+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:11.478105+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'9500", 'ondisk_log_start': "46'9500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3026944, 'num_objects': 739, 'num_object_clones': 0, 'num_object_copies': 1478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 739, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16197, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2785}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:17.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:17.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'19817", 'reported_seq': 20639, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:11.598889+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:11.598889+0000', 'last_peered': '2024-09-10T16:02:11.598889+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:11.598889+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'9800", 'ondisk_log_start': "46'9800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3117056, 'num_objects': 761, 'num_object_clones': 0, 'num_object_copies': 1522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 761, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16681, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2758}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:17.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:17.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'19912", 'reported_seq': 20334, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:16.091424+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:02:16.091424+0000', 'last_peered': '2024-09-10T16:02:16.091424+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:02:16.091424+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'9900", 'ondisk_log_start': "46'9900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2228224, 'num_objects': 544, 'num_object_clones': 0, 'num_object_copies': 1088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17781, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3142}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:17.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:17.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20133", 'reported_seq': 20981, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:11.553526+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:11.553526+0000', 'last_peered': '2024-09-10T16:02:11.553526+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:11.553526+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10100", 'ondisk_log_start': "46'10100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2953216, 'num_objects': 721, 'num_object_clones': 0, 'num_object_copies': 1442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17325, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2757}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:17.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:17.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20000", 'reported_seq': 20756, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:11.539647+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:11.539647+0000', 'last_peered': '2024-09-10T16:02:11.539647+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:11.539647+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'9900", 'ondisk_log_start': "46'9900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2940928, 'num_objects': 718, 'num_object_clones': 0, 'num_object_copies': 1436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16418, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2908}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:17.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:17.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20074", 'reported_seq': 20906, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:11.569440+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:11.569440+0000', 'last_peered': '2024-09-10T16:02:11.569440+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:11.569440+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10000", 'ondisk_log_start': "46'10000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2834432, 'num_objects': 692, 'num_object_clones': 0, 'num_object_copies': 1384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 692, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16680, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2844}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:17.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:17.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'20383", 'reported_seq': 21150, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:11.532339+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:11.532339+0000', 'last_peered': '2024-09-10T16:02:11.532339+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:11.532339+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'10300", 'ondisk_log_start': "46'10300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2929123, 'num_objects': 716, 'num_object_clones': 0, 'num_object_copies': 1432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 716, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17202, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2978}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:17.054 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:17.781 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:17.781 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30767/21502 objects degraded (143.089%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":663,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":10751,"data_bytes":44020238,"bytes_used":1607938048,"bytes_avail":288302354432,"bytes_total":289910292480,"degraded_objects":30767,"degraded_total":21502,"degraded_ratio":1.4308901497535116,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":164,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":484},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:17.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:17.805 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:18.387 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:18.388 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:18.418 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-10T16:02:16.635310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":44020238,"num_objects":10751,"num_object_clones":0,"num_object_copies":21502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10751,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282303,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15883,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160883,"ondisk_log_size":160883,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1570252,"kb_used_data":275868,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281545268,"statfs":{"total":289910292480,"available":288302354432,"internally_reserved":0,"allocated":282488832,"data_stored":281561901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1325386757},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,6,11,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23818240,"num_objects":-5815,"num_object_clones":0,"num_object_copies":-11630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5815,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004443"},"pg_stats":[{"pgid":"2.f","version":"50'20046","reported_seq":23962,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.586084+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:11.586084+0000","last_peered":"2024-09-10T16:02:11.586084+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:11.586084+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2965504,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20046,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1914,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1335}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20491","reported_seq":20609,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:16.016613+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:16.016613+0000","last_peered":"2024-09-10T16:02:16.016613+0000","last_clean":"2024-09-10T16:02:16.016613+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:16.016613+0000","last_undegraded":"2024-09-10T16:02:16.016613+0000","last_fullsized":"2024-09-10T16:02:16.016613+0000","mapping_epoch":36,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2387968,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18827,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20062","reported_seq":20543,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.031129+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:16.031129+0000","last_peered":"2024-09-10T16:02:16.031129+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:16.031129+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2322456,"num_objects":568,"num_object_clones":0,"num_object_copies":1136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":568,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18330,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3061}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20091","reported_seq":20181,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:15.983436+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:15.983436+0000","last_peered":"2024-09-10T16:02:15.983436+0000","last_clean":"2024-09-10T16:02:15.983436+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:15.983436+0000","last_undegraded":"2024-09-10T16:02:15.983436+0000","last_fullsized":"2024-09-10T16:02:15.983436+0000","mapping_epoch":36,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2420736,"num_objects":591,"num_object_clones":0,"num_object_copies":1182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20091,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19608","reported_seq":19745,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.586861+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:11.586861+0000","last_peered":"2024-09-10T16:02:11.586861+0000","last_clean":"2024-09-10T16:02:11.586861+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:11.586861+0000","last_undegraded":"2024-09-10T16:02:11.586861+0000","last_fullsized":"2024-09-10T16:02:11.586861+0000","mapping_epoch":35,"log_start":"46'9600","ondisk_log_start":"46'9600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2961408,"num_objects":723,"num_object_clones":0,"num_object_copies":1446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16679,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20137","reported_seq":28035,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.206407+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:12.206407+0000","last_peered":"2024-09-10T16:02:12.206407+0000","last_clean":"2024-09-10T16:02:12.206407+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:12.206407+0000","last_undegraded":"2024-09-10T16:02:12.206407+0000","last_fullsized":"2024-09-10T16:02:12.206407+0000","mapping_epoch":47,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2990080,"num_objects":730,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16855,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19547","reported_seq":20315,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.478105+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:11.478105+0000","last_peered":"2024-09-10T16:02:11.478105+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:11.478105+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9500","ondisk_log_start":"46'9500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3026944,"num_objects":739,"num_object_clones":0,"num_object_copies":1478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16197,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20182","reported_seq":20324,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:11.527415+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:11.527415+0000","last_peered":"2024-09-10T16:02:11.527415+0000","last_clean":"2024-09-10T16:02:11.527415+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:11.527415+0000","last_undegraded":"2024-09-10T16:02:11.527415+0000","last_fullsized":"2024-09-10T16:02:11.527415+0000","mapping_epoch":35,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2998272,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16941,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20069","reported_seq":28207,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:12.128416+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:12.128416+0000","last_peered":"2024-09-10T16:02:12.128416+0000","last_clean":"2024-09-10T16:02:12.128416+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:12.128416+0000","last_undegraded":"2024-09-10T16:02:12.128416+0000","last_fullsized":"2024-09-10T16:02:12.128416+0000","mapping_epoch":45,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2662400,"num_objects":650,"num_object_clones":0,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19817","reported_seq":20639,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.598889+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:11.598889+0000","last_peered":"2024-09-10T16:02:11.598889+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:11.598889+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3117056,"num_objects":761,"num_object_clones":0,"num_object_copies":1522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16681,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20228","reported_seq":28038,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:16.082959+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:16.082959+0000","last_peered":"2024-09-10T16:02:16.082959+0000","last_clean":"2024-09-10T16:02:16.082959+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:16.082959+0000","last_undegraded":"2024-09-10T16:02:16.082959+0000","last_fullsized":"2024-09-10T16:02:16.082959+0000","mapping_epoch":49,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2281472,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18202,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19912","reported_seq":20334,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.091424+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:16.091424+0000","last_peered":"2024-09-10T16:02:16.091424+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:16.091424+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":544,"num_object_clones":0,"num_object_copies":1088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17781,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3142}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20133","reported_seq":20981,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.553526+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:11.553526+0000","last_peered":"2024-09-10T16:02:11.553526+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:11.553526+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2953216,"num_objects":721,"num_object_clones":0,"num_object_copies":1442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17325,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20000","reported_seq":20756,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.539647+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:11.539647+0000","last_peered":"2024-09-10T16:02:11.539647+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:11.539647+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2940928,"num_objects":718,"num_object_clones":0,"num_object_copies":1436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16418,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20074","reported_seq":20906,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.569440+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:11.569440+0000","last_peered":"2024-09-10T16:02:11.569440+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:11.569440+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2834432,"num_objects":692,"num_object_clones":0,"num_object_copies":1384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16680,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20383","reported_seq":21150,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:11.532339+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:11.532339+0000","last_peered":"2024-09-10T16:02:11.532339+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:11.532339+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2929123,"num_objects":716,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17202,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":44020219,"num_objects":10749,"num_object_clones":0,"num_object_copies":21498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10749,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282301,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15881,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":279408640,"data_stored":279331849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160880,"ondisk_log_size":160880,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626544,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499808,"kb_used_data":35024,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93872032,"statfs":{"total":96636764160,"available":96124960768,"internally_reserved":0,"allocated":35864576,"data_stored":35515703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475917998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149825,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570476,"kb_used_data":97372,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93801364,"statfs":{"total":96636764160,"available":96052596736,"internally_reserved":0,"allocated":99708928,"data_stored":99421935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":484437677},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953597,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499968,"kb_used_data":143472,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93871872,"statfs":{"total":96636764160,"available":96124796928,"internally_reserved":0,"allocated":146915328,"data_stored":146624263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":99061760,"data_stored":99058147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":34164736,"data_stored":34099243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146182144,"data_stored":146174459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:21.421 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:21.986 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:21.989 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:22.012 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-10T16:02:20.637441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":36860430,"num_objects":9003,"num_object_clones":0,"num_object_copies":18006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9003,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15974,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1550956,"kb_used_data":260720,"kb_used_omap":12,"kb_used_meta":1290163,"kb_avail":281564564,"statfs":{"total":289910292480,"available":288322113536,"internally_reserved":0,"allocated":266977280,"data_stored":266050349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1321126917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,5,9,16,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20762624,"num_objects":-5069,"num_object_clones":0,"num_object_copies":-10138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5069,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004943"},"pg_stats":[{"pgid":"2.f","version":"50'20196","reported_seq":24293,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.596181+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:16.596181+0000","last_peered":"2024-09-10T16:02:16.596181+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:16.596181+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2351104,"num_objects":574,"num_object_clones":0,"num_object_copies":1148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20196,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2005,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1394}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20491","reported_seq":20609,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:16.016613+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:16.016613+0000","last_peered":"2024-09-10T16:02:16.016613+0000","last_clean":"2024-09-10T16:02:16.016613+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:16.016613+0000","last_undegraded":"2024-09-10T16:02:16.016613+0000","last_fullsized":"2024-09-10T16:02:16.016613+0000","mapping_epoch":36,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2387968,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18827,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20062","reported_seq":20543,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.031129+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:16.031129+0000","last_peered":"2024-09-10T16:02:16.031129+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:16.031129+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2322456,"num_objects":568,"num_object_clones":0,"num_object_copies":1136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":568,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18330,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3061}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20091","reported_seq":20181,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:15.983436+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:15.983436+0000","last_peered":"2024-09-10T16:02:15.983436+0000","last_clean":"2024-09-10T16:02:15.983436+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:15.983436+0000","last_undegraded":"2024-09-10T16:02:15.983436+0000","last_fullsized":"2024-09-10T16:02:15.983436+0000","mapping_epoch":36,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2420736,"num_objects":591,"num_object_clones":0,"num_object_copies":1182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20091,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19763","reported_seq":19900,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:16.621361+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:16.621361+0000","last_peered":"2024-09-10T16:02:16.621361+0000","last_clean":"2024-09-10T16:02:16.621361+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:16.621361+0000","last_undegraded":"2024-09-10T16:02:16.621361+0000","last_fullsized":"2024-09-10T16:02:16.621361+0000","mapping_epoch":35,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2326528,"num_objects":568,"num_object_clones":0,"num_object_copies":1136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16834,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20306","reported_seq":28204,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:17.309048+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:17.309048+0000","last_peered":"2024-09-10T16:02:17.309048+0000","last_clean":"2024-09-10T16:02:17.309048+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:17.309048+0000","last_undegraded":"2024-09-10T16:02:17.309048+0000","last_fullsized":"2024-09-10T16:02:17.309048+0000","mapping_epoch":47,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2297856,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17024,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19701","reported_seq":20471,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.613504+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:16.613504+0000","last_peered":"2024-09-10T16:02:16.613504+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:16.613504+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2396160,"num_objects":585,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16351,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20365","reported_seq":20507,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:16.607071+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:16.607071+0000","last_peered":"2024-09-10T16:02:16.607071+0000","last_clean":"2024-09-10T16:02:16.607071+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:16.607071+0000","last_undegraded":"2024-09-10T16:02:16.607071+0000","last_fullsized":"2024-09-10T16:02:16.607071+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2248704,"num_objects":549,"num_object_clones":0,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17124,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20219","reported_seq":28357,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:17.363947+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:17.363947+0000","last_peered":"2024-09-10T16:02:17.363947+0000","last_clean":"2024-09-10T16:02:17.363947+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:17.363947+0000","last_undegraded":"2024-09-10T16:02:17.363947+0000","last_fullsized":"2024-09-10T16:02:17.363947+0000","mapping_epoch":45,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2048000,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18196,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19988","reported_seq":20809,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.627372+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:16.627372+0000","last_peered":"2024-09-10T16:02:16.627372+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:16.627372+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2416640,"num_objects":590,"num_object_clones":0,"num_object_copies":1180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16852,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2929}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20228","reported_seq":28038,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:16.082959+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:16.082959+0000","last_peered":"2024-09-10T16:02:16.082959+0000","last_clean":"2024-09-10T16:02:16.082959+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:16.082959+0000","last_undegraded":"2024-09-10T16:02:16.082959+0000","last_fullsized":"2024-09-10T16:02:16.082959+0000","mapping_epoch":49,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2281472,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18202,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19912","reported_seq":20334,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.091424+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:16.091424+0000","last_peered":"2024-09-10T16:02:16.091424+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:16.091424+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":544,"num_object_clones":0,"num_object_copies":1088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17781,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3142}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20294","reported_seq":21142,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.594250+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:16.594250+0000","last_peered":"2024-09-10T16:02:16.594250+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:16.594250+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":560,"num_object_clones":0,"num_object_copies":1120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17486,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20157","reported_seq":20913,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.592823+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:16.592823+0000","last_peered":"2024-09-10T16:02:16.592823+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:16.592823+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2297856,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16575,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20228","reported_seq":21060,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.605440+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:16.605440+0000","last_peered":"2024-09-10T16:02:16.605440+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:16.605440+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2203648,"num_objects":538,"num_object_clones":0,"num_object_copies":1076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16834,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20527","reported_seq":21293,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.637966+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:16.637966+0000","last_peered":"2024-09-10T16:02:16.637966+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:16.637966+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2339299,"num_objects":572,"num_object_clones":0,"num_object_copies":1144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17346,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3122}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":36860411,"num_objects":9001,"num_object_clones":0,"num_object_copies":18002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9001,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15972,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":263929856,"data_stored":263853065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626545,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500708,"kb_used_data":27596,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93871132,"statfs":{"total":96636764160,"available":96124039168,"internally_reserved":0,"allocated":28258304,"data_stored":27909431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484437678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149826,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550280,"kb_used_data":89652,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93821560,"statfs":{"total":96636764160,"available":96073277440,"internally_reserved":0,"allocated":91803648,"data_stored":91516655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":471658157},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953597,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499968,"kb_used_data":143472,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93871872,"statfs":{"total":96636764160,"available":96124796928,"internally_reserved":0,"allocated":146915328,"data_stored":146624263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":91246592,"data_stored":91242979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26501120,"data_stored":26435627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146182144,"data_stored":146174459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:22.014 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:22.582 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:22.608 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-10T16:02:20.637441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":36860430,"num_objects":9003,"num_object_clones":0,"num_object_copies":18006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9003,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15974,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1550956,"kb_used_data":260720,"kb_used_omap":12,"kb_used_meta":1290163,"kb_avail":281564564,"statfs":{"total":289910292480,"available":288322113536,"internally_reserved":0,"allocated":266977280,"data_stored":266050349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1321126917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,5,9,16,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20762624,"num_objects":-5069,"num_object_clones":0,"num_object_copies":-10138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5069,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004943"},"pg_stats":[{"pgid":"2.f","version":"50'20196","reported_seq":24293,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.596181+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:16.596181+0000","last_peered":"2024-09-10T16:02:16.596181+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:16.596181+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2351104,"num_objects":574,"num_object_clones":0,"num_object_copies":1148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20196,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2005,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1394}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20491","reported_seq":20609,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:16.016613+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:16.016613+0000","last_peered":"2024-09-10T16:02:16.016613+0000","last_clean":"2024-09-10T16:02:16.016613+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:16.016613+0000","last_undegraded":"2024-09-10T16:02:16.016613+0000","last_fullsized":"2024-09-10T16:02:16.016613+0000","mapping_epoch":36,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2387968,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18827,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20062","reported_seq":20543,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.031129+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:16.031129+0000","last_peered":"2024-09-10T16:02:16.031129+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:16.031129+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2322456,"num_objects":568,"num_object_clones":0,"num_object_copies":1136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":568,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18330,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3061}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20091","reported_seq":20181,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:15.983436+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:15.983436+0000","last_peered":"2024-09-10T16:02:15.983436+0000","last_clean":"2024-09-10T16:02:15.983436+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:15.983436+0000","last_undegraded":"2024-09-10T16:02:15.983436+0000","last_fullsized":"2024-09-10T16:02:15.983436+0000","mapping_epoch":36,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2420736,"num_objects":591,"num_object_clones":0,"num_object_copies":1182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20091,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19763","reported_seq":19900,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:16.621361+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:16.621361+0000","last_peered":"2024-09-10T16:02:16.621361+0000","last_clean":"2024-09-10T16:02:16.621361+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:16.621361+0000","last_undegraded":"2024-09-10T16:02:16.621361+0000","last_fullsized":"2024-09-10T16:02:16.621361+0000","mapping_epoch":35,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2326528,"num_objects":568,"num_object_clones":0,"num_object_copies":1136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16834,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20306","reported_seq":28204,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:17.309048+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:17.309048+0000","last_peered":"2024-09-10T16:02:17.309048+0000","last_clean":"2024-09-10T16:02:17.309048+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:17.309048+0000","last_undegraded":"2024-09-10T16:02:17.309048+0000","last_fullsized":"2024-09-10T16:02:17.309048+0000","mapping_epoch":47,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2297856,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17024,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19701","reported_seq":20471,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.613504+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:16.613504+0000","last_peered":"2024-09-10T16:02:16.613504+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:16.613504+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9700","ondisk_log_start":"46'9700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2396160,"num_objects":585,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16351,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20365","reported_seq":20507,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:16.607071+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:16.607071+0000","last_peered":"2024-09-10T16:02:16.607071+0000","last_clean":"2024-09-10T16:02:16.607071+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:16.607071+0000","last_undegraded":"2024-09-10T16:02:16.607071+0000","last_fullsized":"2024-09-10T16:02:16.607071+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2248704,"num_objects":549,"num_object_clones":0,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17124,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20219","reported_seq":28357,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:17.363947+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:17.363947+0000","last_peered":"2024-09-10T16:02:17.363947+0000","last_clean":"2024-09-10T16:02:17.363947+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:17.363947+0000","last_undegraded":"2024-09-10T16:02:17.363947+0000","last_fullsized":"2024-09-10T16:02:17.363947+0000","mapping_epoch":45,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2048000,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18196,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'19988","reported_seq":20809,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.627372+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:16.627372+0000","last_peered":"2024-09-10T16:02:16.627372+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:16.627372+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2416640,"num_objects":590,"num_object_clones":0,"num_object_copies":1180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16852,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2929}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20228","reported_seq":28038,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:16.082959+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:16.082959+0000","last_peered":"2024-09-10T16:02:16.082959+0000","last_clean":"2024-09-10T16:02:16.082959+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:16.082959+0000","last_undegraded":"2024-09-10T16:02:16.082959+0000","last_fullsized":"2024-09-10T16:02:16.082959+0000","mapping_epoch":49,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2281472,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18202,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'19912","reported_seq":20334,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.091424+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:16.091424+0000","last_peered":"2024-09-10T16:02:16.091424+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:16.091424+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2228224,"num_objects":544,"num_object_clones":0,"num_object_copies":1088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17781,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3142}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20294","reported_seq":21142,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.594250+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:16.594250+0000","last_peered":"2024-09-10T16:02:16.594250+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:16.594250+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":560,"num_object_clones":0,"num_object_copies":1120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17486,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20157","reported_seq":20913,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.592823+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:16.592823+0000","last_peered":"2024-09-10T16:02:16.592823+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:16.592823+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2297856,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16575,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20228","reported_seq":21060,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.605440+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:16.605440+0000","last_peered":"2024-09-10T16:02:16.605440+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:16.605440+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2203648,"num_objects":538,"num_object_clones":0,"num_object_copies":1076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16834,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20527","reported_seq":21293,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:16.637966+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:16.637966+0000","last_peered":"2024-09-10T16:02:16.637966+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:16.637966+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2339299,"num_objects":572,"num_object_clones":0,"num_object_copies":1144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17346,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3122}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":36860411,"num_objects":9001,"num_object_clones":0,"num_object_copies":18002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9001,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15972,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":263929856,"data_stored":263853065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626545,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500708,"kb_used_data":27596,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93871132,"statfs":{"total":96636764160,"available":96124039168,"internally_reserved":0,"allocated":28258304,"data_stored":27909431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484437678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149826,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550280,"kb_used_data":89652,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93821560,"statfs":{"total":96636764160,"available":96073277440,"internally_reserved":0,"allocated":91803648,"data_stored":91516655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":471658157},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953597,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499968,"kb_used_data":143472,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93871872,"statfs":{"total":96636764160,"available":96124796928,"internally_reserved":0,"allocated":146915328,"data_stored":146624263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":91246592,"data_stored":91242979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26501120,"data_stored":26435627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146182144,"data_stored":146174459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'20196", 'reported_seq': 24293, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:16.596181+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:02:16.596181+0000', 'last_peered': '2024-09-10T16:02:16.596181+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:02:16.596181+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'10100", 'ondisk_log_start': "46'10100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2351104, 'num_objects': 574, 'num_object_clones': 0, 'num_object_copies': 1148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1968, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 574, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20196, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2005, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1394}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20062", 'reported_seq': 20543, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:16.031129+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:16.031129+0000', 'last_peered': '2024-09-10T16:02:16.031129+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:16.031129+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10000", 'ondisk_log_start': "46'10000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2322456, 'num_objects': 568, 'num_object_clones': 0, 'num_object_copies': 1136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 568, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18330, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3061}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:22.610 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:22.610 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'19701", 'reported_seq': 20471, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:16.613504+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:16.613504+0000', 'last_peered': '2024-09-10T16:02:16.613504+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:16.613504+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'9700", 'ondisk_log_start': "46'9700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2396160, 'num_objects': 585, 'num_object_clones': 0, 'num_object_copies': 1170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 585, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16351, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:22.610 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:22.610 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'19988", 'reported_seq': 20809, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:16.627372+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:16.627372+0000', 'last_peered': '2024-09-10T16:02:16.627372+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:16.627372+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'9900", 'ondisk_log_start': "46'9900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2416640, 'num_objects': 590, 'num_object_clones': 0, 'num_object_copies': 1180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 590, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16852, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2929}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:22.610 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:22.610 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'19912", 'reported_seq': 20334, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:16.091424+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:02:16.091424+0000', 'last_peered': '2024-09-10T16:02:16.091424+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:02:16.091424+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'9900", 'ondisk_log_start': "46'9900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2228224, 'num_objects': 544, 'num_object_clones': 0, 'num_object_copies': 1088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17781, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3142}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:22.610 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:22.610 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20294", 'reported_seq': 21142, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:16.594250+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:16.594250+0000', 'last_peered': '2024-09-10T16:02:16.594250+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:16.594250+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2293760, 'num_objects': 560, 'num_object_clones': 0, 'num_object_copies': 1120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 560, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17486, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2918}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:22.610 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:22.610 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20157", 'reported_seq': 20913, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:16.592823+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:16.592823+0000', 'last_peered': '2024-09-10T16:02:16.592823+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:16.592823+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10100", 'ondisk_log_start': "46'10100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2297856, 'num_objects': 561, 'num_object_clones': 0, 'num_object_copies': 1122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 561, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16575, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:22.610 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:22.611 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20228", 'reported_seq': 21060, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:16.605440+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:16.605440+0000', 'last_peered': '2024-09-10T16:02:16.605440+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:16.605440+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2203648, 'num_objects': 538, 'num_object_clones': 0, 'num_object_copies': 1076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 538, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16834, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2998}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:22.611 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:22.611 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'20527", 'reported_seq': 21293, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:16.637966+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:16.637966+0000', 'last_peered': '2024-09-10T16:02:16.637966+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:16.637966+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2339299, 'num_objects': 572, 'num_object_clones': 0, 'num_object_copies': 1144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 572, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17346, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3122}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:22.611 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:23.300 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:23.300 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30581/13012 objects degraded (235.022%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":669,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":6506,"data_bytes":26632718,"bytes_used":1563627520,"bytes_avail":288346664960,"bytes_total":289910292480,"degraded_objects":30581,"degraded_total":13012,"degraded_ratio":2.3502151859821705,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":276,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":630},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:23.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:23.326 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:23.881 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:23.881 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:23.905 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-10T16:02:22.638565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26632718,"num_objects":6506,"num_object_clones":0,"num_object_copies":13012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6506,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286548,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16069,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526980,"kb_used_data":240904,"kb_used_omap":12,"kb_used_meta":1286003,"kb_avail":281588540,"statfs":{"total":289910292480,"available":288346664960,"internally_reserved":0,"allocated":246685696,"data_stored":245758765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1316867077},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,6,9,11,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30990336,"num_objects":-7566,"num_object_clones":0,"num_object_copies":-15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-276,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7566,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005317"},"pg_stats":[{"pgid":"2.f","version":"50'20347","reported_seq":24635,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.677715+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:21.677715+0000","last_peered":"2024-09-10T16:02:21.677715+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:21.677715+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1732608,"num_objects":423,"num_object_clones":0,"num_object_copies":846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1873,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20347,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20638","reported_seq":20756,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:21.005433+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:21.005433+0000","last_peered":"2024-09-10T16:02:21.005433+0000","last_clean":"2024-09-10T16:02:21.005433+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:21.005433+0000","last_undegraded":"2024-09-10T16:02:21.005433+0000","last_fullsized":"2024-09-10T16:02:21.005433+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1785856,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18974,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20199","reported_seq":20680,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.116253+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:21.116253+0000","last_peered":"2024-09-10T16:02:21.116253+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:21.116253+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1761304,"num_objects":431,"num_object_clones":0,"num_object_copies":862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18467,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3198}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20248","reported_seq":20336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:21.119911+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:21.119911+0000","last_peered":"2024-09-10T16:02:21.119911+0000","last_clean":"2024-09-10T16:02:21.119911+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:21.119911+0000","last_undegraded":"2024-09-10T16:02:21.119911+0000","last_fullsized":"2024-09-10T16:02:21.119911+0000","mapping_epoch":36,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1777664,"num_objects":434,"num_object_clones":0,"num_object_copies":868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20248,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19926","reported_seq":20063,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:21.637836+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:21.637836+0000","last_peered":"2024-09-10T16:02:21.637836+0000","last_clean":"2024-09-10T16:02:21.637836+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:21.637836+0000","last_undegraded":"2024-09-10T16:02:21.637836+0000","last_fullsized":"2024-09-10T16:02:21.637836+0000","mapping_epoch":35,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1658880,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16997,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20474","reported_seq":28372,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:22.380285+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:22.380285+0000","last_peered":"2024-09-10T16:02:22.380285+0000","last_clean":"2024-09-10T16:02:22.380285+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:22.380285+0000","last_undegraded":"2024-09-10T16:02:22.380285+0000","last_fullsized":"2024-09-10T16:02:22.380285+0000","mapping_epoch":47,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1609728,"num_objects":393,"num_object_clones":0,"num_object_copies":786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17192,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19871","reported_seq":20642,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.571682+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:21.571682+0000","last_peered":"2024-09-10T16:02:21.571682+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:21.571682+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1699840,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16521,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20523","reported_seq":20665,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:21.563333+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:21.563333+0000","last_peered":"2024-09-10T16:02:21.563333+0000","last_clean":"2024-09-10T16:02:21.563333+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:21.563333+0000","last_undegraded":"2024-09-10T16:02:21.563333+0000","last_fullsized":"2024-09-10T16:02:21.563333+0000","mapping_epoch":35,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1601536,"num_objects":391,"num_object_clones":0,"num_object_copies":782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17282,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20364","reported_seq":28502,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:22.138891+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:22.138891+0000","last_peered":"2024-09-10T16:02:22.138891+0000","last_clean":"2024-09-10T16:02:22.138891+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:22.138891+0000","last_undegraded":"2024-09-10T16:02:22.138891+0000","last_fullsized":"2024-09-10T16:02:22.138891+0000","mapping_epoch":45,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1454080,"num_objects":355,"num_object_clones":0,"num_object_copies":710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18341,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20158","reported_seq":20982,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.639011+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:21.639011+0000","last_peered":"2024-09-10T16:02:21.639011+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:21.639011+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1720320,"num_objects":420,"num_object_clones":0,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20371","reported_seq":28181,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:21.093105+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:21.093105+0000","last_peered":"2024-09-10T16:02:21.093105+0000","last_clean":"2024-09-10T16:02:21.093105+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:21.093105+0000","last_undegraded":"2024-09-10T16:02:21.093105+0000","last_fullsized":"2024-09-10T16:02:21.093105+0000","mapping_epoch":49,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1695744,"num_objects":414,"num_object_clones":0,"num_object_copies":828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18345,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20080","reported_seq":20503,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.137526+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:21.137526+0000","last_peered":"2024-09-10T16:02:21.137526+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:21.137526+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1540096,"num_objects":376,"num_object_clones":0,"num_object_copies":752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17949,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20458","reported_seq":21306,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.654717+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:21.654717+0000","last_peered":"2024-09-10T16:02:21.654717+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:21.654717+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1622016,"num_objects":396,"num_object_clones":0,"num_object_copies":792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17650,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20318","reported_seq":21072,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.622704+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:21.622704+0000","last_peered":"2024-09-10T16:02:21.622704+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:21.622704+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":400,"num_object_clones":0,"num_object_copies":800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16736,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20364","reported_seq":21196,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.639303+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:21.639303+0000","last_peered":"2024-09-10T16:02:21.639303+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:21.639303+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1646592,"num_objects":402,"num_object_clones":0,"num_object_copies":804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16970,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20686","reported_seq":21453,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.603160+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:21.603160+0000","last_peered":"2024-09-10T16:02:21.603160+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:21.603160+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1688035,"num_objects":413,"num_object_clones":0,"num_object_copies":826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17505,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":26632699,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286546,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16067,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":243388416,"data_stored":243311625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626546,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480588,"kb_used_data":19964,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93891252,"statfs":{"total":96636764160,"available":96144642048,"internally_reserved":0,"allocated":20443136,"data_stored":20094263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471658158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149827,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547072,"kb_used_data":82276,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93824768,"statfs":{"total":96636764160,"available":96076562432,"internally_reserved":0,"allocated":84250624,"data_stored":83963631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,3,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953598,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499320,"kb_used_data":138664,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93872520,"statfs":{"total":96636764160,"available":96125460480,"internally_reserved":0,"allocated":141991936,"data_stored":141700871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":83423232,"data_stored":83419619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18788352,"data_stored":18722859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":141176832,"data_stored":141169147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:26.908 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:27.360 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:27.360 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:27.371 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-10T16:02:26.639952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23405070,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":287336,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16069,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160716,"ondisk_log_size":160716,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526252,"kb_used_data":236012,"kb_used_omap":12,"kb_used_meta":1290163,"kb_avail":281589268,"statfs":{"total":289910292480,"available":288347410432,"internally_reserved":0,"allocated":241676288,"data_stored":240749357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1321126917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,4,10,8,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23859200,"num_objects":-5825,"num_object_clones":0,"num_object_copies":-11650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5825,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005231"},"pg_stats":[{"pgid":"2.f","version":"50'20347","reported_seq":24635,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.677715+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:21.677715+0000","last_peered":"2024-09-10T16:02:21.677715+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:21.677715+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1732608,"num_objects":423,"num_object_clones":0,"num_object_copies":846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1873,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20347,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20806","reported_seq":20924,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:26.070225+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:26.070225+0000","last_peered":"2024-09-10T16:02:26.070225+0000","last_clean":"2024-09-10T16:02:26.070225+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:26.070225+0000","last_undegraded":"2024-09-10T16:02:26.070225+0000","last_fullsized":"2024-09-10T16:02:26.070225+0000","mapping_epoch":36,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1097728,"num_objects":268,"num_object_clones":0,"num_object_copies":536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19142,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20372","reported_seq":20853,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.160264+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:26.160264+0000","last_peered":"2024-09-10T16:02:26.160264+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:26.160264+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052696,"num_objects":258,"num_object_clones":0,"num_object_copies":516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":258,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18640,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3371}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20402","reported_seq":20492,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:26.128134+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:26.128134+0000","last_peered":"2024-09-10T16:02:26.128134+0000","last_clean":"2024-09-10T16:02:26.128134+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:26.128134+0000","last_undegraded":"2024-09-10T16:02:26.128134+0000","last_fullsized":"2024-09-10T16:02:26.128134+0000","mapping_epoch":36,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1146880,"num_objects":280,"num_object_clones":0,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20402,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19926","reported_seq":20063,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:21.637836+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:21.637836+0000","last_peered":"2024-09-10T16:02:21.637836+0000","last_clean":"2024-09-10T16:02:21.637836+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:21.637836+0000","last_undegraded":"2024-09-10T16:02:21.637836+0000","last_fullsized":"2024-09-10T16:02:21.637836+0000","mapping_epoch":35,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1658880,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16997,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20474","reported_seq":28372,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:22.380285+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:22.380285+0000","last_peered":"2024-09-10T16:02:22.380285+0000","last_clean":"2024-09-10T16:02:22.380285+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:22.380285+0000","last_undegraded":"2024-09-10T16:02:22.380285+0000","last_fullsized":"2024-09-10T16:02:22.380285+0000","mapping_epoch":47,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1609728,"num_objects":393,"num_object_clones":0,"num_object_copies":786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17192,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19871","reported_seq":20642,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.571682+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:21.571682+0000","last_peered":"2024-09-10T16:02:21.571682+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:21.571682+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1699840,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16521,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20523","reported_seq":20665,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:21.563333+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:21.563333+0000","last_peered":"2024-09-10T16:02:21.563333+0000","last_clean":"2024-09-10T16:02:21.563333+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:21.563333+0000","last_undegraded":"2024-09-10T16:02:21.563333+0000","last_fullsized":"2024-09-10T16:02:21.563333+0000","mapping_epoch":35,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1601536,"num_objects":391,"num_object_clones":0,"num_object_copies":782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17282,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20364","reported_seq":28502,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:22.138891+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:22.138891+0000","last_peered":"2024-09-10T16:02:22.138891+0000","last_clean":"2024-09-10T16:02:22.138891+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:22.138891+0000","last_undegraded":"2024-09-10T16:02:22.138891+0000","last_fullsized":"2024-09-10T16:02:22.138891+0000","mapping_epoch":45,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1454080,"num_objects":355,"num_object_clones":0,"num_object_copies":710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18341,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20158","reported_seq":20982,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.639011+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:21.639011+0000","last_peered":"2024-09-10T16:02:21.639011+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:21.639011+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1720320,"num_objects":420,"num_object_clones":0,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20531","reported_seq":28341,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:26.161393+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:26.161393+0000","last_peered":"2024-09-10T16:02:26.161393+0000","last_clean":"2024-09-10T16:02:26.161393+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:26.161393+0000","last_undegraded":"2024-09-10T16:02:26.161393+0000","last_fullsized":"2024-09-10T16:02:26.161393+0000","mapping_epoch":49,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1040384,"num_objects":254,"num_object_clones":0,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18505,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20213","reported_seq":20636,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.038662+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:26.038662+0000","last_peered":"2024-09-10T16:02:26.038662+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:26.038662+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":995328,"num_objects":243,"num_object_clones":0,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18082,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3443}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20458","reported_seq":21306,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.654717+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:21.654717+0000","last_peered":"2024-09-10T16:02:21.654717+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:21.654717+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1622016,"num_objects":396,"num_object_clones":0,"num_object_copies":792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17650,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20318","reported_seq":21072,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.622704+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:21.622704+0000","last_peered":"2024-09-10T16:02:21.622704+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:21.622704+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":400,"num_object_clones":0,"num_object_copies":800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16736,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20364","reported_seq":21196,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.639303+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:21.639303+0000","last_peered":"2024-09-10T16:02:21.639303+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:21.639303+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1646592,"num_objects":402,"num_object_clones":0,"num_object_copies":804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16970,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20686","reported_seq":21453,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.603160+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:21.603160+0000","last_peered":"2024-09-10T16:02:21.603160+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:21.603160+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1688035,"num_objects":413,"num_object_clones":0,"num_object_copies":826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17505,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":23405051,"num_objects":5716,"num_object_clones":0,"num_object_copies":11432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5716,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":287334,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16067,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":238276608,"data_stored":238199817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626546,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480588,"kb_used_data":19964,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93891252,"statfs":{"total":96636764160,"available":96144642048,"internally_reserved":0,"allocated":20443136,"data_stored":20094263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471658158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149827,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547072,"kb_used_data":82276,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93824768,"statfs":{"total":96636764160,"available":96076562432,"internally_reserved":0,"allocated":84250624,"data_stored":83963631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,3,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953599,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498592,"kb_used_data":133772,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93873248,"statfs":{"total":96636764160,"available":96126205952,"internally_reserved":0,"allocated":136982528,"data_stored":136691463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":83423232,"data_stored":83419619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18788352,"data_stored":18722859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136065024,"data_stored":136057339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:27.373 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:27.869 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:27.872 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:27.893 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-10T16:02:26.639952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23405070,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":287336,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16069,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160716,"ondisk_log_size":160716,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526252,"kb_used_data":236012,"kb_used_omap":12,"kb_used_meta":1290163,"kb_avail":281589268,"statfs":{"total":289910292480,"available":288347410432,"internally_reserved":0,"allocated":241676288,"data_stored":240749357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1321126917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,4,10,8,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23859200,"num_objects":-5825,"num_object_clones":0,"num_object_copies":-11650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5825,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005231"},"pg_stats":[{"pgid":"2.f","version":"50'20347","reported_seq":24635,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.677715+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:21.677715+0000","last_peered":"2024-09-10T16:02:21.677715+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:21.677715+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1732608,"num_objects":423,"num_object_clones":0,"num_object_copies":846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1873,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20347,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20806","reported_seq":20924,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:26.070225+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:26.070225+0000","last_peered":"2024-09-10T16:02:26.070225+0000","last_clean":"2024-09-10T16:02:26.070225+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:26.070225+0000","last_undegraded":"2024-09-10T16:02:26.070225+0000","last_fullsized":"2024-09-10T16:02:26.070225+0000","mapping_epoch":36,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1097728,"num_objects":268,"num_object_clones":0,"num_object_copies":536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19142,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20372","reported_seq":20853,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.160264+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:26.160264+0000","last_peered":"2024-09-10T16:02:26.160264+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:26.160264+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052696,"num_objects":258,"num_object_clones":0,"num_object_copies":516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":258,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18640,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3371}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20402","reported_seq":20492,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:26.128134+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:26.128134+0000","last_peered":"2024-09-10T16:02:26.128134+0000","last_clean":"2024-09-10T16:02:26.128134+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:26.128134+0000","last_undegraded":"2024-09-10T16:02:26.128134+0000","last_fullsized":"2024-09-10T16:02:26.128134+0000","mapping_epoch":36,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1146880,"num_objects":280,"num_object_clones":0,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20402,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'19926","reported_seq":20063,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:21.637836+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:21.637836+0000","last_peered":"2024-09-10T16:02:21.637836+0000","last_clean":"2024-09-10T16:02:21.637836+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:21.637836+0000","last_undegraded":"2024-09-10T16:02:21.637836+0000","last_fullsized":"2024-09-10T16:02:21.637836+0000","mapping_epoch":35,"log_start":"46'9900","ondisk_log_start":"46'9900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1658880,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16997,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20474","reported_seq":28372,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:22.380285+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:22.380285+0000","last_peered":"2024-09-10T16:02:22.380285+0000","last_clean":"2024-09-10T16:02:22.380285+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:22.380285+0000","last_undegraded":"2024-09-10T16:02:22.380285+0000","last_fullsized":"2024-09-10T16:02:22.380285+0000","mapping_epoch":47,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1609728,"num_objects":393,"num_object_clones":0,"num_object_copies":786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17192,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'19871","reported_seq":20642,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.571682+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:21.571682+0000","last_peered":"2024-09-10T16:02:21.571682+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:21.571682+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'9800","ondisk_log_start":"46'9800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1699840,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16521,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20523","reported_seq":20665,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:21.563333+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:21.563333+0000","last_peered":"2024-09-10T16:02:21.563333+0000","last_clean":"2024-09-10T16:02:21.563333+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:21.563333+0000","last_undegraded":"2024-09-10T16:02:21.563333+0000","last_fullsized":"2024-09-10T16:02:21.563333+0000","mapping_epoch":35,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1601536,"num_objects":391,"num_object_clones":0,"num_object_copies":782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17282,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20364","reported_seq":28502,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:22.138891+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:22.138891+0000","last_peered":"2024-09-10T16:02:22.138891+0000","last_clean":"2024-09-10T16:02:22.138891+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:22.138891+0000","last_undegraded":"2024-09-10T16:02:22.138891+0000","last_fullsized":"2024-09-10T16:02:22.138891+0000","mapping_epoch":45,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1454080,"num_objects":355,"num_object_clones":0,"num_object_copies":710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18341,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20158","reported_seq":20982,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.639011+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:21.639011+0000","last_peered":"2024-09-10T16:02:21.639011+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:21.639011+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1720320,"num_objects":420,"num_object_clones":0,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20531","reported_seq":28341,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:26.161393+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:26.161393+0000","last_peered":"2024-09-10T16:02:26.161393+0000","last_clean":"2024-09-10T16:02:26.161393+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:26.161393+0000","last_undegraded":"2024-09-10T16:02:26.161393+0000","last_fullsized":"2024-09-10T16:02:26.161393+0000","mapping_epoch":49,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1040384,"num_objects":254,"num_object_clones":0,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18505,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20213","reported_seq":20636,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.038662+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:26.038662+0000","last_peered":"2024-09-10T16:02:26.038662+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:26.038662+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":995328,"num_objects":243,"num_object_clones":0,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18082,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3443}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20458","reported_seq":21306,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.654717+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:21.654717+0000","last_peered":"2024-09-10T16:02:21.654717+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:21.654717+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1622016,"num_objects":396,"num_object_clones":0,"num_object_copies":792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17650,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20318","reported_seq":21072,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.622704+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:21.622704+0000","last_peered":"2024-09-10T16:02:21.622704+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:21.622704+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":400,"num_object_clones":0,"num_object_copies":800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16736,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20364","reported_seq":21196,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.639303+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:21.639303+0000","last_peered":"2024-09-10T16:02:21.639303+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:21.639303+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1646592,"num_objects":402,"num_object_clones":0,"num_object_copies":804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16970,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20686","reported_seq":21453,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:21.603160+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:21.603160+0000","last_peered":"2024-09-10T16:02:21.603160+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:21.603160+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1688035,"num_objects":413,"num_object_clones":0,"num_object_copies":826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17505,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":23405051,"num_objects":5716,"num_object_clones":0,"num_object_copies":11432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5716,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":287334,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16067,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":238276608,"data_stored":238199817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626546,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480588,"kb_used_data":19964,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93891252,"statfs":{"total":96636764160,"available":96144642048,"internally_reserved":0,"allocated":20443136,"data_stored":20094263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471658158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149827,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547072,"kb_used_data":82276,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93824768,"statfs":{"total":96636764160,"available":96076562432,"internally_reserved":0,"allocated":84250624,"data_stored":83963631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,3,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953599,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498592,"kb_used_data":133772,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93873248,"statfs":{"total":96636764160,"available":96126205952,"internally_reserved":0,"allocated":136982528,"data_stored":136691463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":83423232,"data_stored":83419619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18788352,"data_stored":18722859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136065024,"data_stored":136057339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:27.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:27.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'20347", 'reported_seq': 24635, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:21.677715+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:02:21.677715+0000', 'last_peered': '2024-09-10T16:02:21.677715+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:02:21.677715+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'10300", 'ondisk_log_start': "46'10300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1732608, 'num_objects': 423, 'num_object_clones': 0, 'num_object_copies': 846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1873, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 423, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20347, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2100, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1450}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:27.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:27.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20372", 'reported_seq': 20853, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:26.160264+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:26.160264+0000', 'last_peered': '2024-09-10T16:02:26.160264+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:26.160264+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10300", 'ondisk_log_start': "46'10300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1052696, 'num_objects': 258, 'num_object_clones': 0, 'num_object_copies': 516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 258, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18640, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:27.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:27.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'19871", 'reported_seq': 20642, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:21.571682+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:21.571682+0000', 'last_peered': '2024-09-10T16:02:21.571682+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:21.571682+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'9800", 'ondisk_log_start': "46'9800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1699840, 'num_objects': 415, 'num_object_clones': 0, 'num_object_copies': 830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 415, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16521, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:27.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:27.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20158", 'reported_seq': 20982, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:21.639011+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:21.639011+0000', 'last_peered': '2024-09-10T16:02:21.639011+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:21.639011+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10100", 'ondisk_log_start': "46'10100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1720320, 'num_objects': 420, 'num_object_clones': 0, 'num_object_copies': 840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3099}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:27.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:27.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'20213", 'reported_seq': 20636, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:26.038662+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:02:26.038662+0000', 'last_peered': '2024-09-10T16:02:26.038662+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:02:26.038662+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 995328, 'num_objects': 243, 'num_object_clones': 0, 'num_object_copies': 486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 243, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18082, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3443}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:27.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:27.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20458", 'reported_seq': 21306, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:21.654717+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:21.654717+0000', 'last_peered': '2024-09-10T16:02:21.654717+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:21.654717+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10400", 'ondisk_log_start': "46'10400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1622016, 'num_objects': 396, 'num_object_clones': 0, 'num_object_copies': 792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 396, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17650, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:27.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:27.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20318", 'reported_seq': 21072, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:21.622704+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:21.622704+0000', 'last_peered': '2024-09-10T16:02:21.622704+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:21.622704+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10300", 'ondisk_log_start': "46'10300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1638400, 'num_objects': 400, 'num_object_clones': 0, 'num_object_copies': 800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 400, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16736, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:27.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:27.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20364", 'reported_seq': 21196, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:21.639303+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:21.639303+0000', 'last_peered': '2024-09-10T16:02:21.639303+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:21.639303+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10300", 'ondisk_log_start': "46'10300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1646592, 'num_objects': 402, 'num_object_clones': 0, 'num_object_copies': 804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 402, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16970, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3134}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:27.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:27.897 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'20686", 'reported_seq': 21453, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:21.603160+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:21.603160+0000', 'last_peered': '2024-09-10T16:02:21.603160+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:21.603160+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1688035, 'num_objects': 413, 'num_object_clones': 0, 'num_object_copies': 826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 413, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17505, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3281}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:27.897 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:28.573 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:28.573 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30581/11436 objects degraded (267.410%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":674,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5718,"data_bytes":23405070,"bytes_used":1562882048,"bytes_avail":288347410432,"bytes_total":289910292480,"degraded_objects":30581,"degraded_total":11436,"degraded_ratio":2.6740993354319693,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":186,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":485},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:28.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:28.596 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:29.160 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:29.161 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:29.196 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-09-10T16:02:28.640723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16474638,"num_objects":4026,"num_object_clones":0,"num_object_copies":8052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":289028,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16157,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160508,"ondisk_log_size":160508,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1528040,"kb_used_data":221156,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281587480,"statfs":{"total":289910292480,"available":288345579520,"internally_reserved":0,"allocated":226463744,"data_stored":225536813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1338166277},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7,2,4,12,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27545600,"num_objects":-6725,"num_object_clones":0,"num_object_copies":-13450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6725,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005413"},"pg_stats":[{"pgid":"2.f","version":"50'20512","reported_seq":24976,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.643962+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:26.643962+0000","last_peered":"2024-09-10T16:02:26.643962+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:26.643962+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1056768,"num_objects":258,"num_object_clones":0,"num_object_copies":516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20512,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2188,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1527}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20806","reported_seq":20924,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:26.070225+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:26.070225+0000","last_peered":"2024-09-10T16:02:26.070225+0000","last_clean":"2024-09-10T16:02:26.070225+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:26.070225+0000","last_undegraded":"2024-09-10T16:02:26.070225+0000","last_fullsized":"2024-09-10T16:02:26.070225+0000","mapping_epoch":36,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1097728,"num_objects":268,"num_object_clones":0,"num_object_copies":536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19142,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20372","reported_seq":20853,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.160264+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:26.160264+0000","last_peered":"2024-09-10T16:02:26.160264+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:26.160264+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052696,"num_objects":258,"num_object_clones":0,"num_object_copies":516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":258,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18640,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3371}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20402","reported_seq":20492,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:26.128134+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:26.128134+0000","last_peered":"2024-09-10T16:02:26.128134+0000","last_clean":"2024-09-10T16:02:26.128134+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:26.128134+0000","last_undegraded":"2024-09-10T16:02:26.128134+0000","last_fullsized":"2024-09-10T16:02:26.128134+0000","mapping_epoch":36,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1146880,"num_objects":280,"num_object_clones":0,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20402,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20096","reported_seq":20233,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:26.624595+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:26.624595+0000","last_peered":"2024-09-10T16:02:26.624595+0000","last_clean":"2024-09-10T16:02:26.624595+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:26.624595+0000","last_undegraded":"2024-09-10T16:02:26.624595+0000","last_fullsized":"2024-09-10T16:02:26.624595+0000","mapping_epoch":35,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":962560,"num_objects":235,"num_object_clones":0,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17167,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20622","reported_seq":28520,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:27.447385+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:27.447385+0000","last_peered":"2024-09-10T16:02:27.447385+0000","last_clean":"2024-09-10T16:02:27.447385+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:27.447385+0000","last_undegraded":"2024-09-10T16:02:27.447385+0000","last_fullsized":"2024-09-10T16:02:27.447385+0000","mapping_epoch":47,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1003520,"num_objects":245,"num_object_clones":0,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17340,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20029","reported_seq":20800,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.623186+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:26.623186+0000","last_peered":"2024-09-10T16:02:26.623186+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:26.623186+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10000","ondisk_log_start":"46'10000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052672,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16679,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3267}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20680","reported_seq":20822,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:26.675467+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:26.675467+0000","last_peered":"2024-09-10T16:02:26.675467+0000","last_clean":"2024-09-10T16:02:26.675467+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:26.675467+0000","last_undegraded":"2024-09-10T16:02:26.675467+0000","last_fullsized":"2024-09-10T16:02:26.675467+0000","mapping_epoch":35,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":958464,"num_objects":234,"num_object_clones":0,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17439,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20502","reported_seq":28640,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:27.241146+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:27.241146+0000","last_peered":"2024-09-10T16:02:27.241146+0000","last_clean":"2024-09-10T16:02:27.241146+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:27.241146+0000","last_undegraded":"2024-09-10T16:02:27.241146+0000","last_fullsized":"2024-09-10T16:02:27.241146+0000","mapping_epoch":45,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":888832,"num_objects":217,"num_object_clones":0,"num_object_copies":434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18479,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20327","reported_seq":21152,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.660018+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:26.660018+0000","last_peered":"2024-09-10T16:02:26.660018+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:26.660018+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1028096,"num_objects":251,"num_object_clones":0,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17191,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3268}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20531","reported_seq":28341,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:26.161393+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:26.161393+0000","last_peered":"2024-09-10T16:02:26.161393+0000","last_clean":"2024-09-10T16:02:26.161393+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:26.161393+0000","last_undegraded":"2024-09-10T16:02:26.161393+0000","last_fullsized":"2024-09-10T16:02:26.161393+0000","mapping_epoch":49,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1040384,"num_objects":254,"num_object_clones":0,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18505,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20213","reported_seq":20636,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.038662+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:26.038662+0000","last_peered":"2024-09-10T16:02:26.038662+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:26.038662+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":995328,"num_objects":243,"num_object_clones":0,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18082,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3443}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20608","reported_seq":21455,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.658129+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:26.658129+0000","last_peered":"2024-09-10T16:02:26.658129+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:26.658129+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1007616,"num_objects":246,"num_object_clones":0,"num_object_copies":492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17800,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20463","reported_seq":21219,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.469937+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:26.469937+0000","last_peered":"2024-09-10T16:02:26.469937+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:26.469937+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1044480,"num_objects":255,"num_object_clones":0,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16881,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20509","reported_seq":21341,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.577348+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:26.577348+0000","last_peered":"2024-09-10T16:02:26.577348+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:26.577348+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052672,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17115,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20833","reported_seq":21599,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:26.571876+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:26.571876+0000","last_peered":"2024-09-10T16:02:26.571876+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:26.571876+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1085923,"num_objects":266,"num_object_clones":0,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17652,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":16474619,"num_objects":4024,"num_object_clones":0,"num_object_copies":8048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4024,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":289026,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16155,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":223080448,"data_stored":223003657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160505,"ondisk_log_size":160505,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481552,"kb_used_data":12596,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93890288,"statfs":{"total":96636764160,"available":96143654912,"internally_reserved":0,"allocated":12898304,"data_stored":12549431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480177838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149828,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547896,"kb_used_data":74788,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93823944,"statfs":{"total":96636764160,"available":96075718656,"internally_reserved":0,"allocated":76582912,"data_stored":76295919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":484437677},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,6,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953599,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498592,"kb_used_data":133772,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93873248,"statfs":{"total":96636764160,"available":96126205952,"internally_reserved":0,"allocated":136982528,"data_stored":136691463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":75649024,"data_stored":75645411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11366400,"data_stored":11300907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136065024,"data_stored":136057339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:32.198 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:33.400 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:33.402 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:33.430 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-10T16:02:32.643362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6431246,"num_objects":1574,"num_object_clones":0,"num_object_copies":3148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1574,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291480,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16247,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160960,"ondisk_log_size":160960,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488108,"kb_used_data":202032,"kb_used_omap":12,"kb_used_meta":1286003,"kb_avail":281627412,"statfs":{"total":289910292480,"available":288386469888,"internally_reserved":0,"allocated":206880768,"data_stored":205953837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1316867077},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,6,4,5,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30429184,"num_objects":-7429,"num_object_clones":0,"num_object_copies":-14858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7429,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005920"},"pg_stats":[{"pgid":"2.f","version":"50'20673","reported_seq":25317,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.725190+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:31.725190+0000","last_peered":"2024-09-10T16:02:31.725190+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:31.725190+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397312,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1695,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20673,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20959","reported_seq":21077,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.161265+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:31.161265+0000","last_peered":"2024-09-10T16:02:31.161265+0000","last_clean":"2024-09-10T16:02:31.161265+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:31.161265+0000","last_undegraded":"2024-09-10T16:02:31.161265+0000","last_fullsized":"2024-09-10T16:02:31.161265+0000","mapping_epoch":36,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":471040,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19295,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20506","reported_seq":20987,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.187978+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:31.187978+0000","last_peered":"2024-09-10T16:02:31.187978+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:31.187978+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":503832,"num_objects":124,"num_object_clones":0,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18774,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20560","reported_seq":20648,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.188559+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:31.188559+0000","last_peered":"2024-09-10T16:02:31.188559+0000","last_clean":"2024-09-10T16:02:31.188559+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:31.188559+0000","last_undegraded":"2024-09-10T16:02:31.188559+0000","last_fullsized":"2024-09-10T16:02:31.188559+0000","mapping_epoch":36,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":499712,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20560,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20243","reported_seq":20380,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.687032+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:31.687032+0000","last_peered":"2024-09-10T16:02:31.687032+0000","last_clean":"2024-09-10T16:02:31.687032+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:31.687032+0000","last_undegraded":"2024-09-10T16:02:31.687032+0000","last_fullsized":"2024-09-10T16:02:31.687032+0000","mapping_epoch":35,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360448,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17314,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20791","reported_seq":28689,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.505145+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:32.505145+0000","last_peered":"2024-09-10T16:02:32.505145+0000","last_clean":"2024-09-10T16:02:32.505145+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:32.505145+0000","last_undegraded":"2024-09-10T16:02:32.505145+0000","last_fullsized":"2024-09-10T16:02:32.505145+0000","mapping_epoch":47,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311296,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17509,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20191","reported_seq":20962,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.645727+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:31.645727+0000","last_peered":"2024-09-10T16:02:31.645727+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:31.645727+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389120,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16841,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20817","reported_seq":20958,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.700050+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:31.700050+0000","last_peered":"2024-09-10T16:02:31.700050+0000","last_clean":"2024-09-10T16:02:31.700050+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:31.700050+0000","last_undegraded":"2024-09-10T16:02:31.700050+0000","last_fullsized":"2024-09-10T16:02:31.700050+0000","mapping_epoch":35,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397312,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17576,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20656","reported_seq":28794,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.530341+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:32.530341+0000","last_peered":"2024-09-10T16:02:32.530341+0000","last_clean":"2024-09-10T16:02:32.530341+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:32.530341+0000","last_undegraded":"2024-09-10T16:02:32.530341+0000","last_fullsized":"2024-09-10T16:02:32.530341+0000","mapping_epoch":45,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258048,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18633,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20482","reported_seq":21306,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.701744+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:31.701744+0000","last_peered":"2024-09-10T16:02:31.701744+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:31.701744+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17346,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3423}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20693","reported_seq":28503,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.188251+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:31.188251+0000","last_peered":"2024-09-10T16:02:31.188251+0000","last_clean":"2024-09-10T16:02:31.188251+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:31.188251+0000","last_undegraded":"2024-09-10T16:02:31.188251+0000","last_fullsized":"2024-09-10T16:02:31.188251+0000","mapping_epoch":49,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":376832,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18667,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20357","reported_seq":20780,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.078879+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:31.078879+0000","last_peered":"2024-09-10T16:02:31.078879+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:31.078879+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":405504,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18226,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3587}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20759","reported_seq":21606,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.670643+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:31.670643+0000","last_peered":"2024-09-10T16:02:31.670643+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:31.670643+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389120,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17951,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20615","reported_seq":21371,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.663642+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:31.663642+0000","last_peered":"2024-09-10T16:02:31.663642+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:31.663642+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421888,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17033,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20657","reported_seq":21489,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.632515+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:31.632515+0000","last_peered":"2024-09-10T16:02:31.632515+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:31.632515+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446464,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17263,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20998","reported_seq":21765,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.661448+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:31.661448+0000","last_peered":"2024-09-10T16:02:31.661448+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:31.661448+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410083,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17817,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3593}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":6431227,"num_objects":1572,"num_object_clones":0,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291478,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16245,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202924032,"data_stored":202847241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160957,"ondisk_log_size":160957,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626548,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478396,"kb_used_data":5292,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93893444,"statfs":{"total":96636764160,"available":96146886656,"internally_reserved":0,"allocated":5419008,"data_stored":5070135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484437678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149829,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528300,"kb_used_data":67672,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93843540,"statfs":{"total":96636764160,"available":96095784960,"internally_reserved":0,"allocated":69296128,"data_stored":69009135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":471658157},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953600,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481412,"kb_used_data":129068,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93890428,"statfs":{"total":96636764160,"available":96143798272,"internally_reserved":0,"allocated":132165632,"data_stored":131874567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68112384,"data_stored":68108771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3530752,"data_stored":3465259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":131280896,"data_stored":131273211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:33.432 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:34.003 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:34.004 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:34.032 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-10T16:02:32.643362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6431246,"num_objects":1574,"num_object_clones":0,"num_object_copies":3148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1574,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291480,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16247,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160960,"ondisk_log_size":160960,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488108,"kb_used_data":202032,"kb_used_omap":12,"kb_used_meta":1286003,"kb_avail":281627412,"statfs":{"total":289910292480,"available":288386469888,"internally_reserved":0,"allocated":206880768,"data_stored":205953837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1316867077},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,6,4,5,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30429184,"num_objects":-7429,"num_object_clones":0,"num_object_copies":-14858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7429,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005920"},"pg_stats":[{"pgid":"2.f","version":"50'20673","reported_seq":25317,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.725190+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:31.725190+0000","last_peered":"2024-09-10T16:02:31.725190+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:31.725190+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397312,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1695,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20673,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20959","reported_seq":21077,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.161265+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:31.161265+0000","last_peered":"2024-09-10T16:02:31.161265+0000","last_clean":"2024-09-10T16:02:31.161265+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:31.161265+0000","last_undegraded":"2024-09-10T16:02:31.161265+0000","last_fullsized":"2024-09-10T16:02:31.161265+0000","mapping_epoch":36,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":471040,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19295,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20506","reported_seq":20987,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.187978+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:31.187978+0000","last_peered":"2024-09-10T16:02:31.187978+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:31.187978+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":503832,"num_objects":124,"num_object_clones":0,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18774,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20560","reported_seq":20648,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.188559+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:31.188559+0000","last_peered":"2024-09-10T16:02:31.188559+0000","last_clean":"2024-09-10T16:02:31.188559+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:31.188559+0000","last_undegraded":"2024-09-10T16:02:31.188559+0000","last_fullsized":"2024-09-10T16:02:31.188559+0000","mapping_epoch":36,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":499712,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20560,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20243","reported_seq":20380,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.687032+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:31.687032+0000","last_peered":"2024-09-10T16:02:31.687032+0000","last_clean":"2024-09-10T16:02:31.687032+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:31.687032+0000","last_undegraded":"2024-09-10T16:02:31.687032+0000","last_fullsized":"2024-09-10T16:02:31.687032+0000","mapping_epoch":35,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360448,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17314,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20791","reported_seq":28689,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.505145+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:32.505145+0000","last_peered":"2024-09-10T16:02:32.505145+0000","last_clean":"2024-09-10T16:02:32.505145+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:32.505145+0000","last_undegraded":"2024-09-10T16:02:32.505145+0000","last_fullsized":"2024-09-10T16:02:32.505145+0000","mapping_epoch":47,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311296,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17509,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20191","reported_seq":20962,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.645727+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:31.645727+0000","last_peered":"2024-09-10T16:02:31.645727+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:31.645727+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389120,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16841,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20817","reported_seq":20958,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.700050+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:31.700050+0000","last_peered":"2024-09-10T16:02:31.700050+0000","last_clean":"2024-09-10T16:02:31.700050+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:31.700050+0000","last_undegraded":"2024-09-10T16:02:31.700050+0000","last_fullsized":"2024-09-10T16:02:31.700050+0000","mapping_epoch":35,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397312,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17576,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20656","reported_seq":28794,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.530341+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:32.530341+0000","last_peered":"2024-09-10T16:02:32.530341+0000","last_clean":"2024-09-10T16:02:32.530341+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:32.530341+0000","last_undegraded":"2024-09-10T16:02:32.530341+0000","last_fullsized":"2024-09-10T16:02:32.530341+0000","mapping_epoch":45,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258048,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18633,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20482","reported_seq":21306,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.701744+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:31.701744+0000","last_peered":"2024-09-10T16:02:31.701744+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:31.701744+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17346,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3423}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20693","reported_seq":28503,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.188251+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:31.188251+0000","last_peered":"2024-09-10T16:02:31.188251+0000","last_clean":"2024-09-10T16:02:31.188251+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:31.188251+0000","last_undegraded":"2024-09-10T16:02:31.188251+0000","last_fullsized":"2024-09-10T16:02:31.188251+0000","mapping_epoch":49,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":376832,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18667,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20357","reported_seq":20780,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.078879+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:31.078879+0000","last_peered":"2024-09-10T16:02:31.078879+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:31.078879+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":405504,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18226,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3587}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20759","reported_seq":21606,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.670643+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:31.670643+0000","last_peered":"2024-09-10T16:02:31.670643+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:31.670643+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389120,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17951,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20615","reported_seq":21371,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.663642+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:31.663642+0000","last_peered":"2024-09-10T16:02:31.663642+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:31.663642+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421888,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17033,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20657","reported_seq":21489,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.632515+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:31.632515+0000","last_peered":"2024-09-10T16:02:31.632515+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:31.632515+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446464,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17263,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20998","reported_seq":21765,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.661448+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:31.661448+0000","last_peered":"2024-09-10T16:02:31.661448+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:31.661448+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410083,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17817,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3593}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":6431227,"num_objects":1572,"num_object_clones":0,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291478,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16245,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202924032,"data_stored":202847241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160957,"ondisk_log_size":160957,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626548,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478396,"kb_used_data":5292,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93893444,"statfs":{"total":96636764160,"available":96146886656,"internally_reserved":0,"allocated":5419008,"data_stored":5070135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484437678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149829,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528300,"kb_used_data":67672,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93843540,"statfs":{"total":96636764160,"available":96095784960,"internally_reserved":0,"allocated":69296128,"data_stored":69009135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":471658157},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953600,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481412,"kb_used_data":129068,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93890428,"statfs":{"total":96636764160,"available":96143798272,"internally_reserved":0,"allocated":132165632,"data_stored":131874567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68112384,"data_stored":68108771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3530752,"data_stored":3465259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":131280896,"data_stored":131273211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:34.033 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:34.033 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'20673", 'reported_seq': 25317, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:31.725190+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:02:31.725190+0000', 'last_peered': '2024-09-10T16:02:31.725190+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:02:31.725190+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 397312, 'num_objects': 97, 'num_object_clones': 0, 'num_object_copies': 194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1695, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 97, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20673, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2278, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1598}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:34.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:34.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20506", 'reported_seq': 20987, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:31.187978+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:31.187978+0000', 'last_peered': '2024-09-10T16:02:31.187978+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:31.187978+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 503832, 'num_objects': 124, 'num_object_clones': 0, 'num_object_copies': 248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 124, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18774, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3505}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:34.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:34.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20191", 'reported_seq': 20962, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:31.645727+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:31.645727+0000', 'last_peered': '2024-09-10T16:02:31.645727+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:31.645727+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10100", 'ondisk_log_start': "46'10100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 389120, 'num_objects': 95, 'num_object_clones': 0, 'num_object_copies': 190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 95, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16841, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3429}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:34.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:34.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20482", 'reported_seq': 21306, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:31.701744+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:31.701744+0000', 'last_peered': '2024-09-10T16:02:31.701744+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:31.701744+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10400", 'ondisk_log_start': "46'10400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 393216, 'num_objects': 96, 'num_object_clones': 0, 'num_object_copies': 192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 96, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17346, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3423}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:34.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:34.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'20357", 'reported_seq': 20780, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:31.078879+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:02:31.078879+0000', 'last_peered': '2024-09-10T16:02:31.078879+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:02:31.078879+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'10300", 'ondisk_log_start': "46'10300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 405504, 'num_objects': 99, 'num_object_clones': 0, 'num_object_copies': 198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 99, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18226, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3587}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:34.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:34.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20759", 'reported_seq': 21606, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:31.670643+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:31.670643+0000', 'last_peered': '2024-09-10T16:02:31.670643+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:31.670643+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 389120, 'num_objects': 95, 'num_object_clones': 0, 'num_object_copies': 190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 95, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17951, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3383}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:34.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:34.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20615", 'reported_seq': 21371, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:31.663642+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:31.663642+0000', 'last_peered': '2024-09-10T16:02:31.663642+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:31.663642+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 421888, 'num_objects': 103, 'num_object_clones': 0, 'num_object_copies': 206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 103, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17033, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:34.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:34.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20657", 'reported_seq': 21489, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:31.632515+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:31.632515+0000', 'last_peered': '2024-09-10T16:02:31.632515+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:31.632515+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 446464, 'num_objects': 109, 'num_object_clones': 0, 'num_object_copies': 218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 109, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17263, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3427}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:34.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:34.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'20998", 'reported_seq': 21765, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:31.661448+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:31.661448+0000', 'last_peered': '2024-09-10T16:02:31.661448+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:31.661448+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'10900", 'ondisk_log_start': "46'10900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 410083, 'num_objects': 101, 'num_object_clones': 0, 'num_object_copies': 202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17817, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3593}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:34.035 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:34.720 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:34.720 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30403/3148 objects degraded (965.788%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":680,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":1574,"data_bytes":6431246,"bytes_used":1523822592,"bytes_avail":288386469888,"bytes_total":289910292480,"degraded_objects":30403,"degraded_total":3148,"degraded_ratio":9.6578780177890717,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":273,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":618},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:34.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:34.746 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:35.136 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Removed 165722 objects 2024-09-10T16:02:35.136 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi121.stdout:Clean up completed and total clean up time :357.296 2024-09-10T16:02:35.138 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:35.138 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:35.153 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-10T16:02:34.644040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6431246,"num_objects":1574,"num_object_clones":0,"num_object_copies":3148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1574,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291480,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16247,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160960,"ondisk_log_size":160960,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488108,"kb_used_data":202032,"kb_used_omap":12,"kb_used_meta":1286003,"kb_avail":281627412,"statfs":{"total":289910292480,"available":288386469888,"internally_reserved":0,"allocated":206880768,"data_stored":205953837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1316867077},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,6,4,5,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20201472,"num_objects":-4932,"num_object_clones":0,"num_object_copies":-9864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-178,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4932,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005475"},"pg_stats":[{"pgid":"2.f","version":"50'20673","reported_seq":25317,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.725190+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:31.725190+0000","last_peered":"2024-09-10T16:02:31.725190+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:31.725190+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397312,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1695,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20673,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'20959","reported_seq":21077,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.161265+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:31.161265+0000","last_peered":"2024-09-10T16:02:31.161265+0000","last_clean":"2024-09-10T16:02:31.161265+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:31.161265+0000","last_undegraded":"2024-09-10T16:02:31.161265+0000","last_fullsized":"2024-09-10T16:02:31.161265+0000","mapping_epoch":36,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":471040,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19295,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20506","reported_seq":20987,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.187978+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:31.187978+0000","last_peered":"2024-09-10T16:02:31.187978+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:31.187978+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":503832,"num_objects":124,"num_object_clones":0,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18774,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20560","reported_seq":20648,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.188559+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:31.188559+0000","last_peered":"2024-09-10T16:02:31.188559+0000","last_clean":"2024-09-10T16:02:31.188559+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:31.188559+0000","last_undegraded":"2024-09-10T16:02:31.188559+0000","last_fullsized":"2024-09-10T16:02:31.188559+0000","mapping_epoch":36,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":499712,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20560,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20243","reported_seq":20380,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.687032+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:31.687032+0000","last_peered":"2024-09-10T16:02:31.687032+0000","last_clean":"2024-09-10T16:02:31.687032+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:31.687032+0000","last_undegraded":"2024-09-10T16:02:31.687032+0000","last_fullsized":"2024-09-10T16:02:31.687032+0000","mapping_epoch":35,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360448,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17314,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20791","reported_seq":28689,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.505145+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:32.505145+0000","last_peered":"2024-09-10T16:02:32.505145+0000","last_clean":"2024-09-10T16:02:32.505145+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:32.505145+0000","last_undegraded":"2024-09-10T16:02:32.505145+0000","last_fullsized":"2024-09-10T16:02:32.505145+0000","mapping_epoch":47,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311296,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17509,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20191","reported_seq":20962,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.645727+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:31.645727+0000","last_peered":"2024-09-10T16:02:31.645727+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:31.645727+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389120,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16841,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20817","reported_seq":20958,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.700050+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:31.700050+0000","last_peered":"2024-09-10T16:02:31.700050+0000","last_clean":"2024-09-10T16:02:31.700050+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:31.700050+0000","last_undegraded":"2024-09-10T16:02:31.700050+0000","last_fullsized":"2024-09-10T16:02:31.700050+0000","mapping_epoch":35,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397312,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17576,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20656","reported_seq":28794,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.530341+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:32.530341+0000","last_peered":"2024-09-10T16:02:32.530341+0000","last_clean":"2024-09-10T16:02:32.530341+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:32.530341+0000","last_undegraded":"2024-09-10T16:02:32.530341+0000","last_fullsized":"2024-09-10T16:02:32.530341+0000","mapping_epoch":45,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258048,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18633,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20482","reported_seq":21306,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.701744+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:31.701744+0000","last_peered":"2024-09-10T16:02:31.701744+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:31.701744+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17346,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3423}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20693","reported_seq":28503,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.188251+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:31.188251+0000","last_peered":"2024-09-10T16:02:31.188251+0000","last_clean":"2024-09-10T16:02:31.188251+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:31.188251+0000","last_undegraded":"2024-09-10T16:02:31.188251+0000","last_fullsized":"2024-09-10T16:02:31.188251+0000","mapping_epoch":49,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":376832,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18667,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20357","reported_seq":20780,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.078879+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:31.078879+0000","last_peered":"2024-09-10T16:02:31.078879+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:31.078879+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":405504,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18226,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3587}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20759","reported_seq":21606,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.670643+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:31.670643+0000","last_peered":"2024-09-10T16:02:31.670643+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:31.670643+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389120,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17951,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20615","reported_seq":21371,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.663642+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:31.663642+0000","last_peered":"2024-09-10T16:02:31.663642+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:31.663642+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421888,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17033,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20657","reported_seq":21489,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.632515+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:31.632515+0000","last_peered":"2024-09-10T16:02:31.632515+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:31.632515+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446464,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17263,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20998","reported_seq":21765,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.661448+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:31.661448+0000","last_peered":"2024-09-10T16:02:31.661448+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:31.661448+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410083,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17817,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3593}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":6431227,"num_objects":1572,"num_object_clones":0,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291478,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16245,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202924032,"data_stored":202847241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160957,"ondisk_log_size":160957,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626548,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478396,"kb_used_data":5292,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93893444,"statfs":{"total":96636764160,"available":96146886656,"internally_reserved":0,"allocated":5419008,"data_stored":5070135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484437678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149829,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528300,"kb_used_data":67672,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93843540,"statfs":{"total":96636764160,"available":96095784960,"internally_reserved":0,"allocated":69296128,"data_stored":69009135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":471658157},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953600,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481412,"kb_used_data":129068,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93890428,"statfs":{"total":96636764160,"available":96143798272,"internally_reserved":0,"allocated":132165632,"data_stored":131874567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68112384,"data_stored":68108771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3530752,"data_stored":3465259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":131280896,"data_stored":131273211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:38.155 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:38.476 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:38.477 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:38.489 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-10T16:02:36.644467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4174326,"num_objects":1022,"num_object_clones":0,"num_object_copies":2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":292032,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16247,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161012,"ondisk_log_size":161012,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488280,"kb_used_data":198052,"kb_used_omap":12,"kb_used_meta":1290163,"kb_avail":281627240,"statfs":{"total":289910292480,"available":288386293760,"internally_reserved":0,"allocated":202805248,"data_stored":201882389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1321126917},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,5,5,9,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22458392,"num_objects":-5484,"num_object_clones":0,"num_object_copies":-10968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-178,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5484,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005236"},"pg_stats":[{"pgid":"2.f","version":"50'20673","reported_seq":25317,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.725190+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:31.725190+0000","last_peered":"2024-09-10T16:02:31.725190+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:31.725190+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397312,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1695,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20673,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21192,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.095187+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:35.095187+0000","last_peered":"2024-09-10T16:02:35.095187+0000","last_clean":"2024-09-10T16:02:35.095187+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:35.095187+0000","last_undegraded":"2024-09-10T16:02:35.095187+0000","last_fullsized":"2024-09-10T16:02:35.095187+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21111,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.122572+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:35.122572+0000","last_peered":"2024-09-10T16:02:35.122572+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:35.122572+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20772,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.128764+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:35.128764+0000","last_peered":"2024-09-10T16:02:35.128764+0000","last_clean":"2024-09-10T16:02:35.128764+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:35.128764+0000","last_undegraded":"2024-09-10T16:02:35.128764+0000","last_fullsized":"2024-09-10T16:02:35.128764+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20243","reported_seq":20380,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.687032+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:31.687032+0000","last_peered":"2024-09-10T16:02:31.687032+0000","last_clean":"2024-09-10T16:02:31.687032+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:31.687032+0000","last_undegraded":"2024-09-10T16:02:31.687032+0000","last_fullsized":"2024-09-10T16:02:31.687032+0000","mapping_epoch":35,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360448,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17314,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20791","reported_seq":28689,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.505145+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:32.505145+0000","last_peered":"2024-09-10T16:02:32.505145+0000","last_clean":"2024-09-10T16:02:32.505145+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:32.505145+0000","last_undegraded":"2024-09-10T16:02:32.505145+0000","last_fullsized":"2024-09-10T16:02:32.505145+0000","mapping_epoch":47,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311296,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17509,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20191","reported_seq":20962,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.645727+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:31.645727+0000","last_peered":"2024-09-10T16:02:31.645727+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:31.645727+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10100","ondisk_log_start":"46'10100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389120,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16841,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20817","reported_seq":20958,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:31.700050+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:31.700050+0000","last_peered":"2024-09-10T16:02:31.700050+0000","last_clean":"2024-09-10T16:02:31.700050+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:31.700050+0000","last_undegraded":"2024-09-10T16:02:31.700050+0000","last_fullsized":"2024-09-10T16:02:31.700050+0000","mapping_epoch":35,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397312,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17576,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20656","reported_seq":28794,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:32.530341+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:32.530341+0000","last_peered":"2024-09-10T16:02:32.530341+0000","last_clean":"2024-09-10T16:02:32.530341+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:32.530341+0000","last_undegraded":"2024-09-10T16:02:32.530341+0000","last_fullsized":"2024-09-10T16:02:32.530341+0000","mapping_epoch":45,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258048,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18633,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20482","reported_seq":21306,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.701744+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:31.701744+0000","last_peered":"2024-09-10T16:02:31.701744+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:31.701744+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17346,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3423}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949257+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949257+0000","last_peered":"2024-09-10T16:00:38.949257+0000","last_clean":"2024-09-10T16:00:38.949257+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949257+0000","last_undegraded":"2024-09-10T16:00:38.949257+0000","last_fullsized":"2024-09-10T16:00:38.949257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28595,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.077148+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:35.077148+0000","last_peered":"2024-09-10T16:02:35.077148+0000","last_clean":"2024-09-10T16:02:35.077148+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:35.077148+0000","last_undegraded":"2024-09-10T16:02:35.077148+0000","last_fullsized":"2024-09-10T16:02:35.077148+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":20879,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.114956+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:35.114956+0000","last_peered":"2024-09-10T16:02:35.114956+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:35.114956+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3686}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20759","reported_seq":21606,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.670643+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:31.670643+0000","last_peered":"2024-09-10T16:02:31.670643+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:31.670643+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389120,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17951,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20615","reported_seq":21371,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.663642+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:31.663642+0000","last_peered":"2024-09-10T16:02:31.663642+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:31.663642+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421888,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17033,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20657","reported_seq":21489,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.632515+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:31.632515+0000","last_peered":"2024-09-10T16:02:31.632515+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:31.632515+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446464,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17263,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'20998","reported_seq":21765,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:31.661448+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:31.661448+0000","last_peered":"2024-09-10T16:02:31.661448+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:31.661448+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410083,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17817,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3593}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4174307,"num_objects":1020,"num_object_clones":0,"num_object_copies":2040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":292030,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16245,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":199311360,"data_stored":199238641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161009,"ondisk_log_size":161009,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626548,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478396,"kb_used_data":5292,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93893444,"statfs":{"total":96636764160,"available":96146886656,"internally_reserved":0,"allocated":5419008,"data_stored":5070135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484437678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149829,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528300,"kb_used_data":67672,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93843540,"statfs":{"total":96636764160,"available":96095784960,"internally_reserved":0,"allocated":69296128,"data_stored":69009135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":471658157},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953601,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481584,"kb_used_data":125088,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890256,"statfs":{"total":96636764160,"available":96143622144,"internally_reserved":0,"allocated":128090112,"data_stored":127803119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68112384,"data_stored":68108771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3530752,"data_stored":3465259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:38.491 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:38.789 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:38.802 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-10T16:02:38.644803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16589,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473848,"kb_used_data":187784,"kb_used_omap":12,"kb_used_meta":1286003,"kb_avail":281641672,"statfs":{"total":289910292480,"available":288401072128,"internally_reserved":0,"allocated":192290816,"data_stored":191372029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1316867077},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23404568,"num_objects":-5715,"num_object_clones":0,"num_object_copies":-11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5715,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004851"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":26098,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:36.725835+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:36.725835+0000","last_peered":"2024-09-10T16:02:36.725835+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:36.725835+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2620,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1353}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21192,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.095187+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:35.095187+0000","last_peered":"2024-09-10T16:02:35.095187+0000","last_clean":"2024-09-10T16:02:35.095187+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:35.095187+0000","last_undegraded":"2024-09-10T16:02:35.095187+0000","last_fullsized":"2024-09-10T16:02:35.095187+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21111,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.122572+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:35.122572+0000","last_peered":"2024-09-10T16:02:35.122572+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:35.122572+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20772,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.128764+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:35.128764+0000","last_peered":"2024-09-10T16:02:35.128764+0000","last_clean":"2024-09-10T16:02:35.128764+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:35.128764+0000","last_undegraded":"2024-09-10T16:02:35.128764+0000","last_fullsized":"2024-09-10T16:02:35.128764+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20468,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086998+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:35.086998+0000","last_peered":"2024-09-10T16:02:35.086998+0000","last_clean":"2024-09-10T16:02:35.086998+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:35.086998+0000","last_undegraded":"2024-09-10T16:02:35.086998+0000","last_fullsized":"2024-09-10T16:02:35.086998+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.097281+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:35.097281+0000","last_peered":"2024-09-10T16:02:35.097281+0000","last_clean":"2024-09-10T16:02:35.097281+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:35.097281+0000","last_undegraded":"2024-09-10T16:02:35.097281+0000","last_fullsized":"2024-09-10T16:02:35.097281+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21057,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.076561+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:35.076561+0000","last_peered":"2024-09-10T16:02:35.076561+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:35.076561+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21056,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.127033+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:35.127033+0000","last_peered":"2024-09-10T16:02:35.127033+0000","last_clean":"2024-09-10T16:02:35.127033+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:35.127033+0000","last_undegraded":"2024-09-10T16:02:35.127033+0000","last_fullsized":"2024-09-10T16:02:35.127033+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28857,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086419+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:35.086419+0000","last_peered":"2024-09-10T16:02:35.086419+0000","last_clean":"2024-09-10T16:02:35.086419+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:35.086419+0000","last_undegraded":"2024-09-10T16:02:35.086419+0000","last_fullsized":"2024-09-10T16:02:35.086419+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21404,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.102557+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:35.102557+0000","last_peered":"2024-09-10T16:02:35.102557+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:35.102557+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":112,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949302+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949302+0000","last_peered":"2024-09-10T16:00:38.949302+0000","last_clean":"2024-09-10T16:00:38.949302+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949302+0000","last_undegraded":"2024-09-10T16:00:38.949302+0000","last_fullsized":"2024-09-10T16:00:38.949302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28595,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.077148+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:35.077148+0000","last_peered":"2024-09-10T16:02:35.077148+0000","last_clean":"2024-09-10T16:02:35.077148+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:35.077148+0000","last_undegraded":"2024-09-10T16:02:35.077148+0000","last_fullsized":"2024-09-10T16:02:35.077148+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":20879,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.114956+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:35.114956+0000","last_peered":"2024-09-10T16:02:35.114956+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:35.114956+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3686}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21702,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.101762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:35.101762+0000","last_peered":"2024-09-10T16:02:35.101762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:35.101762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21474,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.979249+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:34.979249+0000","last_peered":"2024-09-10T16:02:34.979249+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:34.979249+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21598,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.105714+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:35.105714+0000","last_peered":"2024-09-10T16:02:35.105714+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:35.105714+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21865,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.135045+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:35.135045+0000","last_peered":"2024-09-10T16:02:35.135045+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:35.135045+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16587,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191025152,"data_stored":190956505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626549,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465264,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906576,"statfs":{"total":96636764160,"available":96160333824,"internally_reserved":0,"allocated":491520,"data_stored":146719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475917998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149830,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527000,"kb_used_data":62216,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844840,"statfs":{"total":96636764160,"available":96097116160,"internally_reserved":0,"allocated":63709184,"data_stored":63422191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953601,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481584,"kb_used_data":125088,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890256,"statfs":{"total":96636764160,"available":96143622144,"internally_reserved":0,"allocated":128090112,"data_stored":127803119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":8211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:38.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:38.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'20770", 'reported_seq': 26098, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:36.725835+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:02:36.725835+0000', 'last_peered': '2024-09-10T16:02:36.725835+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:02:36.725835+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1353, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20770, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2620, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1353}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:38.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:38.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 21111, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.122572+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:35.122572+0000', 'last_peered': '2024-09-10T16:02:35.122572+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:35.122572+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:38.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:38.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21057, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.076561+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:35.076561+0000', 'last_peered': '2024-09-10T16:02:35.076561+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:35.076561+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:38.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:38.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21404, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.102557+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:35.102557+0000', 'last_peered': '2024-09-10T16:02:35.102557+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:35.102557+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:38.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:38.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'20456", 'reported_seq': 20879, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.114956+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:02:35.114956+0000', 'last_peered': '2024-09-10T16:02:35.114956+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:02:35.114956+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'10400", 'ondisk_log_start': "46'10400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18325, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3686}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:38.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:38.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20854", 'reported_seq': 21702, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.101762+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:35.101762+0000', 'last_peered': '2024-09-10T16:02:35.101762+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:35.101762+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10800", 'ondisk_log_start': "46'10800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18046, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3478}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:38.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:38.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21474, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:34.979249+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:34.979249+0000', 'last_peered': '2024-09-10T16:02:34.979249+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:34.979249+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:38.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:38.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21598, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.105714+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:35.105714+0000', 'last_peered': '2024-09-10T16:02:35.105714+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:35.105714+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:38.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:38.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21865, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.135045+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:35.135045+0000', 'last_peered': '2024-09-10T16:02:35.135045+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:35.135045+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:38.805 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:39.194 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:39.195 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30061/6 objects degraded (501016.667%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":685,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1509220352,"bytes_avail":288401072128,"bytes_total":289910292480,"degraded_objects":30061,"degraded_total":6,"degraded_ratio":5010.166666666667,"recovering_objects_per_sec":43,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":520,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":476},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:39.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:39.208 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:39.512 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:39.524 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-10T16:02:38.644803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16589,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473848,"kb_used_data":187784,"kb_used_omap":12,"kb_used_meta":1286003,"kb_avail":281641672,"statfs":{"total":289910292480,"available":288401072128,"internally_reserved":0,"allocated":192290816,"data_stored":191372029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1316867077},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23404568,"num_objects":-5715,"num_object_clones":0,"num_object_copies":-11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5715,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004851"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":26098,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:36.725835+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:36.725835+0000","last_peered":"2024-09-10T16:02:36.725835+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:36.725835+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2620,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1353}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21192,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.095187+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:35.095187+0000","last_peered":"2024-09-10T16:02:35.095187+0000","last_clean":"2024-09-10T16:02:35.095187+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:35.095187+0000","last_undegraded":"2024-09-10T16:02:35.095187+0000","last_fullsized":"2024-09-10T16:02:35.095187+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21111,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.122572+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:35.122572+0000","last_peered":"2024-09-10T16:02:35.122572+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:35.122572+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20772,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.128764+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:35.128764+0000","last_peered":"2024-09-10T16:02:35.128764+0000","last_clean":"2024-09-10T16:02:35.128764+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:35.128764+0000","last_undegraded":"2024-09-10T16:02:35.128764+0000","last_fullsized":"2024-09-10T16:02:35.128764+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20468,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086998+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:35.086998+0000","last_peered":"2024-09-10T16:02:35.086998+0000","last_clean":"2024-09-10T16:02:35.086998+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:35.086998+0000","last_undegraded":"2024-09-10T16:02:35.086998+0000","last_fullsized":"2024-09-10T16:02:35.086998+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.097281+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:35.097281+0000","last_peered":"2024-09-10T16:02:35.097281+0000","last_clean":"2024-09-10T16:02:35.097281+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:35.097281+0000","last_undegraded":"2024-09-10T16:02:35.097281+0000","last_fullsized":"2024-09-10T16:02:35.097281+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21057,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.076561+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:35.076561+0000","last_peered":"2024-09-10T16:02:35.076561+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:35.076561+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21056,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.127033+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:35.127033+0000","last_peered":"2024-09-10T16:02:35.127033+0000","last_clean":"2024-09-10T16:02:35.127033+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:35.127033+0000","last_undegraded":"2024-09-10T16:02:35.127033+0000","last_fullsized":"2024-09-10T16:02:35.127033+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28857,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086419+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:35.086419+0000","last_peered":"2024-09-10T16:02:35.086419+0000","last_clean":"2024-09-10T16:02:35.086419+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:35.086419+0000","last_undegraded":"2024-09-10T16:02:35.086419+0000","last_fullsized":"2024-09-10T16:02:35.086419+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21404,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.102557+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:35.102557+0000","last_peered":"2024-09-10T16:02:35.102557+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:35.102557+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":112,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949302+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949302+0000","last_peered":"2024-09-10T16:00:38.949302+0000","last_clean":"2024-09-10T16:00:38.949302+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949302+0000","last_undegraded":"2024-09-10T16:00:38.949302+0000","last_fullsized":"2024-09-10T16:00:38.949302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28595,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.077148+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:35.077148+0000","last_peered":"2024-09-10T16:02:35.077148+0000","last_clean":"2024-09-10T16:02:35.077148+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:35.077148+0000","last_undegraded":"2024-09-10T16:02:35.077148+0000","last_fullsized":"2024-09-10T16:02:35.077148+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":20879,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.114956+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:35.114956+0000","last_peered":"2024-09-10T16:02:35.114956+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:35.114956+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3686}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21702,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.101762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:35.101762+0000","last_peered":"2024-09-10T16:02:35.101762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:35.101762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21474,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.979249+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:34.979249+0000","last_peered":"2024-09-10T16:02:34.979249+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:34.979249+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21598,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.105714+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:35.105714+0000","last_peered":"2024-09-10T16:02:35.105714+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:35.105714+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21865,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.135045+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:35.135045+0000","last_peered":"2024-09-10T16:02:35.135045+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:35.135045+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16587,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191025152,"data_stored":190956505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626549,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465264,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906576,"statfs":{"total":96636764160,"available":96160333824,"internally_reserved":0,"allocated":491520,"data_stored":146719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475917998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149830,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527000,"kb_used_data":62216,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844840,"statfs":{"total":96636764160,"available":96097116160,"internally_reserved":0,"allocated":63709184,"data_stored":63422191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953601,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481584,"kb_used_data":125088,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890256,"statfs":{"total":96636764160,"available":96143622144,"internally_reserved":0,"allocated":128090112,"data_stored":127803119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":8211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:42.527 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:42.850 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:42.850 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:42.862 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-10T16:02:42.645547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17572,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482168,"kb_used_data":187784,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281633352,"statfs":{"total":289910292480,"available":288392552448,"internally_reserved":0,"allocated":192290816,"data_stored":191372029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1325386757},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16474136,"num_objects":-4023,"num_object_clones":0,"num_object_copies":-8046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4023,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1415,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002927"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28064,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:41.727209+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:41.727209+0000","last_peered":"2024-09-10T16:02:41.727209+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:41.727209+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3603,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-370}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21192,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.095187+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:35.095187+0000","last_peered":"2024-09-10T16:02:35.095187+0000","last_clean":"2024-09-10T16:02:35.095187+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:35.095187+0000","last_undegraded":"2024-09-10T16:02:35.095187+0000","last_fullsized":"2024-09-10T16:02:35.095187+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21111,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.122572+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:35.122572+0000","last_peered":"2024-09-10T16:02:35.122572+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:35.122572+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20772,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.128764+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:35.128764+0000","last_peered":"2024-09-10T16:02:35.128764+0000","last_clean":"2024-09-10T16:02:35.128764+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:35.128764+0000","last_undegraded":"2024-09-10T16:02:35.128764+0000","last_fullsized":"2024-09-10T16:02:35.128764+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20468,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086998+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:35.086998+0000","last_peered":"2024-09-10T16:02:35.086998+0000","last_clean":"2024-09-10T16:02:35.086998+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:35.086998+0000","last_undegraded":"2024-09-10T16:02:35.086998+0000","last_fullsized":"2024-09-10T16:02:35.086998+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.097281+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:35.097281+0000","last_peered":"2024-09-10T16:02:35.097281+0000","last_clean":"2024-09-10T16:02:35.097281+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:35.097281+0000","last_undegraded":"2024-09-10T16:02:35.097281+0000","last_fullsized":"2024-09-10T16:02:35.097281+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21057,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.076561+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:35.076561+0000","last_peered":"2024-09-10T16:02:35.076561+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:35.076561+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21056,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.127033+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:35.127033+0000","last_peered":"2024-09-10T16:02:35.127033+0000","last_clean":"2024-09-10T16:02:35.127033+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:35.127033+0000","last_undegraded":"2024-09-10T16:02:35.127033+0000","last_fullsized":"2024-09-10T16:02:35.127033+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28857,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086419+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:35.086419+0000","last_peered":"2024-09-10T16:02:35.086419+0000","last_clean":"2024-09-10T16:02:35.086419+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:35.086419+0000","last_undegraded":"2024-09-10T16:02:35.086419+0000","last_fullsized":"2024-09-10T16:02:35.086419+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21404,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.102557+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:35.102557+0000","last_peered":"2024-09-10T16:02:35.102557+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:35.102557+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":112,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949302+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949302+0000","last_peered":"2024-09-10T16:00:38.949302+0000","last_clean":"2024-09-10T16:00:38.949302+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949302+0000","last_undegraded":"2024-09-10T16:00:38.949302+0000","last_fullsized":"2024-09-10T16:00:38.949302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28595,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.077148+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:35.077148+0000","last_peered":"2024-09-10T16:02:35.077148+0000","last_clean":"2024-09-10T16:02:35.077148+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:35.077148+0000","last_undegraded":"2024-09-10T16:02:35.077148+0000","last_fullsized":"2024-09-10T16:02:35.077148+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":20879,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.114956+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:35.114956+0000","last_peered":"2024-09-10T16:02:35.114956+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:35.114956+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3686}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21702,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.101762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:35.101762+0000","last_peered":"2024-09-10T16:02:35.101762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:35.101762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21474,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.979249+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:34.979249+0000","last_peered":"2024-09-10T16:02:34.979249+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:34.979249+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21598,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.105714+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:35.105714+0000","last_peered":"2024-09-10T16:02:35.105714+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:35.105714+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21865,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.135045+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:35.135045+0000","last_peered":"2024-09-10T16:02:35.135045+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:35.135045+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17570,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191025152,"data_stored":190956505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473584,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898256,"statfs":{"total":96636764160,"available":96151814144,"internally_reserved":0,"allocated":491520,"data_stored":146719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484437678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527000,"kb_used_data":62216,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844840,"statfs":{"total":96636764160,"available":96097116160,"internally_reserved":0,"allocated":63709184,"data_stored":63422191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953602,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481584,"kb_used_data":125088,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890256,"statfs":{"total":96636764160,"available":96143622144,"internally_reserved":0,"allocated":128090112,"data_stored":127803119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":8211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:42.864 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:43.167 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:43.167 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:43.179 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-10T16:02:42.645547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17572,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482168,"kb_used_data":187784,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281633352,"statfs":{"total":289910292480,"available":288392552448,"internally_reserved":0,"allocated":192290816,"data_stored":191372029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1325386757},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16474136,"num_objects":-4023,"num_object_clones":0,"num_object_copies":-8046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4023,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1415,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002927"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28064,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:41.727209+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:41.727209+0000","last_peered":"2024-09-10T16:02:41.727209+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:41.727209+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3603,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-370}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21192,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.095187+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:35.095187+0000","last_peered":"2024-09-10T16:02:35.095187+0000","last_clean":"2024-09-10T16:02:35.095187+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:35.095187+0000","last_undegraded":"2024-09-10T16:02:35.095187+0000","last_fullsized":"2024-09-10T16:02:35.095187+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21111,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.122572+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:35.122572+0000","last_peered":"2024-09-10T16:02:35.122572+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:35.122572+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20772,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.128764+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:35.128764+0000","last_peered":"2024-09-10T16:02:35.128764+0000","last_clean":"2024-09-10T16:02:35.128764+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:35.128764+0000","last_undegraded":"2024-09-10T16:02:35.128764+0000","last_fullsized":"2024-09-10T16:02:35.128764+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20468,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086998+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:35.086998+0000","last_peered":"2024-09-10T16:02:35.086998+0000","last_clean":"2024-09-10T16:02:35.086998+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:35.086998+0000","last_undegraded":"2024-09-10T16:02:35.086998+0000","last_fullsized":"2024-09-10T16:02:35.086998+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.097281+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:35.097281+0000","last_peered":"2024-09-10T16:02:35.097281+0000","last_clean":"2024-09-10T16:02:35.097281+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:35.097281+0000","last_undegraded":"2024-09-10T16:02:35.097281+0000","last_fullsized":"2024-09-10T16:02:35.097281+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21057,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.076561+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:35.076561+0000","last_peered":"2024-09-10T16:02:35.076561+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:35.076561+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21056,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.127033+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:35.127033+0000","last_peered":"2024-09-10T16:02:35.127033+0000","last_clean":"2024-09-10T16:02:35.127033+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:35.127033+0000","last_undegraded":"2024-09-10T16:02:35.127033+0000","last_fullsized":"2024-09-10T16:02:35.127033+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28857,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086419+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:35.086419+0000","last_peered":"2024-09-10T16:02:35.086419+0000","last_clean":"2024-09-10T16:02:35.086419+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:35.086419+0000","last_undegraded":"2024-09-10T16:02:35.086419+0000","last_fullsized":"2024-09-10T16:02:35.086419+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21404,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.102557+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:35.102557+0000","last_peered":"2024-09-10T16:02:35.102557+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:35.102557+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":112,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949302+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949302+0000","last_peered":"2024-09-10T16:00:38.949302+0000","last_clean":"2024-09-10T16:00:38.949302+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949302+0000","last_undegraded":"2024-09-10T16:00:38.949302+0000","last_fullsized":"2024-09-10T16:00:38.949302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28595,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.077148+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:35.077148+0000","last_peered":"2024-09-10T16:02:35.077148+0000","last_clean":"2024-09-10T16:02:35.077148+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:35.077148+0000","last_undegraded":"2024-09-10T16:02:35.077148+0000","last_fullsized":"2024-09-10T16:02:35.077148+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":20879,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.114956+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:35.114956+0000","last_peered":"2024-09-10T16:02:35.114956+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:35.114956+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3686}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21702,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.101762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:35.101762+0000","last_peered":"2024-09-10T16:02:35.101762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:35.101762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21474,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.979249+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:34.979249+0000","last_peered":"2024-09-10T16:02:34.979249+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:34.979249+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21598,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.105714+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:35.105714+0000","last_peered":"2024-09-10T16:02:35.105714+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:35.105714+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21865,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.135045+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:35.135045+0000","last_peered":"2024-09-10T16:02:35.135045+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:35.135045+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17570,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191025152,"data_stored":190956505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473584,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898256,"statfs":{"total":96636764160,"available":96151814144,"internally_reserved":0,"allocated":491520,"data_stored":146719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484437678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527000,"kb_used_data":62216,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844840,"statfs":{"total":96636764160,"available":96097116160,"internally_reserved":0,"allocated":63709184,"data_stored":63422191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953602,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481584,"kb_used_data":125088,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890256,"statfs":{"total":96636764160,"available":96143622144,"internally_reserved":0,"allocated":128090112,"data_stored":127803119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":8211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:43.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:43.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'20770", 'reported_seq': 28064, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:41.727209+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:02:41.727209+0000', 'last_peered': '2024-09-10T16:02:41.727209+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:02:41.727209+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 370, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20770, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3603, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -370}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:43.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 21111, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.122572+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:35.122572+0000', 'last_peered': '2024-09-10T16:02:35.122572+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:35.122572+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:43.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:43.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21057, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.076561+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:35.076561+0000', 'last_peered': '2024-09-10T16:02:35.076561+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:35.076561+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:43.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21404, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.102557+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:35.102557+0000', 'last_peered': '2024-09-10T16:02:35.102557+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:35.102557+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:43.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'20456", 'reported_seq': 20879, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.114956+0000', 'last_change': '2024-09-10T15:53:19.939902+0000', 'last_active': '2024-09-10T16:02:35.114956+0000', 'last_peered': '2024-09-10T16:02:35.114956+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:02:35.114956+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'10400", 'ondisk_log_start': "46'10400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18325, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3686}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:43.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:43.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20854", 'reported_seq': 21702, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.101762+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:35.101762+0000', 'last_peered': '2024-09-10T16:02:35.101762+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:35.101762+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10800", 'ondisk_log_start': "46'10800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18046, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3478}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:43.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21474, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:34.979249+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:34.979249+0000', 'last_peered': '2024-09-10T16:02:34.979249+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:34.979249+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:43.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21598, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.105714+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:35.105714+0000', 'last_peered': '2024-09-10T16:02:35.105714+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:35.105714+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:43.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21865, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.135045+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:35.135045+0000', 'last_peered': '2024-09-10T16:02:35.135045+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:35.135045+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:43.182 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:43.564 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:43.564 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29078/6 objects degraded (484633.333%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":689,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1517740032,"bytes_avail":288392552448,"bytes_total":289910292480,"degraded_objects":29078,"degraded_total":6,"degraded_ratio":4846.333333333333,"recovering_objects_per_sec":117,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1415,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":335},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:43.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:43.580 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:43.890 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:43.891 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:43.909 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-10T16:02:42.645547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17572,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482168,"kb_used_data":187784,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281633352,"statfs":{"total":289910292480,"available":288392552448,"internally_reserved":0,"allocated":192290816,"data_stored":191372029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1325386757},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16474136,"num_objects":-4023,"num_object_clones":0,"num_object_copies":-8046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4023,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1415,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002927"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28064,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:41.727209+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:41.727209+0000","last_peered":"2024-09-10T16:02:41.727209+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:41.727209+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3603,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-370}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21192,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.095187+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:35.095187+0000","last_peered":"2024-09-10T16:02:35.095187+0000","last_clean":"2024-09-10T16:02:35.095187+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:35.095187+0000","last_undegraded":"2024-09-10T16:02:35.095187+0000","last_fullsized":"2024-09-10T16:02:35.095187+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21111,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.122572+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:35.122572+0000","last_peered":"2024-09-10T16:02:35.122572+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:35.122572+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20772,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.128764+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:35.128764+0000","last_peered":"2024-09-10T16:02:35.128764+0000","last_clean":"2024-09-10T16:02:35.128764+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:35.128764+0000","last_undegraded":"2024-09-10T16:02:35.128764+0000","last_fullsized":"2024-09-10T16:02:35.128764+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20468,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086998+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:35.086998+0000","last_peered":"2024-09-10T16:02:35.086998+0000","last_clean":"2024-09-10T16:02:35.086998+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:35.086998+0000","last_undegraded":"2024-09-10T16:02:35.086998+0000","last_fullsized":"2024-09-10T16:02:35.086998+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.097281+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:35.097281+0000","last_peered":"2024-09-10T16:02:35.097281+0000","last_clean":"2024-09-10T16:02:35.097281+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:35.097281+0000","last_undegraded":"2024-09-10T16:02:35.097281+0000","last_fullsized":"2024-09-10T16:02:35.097281+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21057,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.076561+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:35.076561+0000","last_peered":"2024-09-10T16:02:35.076561+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:35.076561+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21056,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.127033+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:35.127033+0000","last_peered":"2024-09-10T16:02:35.127033+0000","last_clean":"2024-09-10T16:02:35.127033+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:35.127033+0000","last_undegraded":"2024-09-10T16:02:35.127033+0000","last_fullsized":"2024-09-10T16:02:35.127033+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28857,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086419+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:35.086419+0000","last_peered":"2024-09-10T16:02:35.086419+0000","last_clean":"2024-09-10T16:02:35.086419+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:35.086419+0000","last_undegraded":"2024-09-10T16:02:35.086419+0000","last_fullsized":"2024-09-10T16:02:35.086419+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.947184+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:00:38.947184+0000","last_peered":"2024-09-10T16:00:38.947184+0000","last_clean":"2024-09-10T16:00:38.947184+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:00:38.947184+0000","last_undegraded":"2024-09-10T16:00:38.947184+0000","last_fullsized":"2024-09-10T16:00:38.947184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21404,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.102557+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:35.102557+0000","last_peered":"2024-09-10T16:02:35.102557+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:35.102557+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":112,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949302+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949302+0000","last_peered":"2024-09-10T16:00:38.949302+0000","last_clean":"2024-09-10T16:00:38.949302+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949302+0000","last_undegraded":"2024-09-10T16:00:38.949302+0000","last_fullsized":"2024-09-10T16:00:38.949302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28595,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.077148+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:35.077148+0000","last_peered":"2024-09-10T16:02:35.077148+0000","last_clean":"2024-09-10T16:02:35.077148+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:35.077148+0000","last_undegraded":"2024-09-10T16:02:35.077148+0000","last_fullsized":"2024-09-10T16:02:35.077148+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":20879,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.114956+0000","last_change":"2024-09-10T15:53:19.939902+0000","last_active":"2024-09-10T16:02:35.114956+0000","last_peered":"2024-09-10T16:02:35.114956+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:35.114956+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3686}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21702,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.101762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:35.101762+0000","last_peered":"2024-09-10T16:02:35.101762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:35.101762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21474,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.979249+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:34.979249+0000","last_peered":"2024-09-10T16:02:34.979249+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:34.979249+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949000+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:00:38.949000+0000","last_peered":"2024-09-10T16:00:38.949000+0000","last_clean":"2024-09-10T16:00:38.949000+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:00:38.949000+0000","last_undegraded":"2024-09-10T16:00:38.949000+0000","last_fullsized":"2024-09-10T16:00:38.949000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21598,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.105714+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:35.105714+0000","last_peered":"2024-09-10T16:02:35.105714+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:35.105714+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21865,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.135045+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:35.135045+0000","last_peered":"2024-09-10T16:02:35.135045+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:35.135045+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17570,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191025152,"data_stored":190956505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473584,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898256,"statfs":{"total":96636764160,"available":96151814144,"internally_reserved":0,"allocated":491520,"data_stored":146719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484437678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527000,"kb_used_data":62216,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844840,"statfs":{"total":96636764160,"available":96097116160,"internally_reserved":0,"allocated":63709184,"data_stored":63422191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953602,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481584,"kb_used_data":125088,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890256,"statfs":{"total":96636764160,"available":96143622144,"internally_reserved":0,"allocated":128090112,"data_stored":127803119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":8211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:46.912 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:47.283 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:47.283 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:47.297 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-10T16:02:46.646326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18042,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482184,"kb_used_data":187800,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281633336,"statfs":{"total":289910292480,"available":288392536064,"internally_reserved":0,"allocated":192307200,"data_stored":191377868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1325386757},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001522"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28064,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:41.727209+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:41.727209+0000","last_peered":"2024-09-10T16:02:41.727209+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:41.727209+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3603,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-370}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20468,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086998+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:35.086998+0000","last_peered":"2024-09-10T16:02:35.086998+0000","last_clean":"2024-09-10T16:02:35.086998+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:35.086998+0000","last_undegraded":"2024-09-10T16:02:35.086998+0000","last_fullsized":"2024-09-10T16:02:35.086998+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.097281+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:35.097281+0000","last_peered":"2024-09-10T16:02:35.097281+0000","last_clean":"2024-09-10T16:02:35.097281+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:35.097281+0000","last_undegraded":"2024-09-10T16:02:35.097281+0000","last_fullsized":"2024-09-10T16:02:35.097281+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21057,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.076561+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:35.076561+0000","last_peered":"2024-09-10T16:02:35.076561+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:35.076561+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21056,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.127033+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:35.127033+0000","last_peered":"2024-09-10T16:02:35.127033+0000","last_clean":"2024-09-10T16:02:35.127033+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:35.127033+0000","last_undegraded":"2024-09-10T16:02:35.127033+0000","last_fullsized":"2024-09-10T16:02:35.127033+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28857,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086419+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:35.086419+0000","last_peered":"2024-09-10T16:02:35.086419+0000","last_clean":"2024-09-10T16:02:35.086419+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:35.086419+0000","last_undegraded":"2024-09-10T16:02:35.086419+0000","last_fullsized":"2024-09-10T16:02:35.086419+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940823+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940823+0000","last_peered":"2024-09-10T16:02:44.940823+0000","last_clean":"2024-09-10T16:02:44.940823+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940823+0000","last_undegraded":"2024-09-10T16:02:44.940823+0000","last_fullsized":"2024-09-10T16:02:44.940823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21404,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.102557+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:35.102557+0000","last_peered":"2024-09-10T16:02:35.102557+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:35.102557+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":112,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949302+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949302+0000","last_peered":"2024-09-10T16:00:38.949302+0000","last_clean":"2024-09-10T16:00:38.949302+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949302+0000","last_undegraded":"2024-09-10T16:00:38.949302+0000","last_fullsized":"2024-09-10T16:00:38.949302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28599,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941068+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941068+0000","last_peered":"2024-09-10T16:02:44.941068+0000","last_clean":"2024-09-10T16:02:44.941068+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941068+0000","last_undegraded":"2024-09-10T16:02:44.941068+0000","last_fullsized":"2024-09-10T16:02:44.941068+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":21824,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:46.198110+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:02:46.198110+0000","last_peered":"2024-09-10T16:02:46.198110+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:46.198110+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":613,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3218}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21702,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.101762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:35.101762+0000","last_peered":"2024-09-10T16:02:35.101762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:35.101762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21474,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.979249+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:34.979249+0000","last_peered":"2024-09-10T16:02:34.979249+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:34.979249+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941733+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941733+0000","last_peered":"2024-09-10T16:02:44.941733+0000","last_clean":"2024-09-10T16:02:44.941733+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941733+0000","last_undegraded":"2024-09-10T16:02:44.941733+0000","last_fullsized":"2024-09-10T16:02:44.941733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21598,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.105714+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:35.105714+0000","last_peered":"2024-09-10T16:02:35.105714+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:35.105714+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21865,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.135045+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:35.135045+0000","last_peered":"2024-09-10T16:02:35.135045+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:35.135045+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18040,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191025152,"data_stored":190956505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473584,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898256,"statfs":{"total":96636764160,"available":96151814144,"internally_reserved":0,"allocated":491520,"data_stored":146719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484437678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527000,"kb_used_data":62216,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844840,"statfs":{"total":96636764160,"available":96097116160,"internally_reserved":0,"allocated":63709184,"data_stored":63422191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953603,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":8211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:47.298 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:47.612 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:47.613 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:47.627 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-10T16:02:46.646326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18042,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482184,"kb_used_data":187800,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281633336,"statfs":{"total":289910292480,"available":288392536064,"internally_reserved":0,"allocated":192307200,"data_stored":191377868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1325386757},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001522"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28064,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:41.727209+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:41.727209+0000","last_peered":"2024-09-10T16:02:41.727209+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:41.727209+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3603,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-370}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20468,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086998+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:35.086998+0000","last_peered":"2024-09-10T16:02:35.086998+0000","last_clean":"2024-09-10T16:02:35.086998+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:35.086998+0000","last_undegraded":"2024-09-10T16:02:35.086998+0000","last_fullsized":"2024-09-10T16:02:35.086998+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.097281+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:35.097281+0000","last_peered":"2024-09-10T16:02:35.097281+0000","last_clean":"2024-09-10T16:02:35.097281+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:35.097281+0000","last_undegraded":"2024-09-10T16:02:35.097281+0000","last_fullsized":"2024-09-10T16:02:35.097281+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21057,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.076561+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:35.076561+0000","last_peered":"2024-09-10T16:02:35.076561+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:35.076561+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21056,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.127033+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:35.127033+0000","last_peered":"2024-09-10T16:02:35.127033+0000","last_clean":"2024-09-10T16:02:35.127033+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:35.127033+0000","last_undegraded":"2024-09-10T16:02:35.127033+0000","last_fullsized":"2024-09-10T16:02:35.127033+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28857,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086419+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:35.086419+0000","last_peered":"2024-09-10T16:02:35.086419+0000","last_clean":"2024-09-10T16:02:35.086419+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:35.086419+0000","last_undegraded":"2024-09-10T16:02:35.086419+0000","last_fullsized":"2024-09-10T16:02:35.086419+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940823+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940823+0000","last_peered":"2024-09-10T16:02:44.940823+0000","last_clean":"2024-09-10T16:02:44.940823+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940823+0000","last_undegraded":"2024-09-10T16:02:44.940823+0000","last_fullsized":"2024-09-10T16:02:44.940823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21404,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.102557+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:35.102557+0000","last_peered":"2024-09-10T16:02:35.102557+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:35.102557+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":112,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949302+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949302+0000","last_peered":"2024-09-10T16:00:38.949302+0000","last_clean":"2024-09-10T16:00:38.949302+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949302+0000","last_undegraded":"2024-09-10T16:00:38.949302+0000","last_fullsized":"2024-09-10T16:00:38.949302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28599,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941068+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941068+0000","last_peered":"2024-09-10T16:02:44.941068+0000","last_clean":"2024-09-10T16:02:44.941068+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941068+0000","last_undegraded":"2024-09-10T16:02:44.941068+0000","last_fullsized":"2024-09-10T16:02:44.941068+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":21824,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:46.198110+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:02:46.198110+0000","last_peered":"2024-09-10T16:02:46.198110+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:46.198110+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":613,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3218}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21702,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.101762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:35.101762+0000","last_peered":"2024-09-10T16:02:35.101762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:35.101762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21474,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.979249+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:34.979249+0000","last_peered":"2024-09-10T16:02:34.979249+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:34.979249+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941733+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941733+0000","last_peered":"2024-09-10T16:02:44.941733+0000","last_clean":"2024-09-10T16:02:44.941733+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941733+0000","last_undegraded":"2024-09-10T16:02:44.941733+0000","last_fullsized":"2024-09-10T16:02:44.941733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21598,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.105714+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:35.105714+0000","last_peered":"2024-09-10T16:02:35.105714+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:35.105714+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21865,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.135045+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:35.135045+0000","last_peered":"2024-09-10T16:02:35.135045+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:35.135045+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18040,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191025152,"data_stored":190956505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473584,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898256,"statfs":{"total":96636764160,"available":96151814144,"internally_reserved":0,"allocated":491520,"data_stored":146719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484437678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527000,"kb_used_data":62216,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844840,"statfs":{"total":96636764160,"available":96097116160,"internally_reserved":0,"allocated":63709184,"data_stored":63422191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953603,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":8211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:47.629 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-10T16:02:47.629 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'20770", 'reported_seq': 28064, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:41.727209+0000', 'last_change': '2024-09-10T16:00:37.532803+0000', 'last_active': '2024-09-10T16:02:41.727209+0000', 'last_peered': '2024-09-10T16:02:41.727209+0000', 'last_clean': '2024-09-10T15:53:17.324494+0000', 'last_became_active': '2024-09-10T15:53:19.918493+0000', 'last_became_peered': '2024-09-10T15:53:19.918493+0000', 'last_unstale': '2024-09-10T16:02:41.727209+0000', 'last_undegraded': '2024-09-10T15:53:19.548452+0000', 'last_fullsized': '2024-09-10T15:53:19.471751+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 370, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20770, 'num_write_kb': 41540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3603, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -370}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:47.630 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:47.630 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 21115, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.941634+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:44.941634+0000', 'last_peered': '2024-09-10T16:02:44.941634+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:44.941634+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:47.630 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:47.630 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21057, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.076561+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:35.076561+0000', 'last_peered': '2024-09-10T16:02:35.076561+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:35.076561+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:47.630 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:47.630 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21404, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.102557+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:35.102557+0000', 'last_peered': '2024-09-10T16:02:35.102557+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:35.102557+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:47.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:47.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'20456", 'reported_seq': 21824, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:46.198110+0000', 'last_change': '2024-09-10T16:02:43.316268+0000', 'last_active': '2024-09-10T16:02:46.198110+0000', 'last_peered': '2024-09-10T16:02:46.198110+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:02:46.198110+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'10400", 'ondisk_log_start': "46'10400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3218, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18325, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 613, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3218}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:47.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:47.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20854", 'reported_seq': 21702, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.101762+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:35.101762+0000', 'last_peered': '2024-09-10T16:02:35.101762+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:35.101762+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10800", 'ondisk_log_start': "46'10800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18046, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3478}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:47.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:47.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21474, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:34.979249+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:34.979249+0000', 'last_peered': '2024-09-10T16:02:34.979249+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:34.979249+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:47.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:47.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21598, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.105714+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:35.105714+0000', 'last_peered': '2024-09-10T16:02:35.105714+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:35.105714+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:47.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:47.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21865, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:35.135045+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:35.135045+0000', 'last_peered': '2024-09-10T16:02:35.135045+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:35.135045+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:47.632 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:48.027 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:48.028 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28608/6 objects degraded (476800.000%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":693,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1517756416,"bytes_avail":288392536064,"bytes_total":289910292480,"degraded_objects":28608,"degraded_total":6,"degraded_ratio":4768,"recovering_objects_per_sec":181,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1453,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:48.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:48.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:48.387 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:48.388 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:48.400 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-10T16:02:46.646326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18042,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482184,"kb_used_data":187800,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281633336,"statfs":{"total":289910292480,"available":288392536064,"internally_reserved":0,"allocated":192307200,"data_stored":191377868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1325386757},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001522"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28064,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:41.727209+0000","last_change":"2024-09-10T16:00:37.532803+0000","last_active":"2024-09-10T16:02:41.727209+0000","last_peered":"2024-09-10T16:02:41.727209+0000","last_clean":"2024-09-10T15:53:17.324494+0000","last_became_active":"2024-09-10T15:53:19.918493+0000","last_became_peered":"2024-09-10T15:53:19.918493+0000","last_unstale":"2024-09-10T16:02:41.727209+0000","last_undegraded":"2024-09-10T15:53:19.548452+0000","last_fullsized":"2024-09-10T15:53:19.471751+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3603,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-370}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20468,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086998+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:35.086998+0000","last_peered":"2024-09-10T16:02:35.086998+0000","last_clean":"2024-09-10T16:02:35.086998+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:35.086998+0000","last_undegraded":"2024-09-10T16:02:35.086998+0000","last_fullsized":"2024-09-10T16:02:35.086998+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.097281+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:35.097281+0000","last_peered":"2024-09-10T16:02:35.097281+0000","last_clean":"2024-09-10T16:02:35.097281+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:35.097281+0000","last_undegraded":"2024-09-10T16:02:35.097281+0000","last_fullsized":"2024-09-10T16:02:35.097281+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21057,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.076561+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:35.076561+0000","last_peered":"2024-09-10T16:02:35.076561+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:35.076561+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21056,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.127033+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:35.127033+0000","last_peered":"2024-09-10T16:02:35.127033+0000","last_clean":"2024-09-10T16:02:35.127033+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:35.127033+0000","last_undegraded":"2024-09-10T16:02:35.127033+0000","last_fullsized":"2024-09-10T16:02:35.127033+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.951532+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:00:38.951532+0000","last_peered":"2024-09-10T16:00:38.951532+0000","last_clean":"2024-09-10T16:00:38.951532+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:00:38.951532+0000","last_undegraded":"2024-09-10T16:00:38.951532+0000","last_fullsized":"2024-09-10T16:00:38.951532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28857,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:02:35.086419+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:35.086419+0000","last_peered":"2024-09-10T16:02:35.086419+0000","last_clean":"2024-09-10T16:02:35.086419+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:35.086419+0000","last_undegraded":"2024-09-10T16:02:35.086419+0000","last_fullsized":"2024-09-10T16:02:35.086419+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940823+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940823+0000","last_peered":"2024-09-10T16:02:44.940823+0000","last_clean":"2024-09-10T16:02:44.940823+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940823+0000","last_undegraded":"2024-09-10T16:02:44.940823+0000","last_fullsized":"2024-09-10T16:02:44.940823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21404,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.102557+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:35.102557+0000","last_peered":"2024-09-10T16:02:35.102557+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:35.102557+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":112,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.949302+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:00:38.949302+0000","last_peered":"2024-09-10T16:00:38.949302+0000","last_clean":"2024-09-10T16:00:38.949302+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:00:38.949302+0000","last_undegraded":"2024-09-10T16:00:38.949302+0000","last_fullsized":"2024-09-10T16:00:38.949302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28599,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941068+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941068+0000","last_peered":"2024-09-10T16:02:44.941068+0000","last_clean":"2024-09-10T16:02:44.941068+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941068+0000","last_undegraded":"2024-09-10T16:02:44.941068+0000","last_fullsized":"2024-09-10T16:02:44.941068+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952629+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:00:38.952629+0000","last_peered":"2024-09-10T16:00:38.952629+0000","last_clean":"2024-09-10T16:00:38.952629+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:00:38.952629+0000","last_undegraded":"2024-09-10T16:00:38.952629+0000","last_fullsized":"2024-09-10T16:00:38.952629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":21824,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:46.198110+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:02:46.198110+0000","last_peered":"2024-09-10T16:02:46.198110+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:46.198110+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":613,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3218}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21702,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.101762+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:35.101762+0000","last_peered":"2024-09-10T16:02:35.101762+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:35.101762+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952705+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:00:38.952705+0000","last_peered":"2024-09-10T16:00:38.952705+0000","last_clean":"2024-09-10T16:00:38.952705+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:00:38.952705+0000","last_undegraded":"2024-09-10T16:00:38.952705+0000","last_fullsized":"2024-09-10T16:00:38.952705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21474,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:34.979249+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:34.979249+0000","last_peered":"2024-09-10T16:02:34.979249+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:34.979249+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941733+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941733+0000","last_peered":"2024-09-10T16:02:44.941733+0000","last_clean":"2024-09-10T16:02:44.941733+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941733+0000","last_undegraded":"2024-09-10T16:02:44.941733+0000","last_fullsized":"2024-09-10T16:02:44.941733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21598,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.105714+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:35.105714+0000","last_peered":"2024-09-10T16:02:35.105714+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:35.105714+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952713+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:00:38.952713+0000","last_peered":"2024-09-10T16:00:38.952713+0000","last_clean":"2024-09-10T16:00:38.952713+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:00:38.952713+0000","last_undegraded":"2024-09-10T16:00:38.952713+0000","last_fullsized":"2024-09-10T16:00:38.952713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-10T16:00:38.952775+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:00:38.952775+0000","last_peered":"2024-09-10T16:00:38.952775+0000","last_clean":"2024-09-10T16:00:38.952775+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:00:38.952775+0000","last_undegraded":"2024-09-10T16:00:38.952775+0000","last_fullsized":"2024-09-10T16:00:38.952775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21865,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:35.135045+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:35.135045+0000","last_peered":"2024-09-10T16:02:35.135045+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:35.135045+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18040,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191025152,"data_stored":190956505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473584,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898256,"statfs":{"total":96636764160,"available":96151814144,"internally_reserved":0,"allocated":491520,"data_stored":146719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484437678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527000,"kb_used_data":62216,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844840,"statfs":{"total":96636764160,"available":96097116160,"internally_reserved":0,"allocated":63709184,"data_stored":63422191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953603,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":8211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:51.402 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:51.763 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:51.763 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:51.775 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-10T16:02:50.647068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18412,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491936,"kb_used_data":187824,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623584,"statfs":{"total":289910292480,"available":288382550016,"internally_reserved":0,"allocated":192331776,"data_stored":191388580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-840,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":840,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001521"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.045741+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:02:45.045741+0000","last_peered":"2024-09-10T16:02:45.045741+0000","last_clean":"2024-09-10T16:02:45.045741+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:02:45.045741+0000","last_undegraded":"2024-09-10T16:02:45.045741+0000","last_fullsized":"2024-09-10T16:02:45.045741+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20472,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.938106+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:44.938106+0000","last_peered":"2024-09-10T16:02:44.938106+0000","last_clean":"2024-09-10T16:02:44.938106+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:44.938106+0000","last_undegraded":"2024-09-10T16:02:44.938106+0000","last_fullsized":"2024-09-10T16:02:44.938106+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28770,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946705+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:44.946705+0000","last_peered":"2024-09-10T16:02:44.946705+0000","last_clean":"2024-09-10T16:02:44.946705+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:44.946705+0000","last_undegraded":"2024-09-10T16:02:44.946705+0000","last_fullsized":"2024-09-10T16:02:44.946705+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938299+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:44.938299+0000","last_peered":"2024-09-10T16:02:44.938299+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:44.938299+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21060,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937941+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:44.937941+0000","last_peered":"2024-09-10T16:02:44.937941+0000","last_clean":"2024-09-10T16:02:44.937941+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:44.937941+0000","last_undegraded":"2024-09-10T16:02:44.937941+0000","last_fullsized":"2024-09-10T16:02:44.937941+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946573+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:02:44.946573+0000","last_peered":"2024-09-10T16:02:44.946573+0000","last_clean":"2024-09-10T16:02:44.946573+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:02:44.946573+0000","last_undegraded":"2024-09-10T16:02:44.946573+0000","last_fullsized":"2024-09-10T16:02:44.946573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28861,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947768+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:44.947768+0000","last_peered":"2024-09-10T16:02:44.947768+0000","last_clean":"2024-09-10T16:02:44.947768+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:44.947768+0000","last_undegraded":"2024-09-10T16:02:44.947768+0000","last_fullsized":"2024-09-10T16:02:44.947768+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940823+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940823+0000","last_peered":"2024-09-10T16:02:44.940823+0000","last_clean":"2024-09-10T16:02:44.940823+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940823+0000","last_undegraded":"2024-09-10T16:02:44.940823+0000","last_fullsized":"2024-09-10T16:02:44.940823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21408,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938137+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:44.938137+0000","last_peered":"2024-09-10T16:02:44.938137+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:44.938137+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":116,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937964+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:02:44.937964+0000","last_peered":"2024-09-10T16:02:44.937964+0000","last_clean":"2024-09-10T16:02:44.937964+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:02:44.937964+0000","last_undegraded":"2024-09-10T16:02:44.937964+0000","last_fullsized":"2024-09-10T16:02:44.937964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28599,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941068+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941068+0000","last_peered":"2024-09-10T16:02:44.941068+0000","last_clean":"2024-09-10T16:02:44.941068+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941068+0000","last_undegraded":"2024-09-10T16:02:44.941068+0000","last_fullsized":"2024-09-10T16:02:44.941068+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947763+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:02:44.947763+0000","last_peered":"2024-09-10T16:02:44.947763+0000","last_clean":"2024-09-10T16:02:44.947763+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:02:44.947763+0000","last_undegraded":"2024-09-10T16:02:44.947763+0000","last_fullsized":"2024-09-10T16:02:44.947763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":21824,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:46.198110+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:02:46.198110+0000","last_peered":"2024-09-10T16:02:46.198110+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:46.198110+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":613,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3218}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21706,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938859+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:44.938859+0000","last_peered":"2024-09-10T16:02:44.938859+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:44.938859+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947767+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:02:44.947767+0000","last_peered":"2024-09-10T16:02:44.947767+0000","last_clean":"2024-09-10T16:02:44.947767+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:02:44.947767+0000","last_undegraded":"2024-09-10T16:02:44.947767+0000","last_fullsized":"2024-09-10T16:02:44.947767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21478,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939514+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:44.939514+0000","last_peered":"2024-09-10T16:02:44.939514+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:44.939514+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941733+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941733+0000","last_peered":"2024-09-10T16:02:44.941733+0000","last_clean":"2024-09-10T16:02:44.941733+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941733+0000","last_undegraded":"2024-09-10T16:02:44.941733+0000","last_fullsized":"2024-09-10T16:02:44.941733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21602,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938740+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:44.938740+0000","last_peered":"2024-09-10T16:02:44.938740+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:44.938740+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947856+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:02:44.947856+0000","last_peered":"2024-09-10T16:02:44.947856+0000","last_clean":"2024-09-10T16:02:44.947856+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:02:44.947856+0000","last_undegraded":"2024-09-10T16:02:44.947856+0000","last_fullsized":"2024-09-10T16:02:44.947856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947928+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:02:44.947928+0000","last_peered":"2024-09-10T16:02:44.947928+0000","last_clean":"2024-09-10T16:02:44.947928+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:02:44.947928+0000","last_undegraded":"2024-09-10T16:02:44.947928+0000","last_fullsized":"2024-09-10T16:02:44.947928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21869,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938526+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:44.938526+0000","last_peered":"2024-09-10T16:02:44.938526+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:44.938526+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18410,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191016960,"data_stored":190955539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626551,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483320,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888520,"statfs":{"total":96636764160,"available":96141844480,"internally_reserved":0,"allocated":499712,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149832,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527016,"kb_used_data":62232,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844824,"statfs":{"total":96636764160,"available":96097099776,"internally_reserved":0,"allocated":63725568,"data_stored":63428030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953603,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:51.776 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:52.088 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:52.088 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:52.101 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-10T16:02:50.647068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18412,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491936,"kb_used_data":187824,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623584,"statfs":{"total":289910292480,"available":288382550016,"internally_reserved":0,"allocated":192331776,"data_stored":191388580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-840,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":840,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001521"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.045741+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:02:45.045741+0000","last_peered":"2024-09-10T16:02:45.045741+0000","last_clean":"2024-09-10T16:02:45.045741+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:02:45.045741+0000","last_undegraded":"2024-09-10T16:02:45.045741+0000","last_fullsized":"2024-09-10T16:02:45.045741+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20472,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.938106+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:44.938106+0000","last_peered":"2024-09-10T16:02:44.938106+0000","last_clean":"2024-09-10T16:02:44.938106+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:44.938106+0000","last_undegraded":"2024-09-10T16:02:44.938106+0000","last_fullsized":"2024-09-10T16:02:44.938106+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28770,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946705+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:44.946705+0000","last_peered":"2024-09-10T16:02:44.946705+0000","last_clean":"2024-09-10T16:02:44.946705+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:44.946705+0000","last_undegraded":"2024-09-10T16:02:44.946705+0000","last_fullsized":"2024-09-10T16:02:44.946705+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938299+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:44.938299+0000","last_peered":"2024-09-10T16:02:44.938299+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:44.938299+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21060,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937941+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:44.937941+0000","last_peered":"2024-09-10T16:02:44.937941+0000","last_clean":"2024-09-10T16:02:44.937941+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:44.937941+0000","last_undegraded":"2024-09-10T16:02:44.937941+0000","last_fullsized":"2024-09-10T16:02:44.937941+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946573+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:02:44.946573+0000","last_peered":"2024-09-10T16:02:44.946573+0000","last_clean":"2024-09-10T16:02:44.946573+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:02:44.946573+0000","last_undegraded":"2024-09-10T16:02:44.946573+0000","last_fullsized":"2024-09-10T16:02:44.946573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28861,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947768+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:44.947768+0000","last_peered":"2024-09-10T16:02:44.947768+0000","last_clean":"2024-09-10T16:02:44.947768+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:44.947768+0000","last_undegraded":"2024-09-10T16:02:44.947768+0000","last_fullsized":"2024-09-10T16:02:44.947768+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940823+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940823+0000","last_peered":"2024-09-10T16:02:44.940823+0000","last_clean":"2024-09-10T16:02:44.940823+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940823+0000","last_undegraded":"2024-09-10T16:02:44.940823+0000","last_fullsized":"2024-09-10T16:02:44.940823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21408,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938137+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:44.938137+0000","last_peered":"2024-09-10T16:02:44.938137+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:44.938137+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":116,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937964+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:02:44.937964+0000","last_peered":"2024-09-10T16:02:44.937964+0000","last_clean":"2024-09-10T16:02:44.937964+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:02:44.937964+0000","last_undegraded":"2024-09-10T16:02:44.937964+0000","last_fullsized":"2024-09-10T16:02:44.937964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28599,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941068+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941068+0000","last_peered":"2024-09-10T16:02:44.941068+0000","last_clean":"2024-09-10T16:02:44.941068+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941068+0000","last_undegraded":"2024-09-10T16:02:44.941068+0000","last_fullsized":"2024-09-10T16:02:44.941068+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947763+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:02:44.947763+0000","last_peered":"2024-09-10T16:02:44.947763+0000","last_clean":"2024-09-10T16:02:44.947763+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:02:44.947763+0000","last_undegraded":"2024-09-10T16:02:44.947763+0000","last_fullsized":"2024-09-10T16:02:44.947763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":21824,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:46.198110+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:02:46.198110+0000","last_peered":"2024-09-10T16:02:46.198110+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:46.198110+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":613,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3218}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21706,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938859+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:44.938859+0000","last_peered":"2024-09-10T16:02:44.938859+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:44.938859+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947767+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:02:44.947767+0000","last_peered":"2024-09-10T16:02:44.947767+0000","last_clean":"2024-09-10T16:02:44.947767+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:02:44.947767+0000","last_undegraded":"2024-09-10T16:02:44.947767+0000","last_fullsized":"2024-09-10T16:02:44.947767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21478,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939514+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:44.939514+0000","last_peered":"2024-09-10T16:02:44.939514+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:44.939514+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941733+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941733+0000","last_peered":"2024-09-10T16:02:44.941733+0000","last_clean":"2024-09-10T16:02:44.941733+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941733+0000","last_undegraded":"2024-09-10T16:02:44.941733+0000","last_fullsized":"2024-09-10T16:02:44.941733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21602,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938740+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:44.938740+0000","last_peered":"2024-09-10T16:02:44.938740+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:44.938740+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947856+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:02:44.947856+0000","last_peered":"2024-09-10T16:02:44.947856+0000","last_clean":"2024-09-10T16:02:44.947856+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:02:44.947856+0000","last_undegraded":"2024-09-10T16:02:44.947856+0000","last_fullsized":"2024-09-10T16:02:44.947856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947928+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:02:44.947928+0000","last_peered":"2024-09-10T16:02:44.947928+0000","last_clean":"2024-09-10T16:02:44.947928+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:02:44.947928+0000","last_undegraded":"2024-09-10T16:02:44.947928+0000","last_fullsized":"2024-09-10T16:02:44.947928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21869,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938526+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:44.938526+0000","last_peered":"2024-09-10T16:02:44.938526+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:44.938526+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18410,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191016960,"data_stored":190955539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626551,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483320,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888520,"statfs":{"total":96636764160,"available":96141844480,"internally_reserved":0,"allocated":499712,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149832,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527016,"kb_used_data":62232,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844824,"statfs":{"total":96636764160,"available":96097099776,"internally_reserved":0,"allocated":63725568,"data_stored":63428030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953603,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:52.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:52.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 21115, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.941634+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:44.941634+0000', 'last_peered': '2024-09-10T16:02:44.941634+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:44.941634+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:52.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:52.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21061, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938299+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:44.938299+0000', 'last_peered': '2024-09-10T16:02:44.938299+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:44.938299+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:52.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:52.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21408, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938137+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:44.938137+0000', 'last_peered': '2024-09-10T16:02:44.938137+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:44.938137+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:52.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:52.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'20456", 'reported_seq': 21824, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:46.198110+0000', 'last_change': '2024-09-10T16:02:43.316268+0000', 'last_active': '2024-09-10T16:02:46.198110+0000', 'last_peered': '2024-09-10T16:02:46.198110+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:02:46.198110+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'10400", 'ondisk_log_start': "46'10400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3218, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18325, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 613, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3218}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:52.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:52.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20854", 'reported_seq': 21706, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938859+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:44.938859+0000', 'last_peered': '2024-09-10T16:02:44.938859+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:44.938859+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10800", 'ondisk_log_start': "46'10800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18046, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3478}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:52.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:52.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21478, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.939514+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:44.939514+0000', 'last_peered': '2024-09-10T16:02:44.939514+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:44.939514+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:52.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:52.103 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21602, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938740+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:44.938740+0000', 'last_peered': '2024-09-10T16:02:44.938740+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:44.938740+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:52.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:52.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21869, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938526+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:44.938526+0000', 'last_peered': '2024-09-10T16:02:44.938526+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:44.938526+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:52.104 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:52.474 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:52.475 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28238/6 objects degraded (470633.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":698,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1527742464,"bytes_avail":288382550016,"bytes_total":289910292480,"degraded_objects":28238,"degraded_total":6,"degraded_ratio":4706.333333333333,"recovering_objects_per_sec":104,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":840,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:52.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:52.488 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:52.795 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:52.796 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:52.808 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-10T16:02:52.647389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19291,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496096,"kb_used_data":187824,"kb_used_omap":12,"kb_used_meta":1308211,"kb_avail":281619424,"statfs":{"total":289910292480,"available":288378290176,"internally_reserved":0,"allocated":192331776,"data_stored":191388580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1339608069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1719,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1719,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.711788"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.045741+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:02:45.045741+0000","last_peered":"2024-09-10T16:02:45.045741+0000","last_clean":"2024-09-10T16:02:45.045741+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:02:45.045741+0000","last_undegraded":"2024-09-10T16:02:45.045741+0000","last_fullsized":"2024-09-10T16:02:45.045741+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20472,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.938106+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:44.938106+0000","last_peered":"2024-09-10T16:02:44.938106+0000","last_clean":"2024-09-10T16:02:44.938106+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:44.938106+0000","last_undegraded":"2024-09-10T16:02:44.938106+0000","last_fullsized":"2024-09-10T16:02:44.938106+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28770,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946705+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:44.946705+0000","last_peered":"2024-09-10T16:02:44.946705+0000","last_clean":"2024-09-10T16:02:44.946705+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:44.946705+0000","last_undegraded":"2024-09-10T16:02:44.946705+0000","last_fullsized":"2024-09-10T16:02:44.946705+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938299+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:44.938299+0000","last_peered":"2024-09-10T16:02:44.938299+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:44.938299+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21060,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937941+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:44.937941+0000","last_peered":"2024-09-10T16:02:44.937941+0000","last_clean":"2024-09-10T16:02:44.937941+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:44.937941+0000","last_undegraded":"2024-09-10T16:02:44.937941+0000","last_fullsized":"2024-09-10T16:02:44.937941+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946573+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:02:44.946573+0000","last_peered":"2024-09-10T16:02:44.946573+0000","last_clean":"2024-09-10T16:02:44.946573+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:02:44.946573+0000","last_undegraded":"2024-09-10T16:02:44.946573+0000","last_fullsized":"2024-09-10T16:02:44.946573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28861,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947768+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:44.947768+0000","last_peered":"2024-09-10T16:02:44.947768+0000","last_clean":"2024-09-10T16:02:44.947768+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:44.947768+0000","last_undegraded":"2024-09-10T16:02:44.947768+0000","last_fullsized":"2024-09-10T16:02:44.947768+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940865+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940865+0000","last_peered":"2024-09-10T16:02:44.940865+0000","last_clean":"2024-09-10T16:02:44.940865+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940865+0000","last_undegraded":"2024-09-10T16:02:44.940865+0000","last_fullsized":"2024-09-10T16:02:44.940865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21408,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938137+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:44.938137+0000","last_peered":"2024-09-10T16:02:44.938137+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:44.938137+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":116,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937964+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:02:44.937964+0000","last_peered":"2024-09-10T16:02:44.937964+0000","last_clean":"2024-09-10T16:02:44.937964+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:02:44.937964+0000","last_undegraded":"2024-09-10T16:02:44.937964+0000","last_fullsized":"2024-09-10T16:02:44.937964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28600,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941113+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941113+0000","last_peered":"2024-09-10T16:02:44.941113+0000","last_clean":"2024-09-10T16:02:44.941113+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941113+0000","last_undegraded":"2024-09-10T16:02:44.941113+0000","last_fullsized":"2024-09-10T16:02:44.941113+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947763+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:02:44.947763+0000","last_peered":"2024-09-10T16:02:44.947763+0000","last_clean":"2024-09-10T16:02:44.947763+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:02:44.947763+0000","last_undegraded":"2024-09-10T16:02:44.947763+0000","last_fullsized":"2024-09-10T16:02:44.947763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":23582,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:51.199388+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:02:51.199388+0000","last_peered":"2024-09-10T16:02:51.199388+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:51.199388+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1492,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2339}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21706,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938859+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:44.938859+0000","last_peered":"2024-09-10T16:02:44.938859+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:44.938859+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947767+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:02:44.947767+0000","last_peered":"2024-09-10T16:02:44.947767+0000","last_clean":"2024-09-10T16:02:44.947767+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:02:44.947767+0000","last_undegraded":"2024-09-10T16:02:44.947767+0000","last_fullsized":"2024-09-10T16:02:44.947767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21478,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939514+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:44.939514+0000","last_peered":"2024-09-10T16:02:44.939514+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:44.939514+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941784+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941784+0000","last_peered":"2024-09-10T16:02:44.941784+0000","last_clean":"2024-09-10T16:02:44.941784+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941784+0000","last_undegraded":"2024-09-10T16:02:44.941784+0000","last_fullsized":"2024-09-10T16:02:44.941784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21603,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939920+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:44.939920+0000","last_peered":"2024-09-10T16:02:44.939920+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:44.939920+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947856+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:02:44.947856+0000","last_peered":"2024-09-10T16:02:44.947856+0000","last_clean":"2024-09-10T16:02:44.947856+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:02:44.947856+0000","last_undegraded":"2024-09-10T16:02:44.947856+0000","last_fullsized":"2024-09-10T16:02:44.947856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947928+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:02:44.947928+0000","last_peered":"2024-09-10T16:02:44.947928+0000","last_clean":"2024-09-10T16:02:44.947928+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:02:44.947928+0000","last_undegraded":"2024-09-10T16:02:44.947928+0000","last_fullsized":"2024-09-10T16:02:44.947928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21870,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939551+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:44.939551+0000","last_peered":"2024-09-10T16:02:44.939551+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:44.939551+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19289,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191016960,"data_stored":190955539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626552,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487480,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884360,"statfs":{"total":96636764160,"available":96137584640,"internally_reserved":0,"allocated":499712,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498658990},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149833,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527016,"kb_used_data":62232,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844824,"statfs":{"total":96636764160,"available":96097099776,"internally_reserved":0,"allocated":63725568,"data_stored":63428030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953604,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:55.811 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:56.117 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:56.118 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:56.130 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-10T16:02:54.647739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19291,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496096,"kb_used_data":187824,"kb_used_omap":12,"kb_used_meta":1308211,"kb_avail":281619424,"statfs":{"total":289910292480,"available":288378290176,"internally_reserved":0,"allocated":192331776,"data_stored":191388580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1339608069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1719,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1719,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001757"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.045741+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:02:45.045741+0000","last_peered":"2024-09-10T16:02:45.045741+0000","last_clean":"2024-09-10T16:02:45.045741+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:02:45.045741+0000","last_undegraded":"2024-09-10T16:02:45.045741+0000","last_fullsized":"2024-09-10T16:02:45.045741+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20472,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.938106+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:44.938106+0000","last_peered":"2024-09-10T16:02:44.938106+0000","last_clean":"2024-09-10T16:02:44.938106+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:44.938106+0000","last_undegraded":"2024-09-10T16:02:44.938106+0000","last_fullsized":"2024-09-10T16:02:44.938106+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28770,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946705+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:44.946705+0000","last_peered":"2024-09-10T16:02:44.946705+0000","last_clean":"2024-09-10T16:02:44.946705+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:44.946705+0000","last_undegraded":"2024-09-10T16:02:44.946705+0000","last_fullsized":"2024-09-10T16:02:44.946705+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938299+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:44.938299+0000","last_peered":"2024-09-10T16:02:44.938299+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:44.938299+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21060,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937941+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:44.937941+0000","last_peered":"2024-09-10T16:02:44.937941+0000","last_clean":"2024-09-10T16:02:44.937941+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:44.937941+0000","last_undegraded":"2024-09-10T16:02:44.937941+0000","last_fullsized":"2024-09-10T16:02:44.937941+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946573+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:02:44.946573+0000","last_peered":"2024-09-10T16:02:44.946573+0000","last_clean":"2024-09-10T16:02:44.946573+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:02:44.946573+0000","last_undegraded":"2024-09-10T16:02:44.946573+0000","last_fullsized":"2024-09-10T16:02:44.946573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28861,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947768+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:44.947768+0000","last_peered":"2024-09-10T16:02:44.947768+0000","last_clean":"2024-09-10T16:02:44.947768+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:44.947768+0000","last_undegraded":"2024-09-10T16:02:44.947768+0000","last_fullsized":"2024-09-10T16:02:44.947768+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940865+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940865+0000","last_peered":"2024-09-10T16:02:44.940865+0000","last_clean":"2024-09-10T16:02:44.940865+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940865+0000","last_undegraded":"2024-09-10T16:02:44.940865+0000","last_fullsized":"2024-09-10T16:02:44.940865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21408,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938137+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:44.938137+0000","last_peered":"2024-09-10T16:02:44.938137+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:44.938137+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":116,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937964+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:02:44.937964+0000","last_peered":"2024-09-10T16:02:44.937964+0000","last_clean":"2024-09-10T16:02:44.937964+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:02:44.937964+0000","last_undegraded":"2024-09-10T16:02:44.937964+0000","last_fullsized":"2024-09-10T16:02:44.937964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28600,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941113+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941113+0000","last_peered":"2024-09-10T16:02:44.941113+0000","last_clean":"2024-09-10T16:02:44.941113+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941113+0000","last_undegraded":"2024-09-10T16:02:44.941113+0000","last_fullsized":"2024-09-10T16:02:44.941113+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947763+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:02:44.947763+0000","last_peered":"2024-09-10T16:02:44.947763+0000","last_clean":"2024-09-10T16:02:44.947763+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:02:44.947763+0000","last_undegraded":"2024-09-10T16:02:44.947763+0000","last_fullsized":"2024-09-10T16:02:44.947763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":23582,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:51.199388+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:02:51.199388+0000","last_peered":"2024-09-10T16:02:51.199388+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:51.199388+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1492,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2339}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21706,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938859+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:44.938859+0000","last_peered":"2024-09-10T16:02:44.938859+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:44.938859+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947767+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:02:44.947767+0000","last_peered":"2024-09-10T16:02:44.947767+0000","last_clean":"2024-09-10T16:02:44.947767+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:02:44.947767+0000","last_undegraded":"2024-09-10T16:02:44.947767+0000","last_fullsized":"2024-09-10T16:02:44.947767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21478,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939514+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:44.939514+0000","last_peered":"2024-09-10T16:02:44.939514+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:44.939514+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941784+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941784+0000","last_peered":"2024-09-10T16:02:44.941784+0000","last_clean":"2024-09-10T16:02:44.941784+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941784+0000","last_undegraded":"2024-09-10T16:02:44.941784+0000","last_fullsized":"2024-09-10T16:02:44.941784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21603,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939920+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:44.939920+0000","last_peered":"2024-09-10T16:02:44.939920+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:44.939920+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947856+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:02:44.947856+0000","last_peered":"2024-09-10T16:02:44.947856+0000","last_clean":"2024-09-10T16:02:44.947856+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:02:44.947856+0000","last_undegraded":"2024-09-10T16:02:44.947856+0000","last_fullsized":"2024-09-10T16:02:44.947856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947928+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:02:44.947928+0000","last_peered":"2024-09-10T16:02:44.947928+0000","last_clean":"2024-09-10T16:02:44.947928+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:02:44.947928+0000","last_undegraded":"2024-09-10T16:02:44.947928+0000","last_fullsized":"2024-09-10T16:02:44.947928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21870,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939551+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:44.939551+0000","last_peered":"2024-09-10T16:02:44.939551+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:44.939551+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19289,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191016960,"data_stored":190955539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626552,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487480,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884360,"statfs":{"total":96636764160,"available":96137584640,"internally_reserved":0,"allocated":499712,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498658990},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149833,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527016,"kb_used_data":62232,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844824,"statfs":{"total":96636764160,"available":96097099776,"internally_reserved":0,"allocated":63725568,"data_stored":63428030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953604,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:56.131 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:56.442 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:56.442 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:56.454 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-10T16:02:54.647739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19291,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496096,"kb_used_data":187824,"kb_used_omap":12,"kb_used_meta":1308211,"kb_avail":281619424,"statfs":{"total":289910292480,"available":288378290176,"internally_reserved":0,"allocated":192331776,"data_stored":191388580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1339608069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1719,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1719,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001757"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.045741+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:02:45.045741+0000","last_peered":"2024-09-10T16:02:45.045741+0000","last_clean":"2024-09-10T16:02:45.045741+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:02:45.045741+0000","last_undegraded":"2024-09-10T16:02:45.045741+0000","last_fullsized":"2024-09-10T16:02:45.045741+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20472,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.938106+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:44.938106+0000","last_peered":"2024-09-10T16:02:44.938106+0000","last_clean":"2024-09-10T16:02:44.938106+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:44.938106+0000","last_undegraded":"2024-09-10T16:02:44.938106+0000","last_fullsized":"2024-09-10T16:02:44.938106+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28770,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946705+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:44.946705+0000","last_peered":"2024-09-10T16:02:44.946705+0000","last_clean":"2024-09-10T16:02:44.946705+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:44.946705+0000","last_undegraded":"2024-09-10T16:02:44.946705+0000","last_fullsized":"2024-09-10T16:02:44.946705+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938299+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:44.938299+0000","last_peered":"2024-09-10T16:02:44.938299+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:44.938299+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21060,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937941+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:44.937941+0000","last_peered":"2024-09-10T16:02:44.937941+0000","last_clean":"2024-09-10T16:02:44.937941+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:44.937941+0000","last_undegraded":"2024-09-10T16:02:44.937941+0000","last_fullsized":"2024-09-10T16:02:44.937941+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946573+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:02:44.946573+0000","last_peered":"2024-09-10T16:02:44.946573+0000","last_clean":"2024-09-10T16:02:44.946573+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:02:44.946573+0000","last_undegraded":"2024-09-10T16:02:44.946573+0000","last_fullsized":"2024-09-10T16:02:44.946573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28861,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947768+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:44.947768+0000","last_peered":"2024-09-10T16:02:44.947768+0000","last_clean":"2024-09-10T16:02:44.947768+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:44.947768+0000","last_undegraded":"2024-09-10T16:02:44.947768+0000","last_fullsized":"2024-09-10T16:02:44.947768+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940865+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940865+0000","last_peered":"2024-09-10T16:02:44.940865+0000","last_clean":"2024-09-10T16:02:44.940865+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940865+0000","last_undegraded":"2024-09-10T16:02:44.940865+0000","last_fullsized":"2024-09-10T16:02:44.940865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21408,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938137+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:44.938137+0000","last_peered":"2024-09-10T16:02:44.938137+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:44.938137+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":116,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937964+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:02:44.937964+0000","last_peered":"2024-09-10T16:02:44.937964+0000","last_clean":"2024-09-10T16:02:44.937964+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:02:44.937964+0000","last_undegraded":"2024-09-10T16:02:44.937964+0000","last_fullsized":"2024-09-10T16:02:44.937964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28600,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941113+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941113+0000","last_peered":"2024-09-10T16:02:44.941113+0000","last_clean":"2024-09-10T16:02:44.941113+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941113+0000","last_undegraded":"2024-09-10T16:02:44.941113+0000","last_fullsized":"2024-09-10T16:02:44.941113+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947763+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:02:44.947763+0000","last_peered":"2024-09-10T16:02:44.947763+0000","last_clean":"2024-09-10T16:02:44.947763+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:02:44.947763+0000","last_undegraded":"2024-09-10T16:02:44.947763+0000","last_fullsized":"2024-09-10T16:02:44.947763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":23582,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:51.199388+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:02:51.199388+0000","last_peered":"2024-09-10T16:02:51.199388+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:51.199388+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1492,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2339}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21706,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938859+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:44.938859+0000","last_peered":"2024-09-10T16:02:44.938859+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:44.938859+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947767+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:02:44.947767+0000","last_peered":"2024-09-10T16:02:44.947767+0000","last_clean":"2024-09-10T16:02:44.947767+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:02:44.947767+0000","last_undegraded":"2024-09-10T16:02:44.947767+0000","last_fullsized":"2024-09-10T16:02:44.947767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21478,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939514+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:44.939514+0000","last_peered":"2024-09-10T16:02:44.939514+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:44.939514+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941784+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941784+0000","last_peered":"2024-09-10T16:02:44.941784+0000","last_clean":"2024-09-10T16:02:44.941784+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941784+0000","last_undegraded":"2024-09-10T16:02:44.941784+0000","last_fullsized":"2024-09-10T16:02:44.941784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21603,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939920+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:44.939920+0000","last_peered":"2024-09-10T16:02:44.939920+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:44.939920+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947856+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:02:44.947856+0000","last_peered":"2024-09-10T16:02:44.947856+0000","last_clean":"2024-09-10T16:02:44.947856+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:02:44.947856+0000","last_undegraded":"2024-09-10T16:02:44.947856+0000","last_fullsized":"2024-09-10T16:02:44.947856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947928+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:02:44.947928+0000","last_peered":"2024-09-10T16:02:44.947928+0000","last_clean":"2024-09-10T16:02:44.947928+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:02:44.947928+0000","last_undegraded":"2024-09-10T16:02:44.947928+0000","last_fullsized":"2024-09-10T16:02:44.947928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21870,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939551+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:44.939551+0000","last_peered":"2024-09-10T16:02:44.939551+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:44.939551+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19289,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191016960,"data_stored":190955539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626552,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487480,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884360,"statfs":{"total":96636764160,"available":96137584640,"internally_reserved":0,"allocated":499712,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498658990},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149833,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527016,"kb_used_data":62232,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844824,"statfs":{"total":96636764160,"available":96097099776,"internally_reserved":0,"allocated":63725568,"data_stored":63428030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953604,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:02:56.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:02:56.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 21115, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.941634+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:44.941634+0000', 'last_peered': '2024-09-10T16:02:44.941634+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:44.941634+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:56.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:02:56.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21061, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938299+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:44.938299+0000', 'last_peered': '2024-09-10T16:02:44.938299+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:44.938299+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:56.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:02:56.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21408, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938137+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:44.938137+0000', 'last_peered': '2024-09-10T16:02:44.938137+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:44.938137+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:56.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:02:56.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'20456", 'reported_seq': 23582, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:51.199388+0000', 'last_change': '2024-09-10T16:02:43.316268+0000', 'last_active': '2024-09-10T16:02:51.199388+0000', 'last_peered': '2024-09-10T16:02:51.199388+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:02:51.199388+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'10400", 'ondisk_log_start': "46'10400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2339, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18325, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1492, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2339}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:02:56.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:02:56.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20854", 'reported_seq': 21706, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938859+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:44.938859+0000', 'last_peered': '2024-09-10T16:02:44.938859+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:44.938859+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10800", 'ondisk_log_start': "46'10800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18046, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3478}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:56.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:02:56.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21478, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.939514+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:44.939514+0000', 'last_peered': '2024-09-10T16:02:44.939514+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:44.939514+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:56.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:02:56.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21603, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.939920+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:44.939920+0000', 'last_peered': '2024-09-10T16:02:44.939920+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:44.939920+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:56.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:02:56.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21870, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.939551+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:44.939551+0000', 'last_peered': '2024-09-10T16:02:44.939551+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:44.939551+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:02:56.457 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:02:56.836 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:56.836 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27359/6 objects degraded (455983.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":702,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1532002304,"bytes_avail":288378290176,"bytes_total":289910292480,"degraded_objects":27359,"degraded_total":6,"degraded_ratio":4559.833333333333,"recovering_objects_per_sec":171,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1719,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:02:56.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:02:56.849 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:02:57.147 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:02:57.147 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:02:57.161 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-10T16:02:56.648048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20042,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496096,"kb_used_data":187824,"kb_used_omap":12,"kb_used_meta":1308211,"kb_avail":281619424,"statfs":{"total":289910292480,"available":288378290176,"internally_reserved":0,"allocated":192331776,"data_stored":191388580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1339608069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2470,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.711355"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.045741+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:02:45.045741+0000","last_peered":"2024-09-10T16:02:45.045741+0000","last_clean":"2024-09-10T16:02:45.045741+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:02:45.045741+0000","last_undegraded":"2024-09-10T16:02:45.045741+0000","last_fullsized":"2024-09-10T16:02:45.045741+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20472,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.938106+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:44.938106+0000","last_peered":"2024-09-10T16:02:44.938106+0000","last_clean":"2024-09-10T16:02:44.938106+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:44.938106+0000","last_undegraded":"2024-09-10T16:02:44.938106+0000","last_fullsized":"2024-09-10T16:02:44.938106+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28770,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946705+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:44.946705+0000","last_peered":"2024-09-10T16:02:44.946705+0000","last_clean":"2024-09-10T16:02:44.946705+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:44.946705+0000","last_undegraded":"2024-09-10T16:02:44.946705+0000","last_fullsized":"2024-09-10T16:02:44.946705+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938299+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:44.938299+0000","last_peered":"2024-09-10T16:02:44.938299+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:44.938299+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21060,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937941+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:44.937941+0000","last_peered":"2024-09-10T16:02:44.937941+0000","last_clean":"2024-09-10T16:02:44.937941+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:44.937941+0000","last_undegraded":"2024-09-10T16:02:44.937941+0000","last_fullsized":"2024-09-10T16:02:44.937941+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946573+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:02:44.946573+0000","last_peered":"2024-09-10T16:02:44.946573+0000","last_clean":"2024-09-10T16:02:44.946573+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:02:44.946573+0000","last_undegraded":"2024-09-10T16:02:44.946573+0000","last_fullsized":"2024-09-10T16:02:44.946573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28861,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947768+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:44.947768+0000","last_peered":"2024-09-10T16:02:44.947768+0000","last_clean":"2024-09-10T16:02:44.947768+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:44.947768+0000","last_undegraded":"2024-09-10T16:02:44.947768+0000","last_fullsized":"2024-09-10T16:02:44.947768+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940865+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940865+0000","last_peered":"2024-09-10T16:02:44.940865+0000","last_clean":"2024-09-10T16:02:44.940865+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940865+0000","last_undegraded":"2024-09-10T16:02:44.940865+0000","last_fullsized":"2024-09-10T16:02:44.940865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21408,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938137+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:44.938137+0000","last_peered":"2024-09-10T16:02:44.938137+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:44.938137+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":116,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937964+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:02:44.937964+0000","last_peered":"2024-09-10T16:02:44.937964+0000","last_clean":"2024-09-10T16:02:44.937964+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:02:44.937964+0000","last_undegraded":"2024-09-10T16:02:44.937964+0000","last_fullsized":"2024-09-10T16:02:44.937964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28600,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941113+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941113+0000","last_peered":"2024-09-10T16:02:44.941113+0000","last_clean":"2024-09-10T16:02:44.941113+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941113+0000","last_undegraded":"2024-09-10T16:02:44.941113+0000","last_fullsized":"2024-09-10T16:02:44.941113+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947763+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:02:44.947763+0000","last_peered":"2024-09-10T16:02:44.947763+0000","last_clean":"2024-09-10T16:02:44.947763+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:02:44.947763+0000","last_undegraded":"2024-09-10T16:02:44.947763+0000","last_fullsized":"2024-09-10T16:02:44.947763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":25085,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:56.202158+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:02:56.202158+0000","last_peered":"2024-09-10T16:02:56.202158+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:56.202158+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21706,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938859+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:44.938859+0000","last_peered":"2024-09-10T16:02:44.938859+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:44.938859+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947767+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:02:44.947767+0000","last_peered":"2024-09-10T16:02:44.947767+0000","last_clean":"2024-09-10T16:02:44.947767+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:02:44.947767+0000","last_undegraded":"2024-09-10T16:02:44.947767+0000","last_fullsized":"2024-09-10T16:02:44.947767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21478,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939514+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:44.939514+0000","last_peered":"2024-09-10T16:02:44.939514+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:44.939514+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941784+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941784+0000","last_peered":"2024-09-10T16:02:44.941784+0000","last_clean":"2024-09-10T16:02:44.941784+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941784+0000","last_undegraded":"2024-09-10T16:02:44.941784+0000","last_fullsized":"2024-09-10T16:02:44.941784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21603,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939920+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:44.939920+0000","last_peered":"2024-09-10T16:02:44.939920+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:44.939920+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947856+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:02:44.947856+0000","last_peered":"2024-09-10T16:02:44.947856+0000","last_clean":"2024-09-10T16:02:44.947856+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:02:44.947856+0000","last_undegraded":"2024-09-10T16:02:44.947856+0000","last_fullsized":"2024-09-10T16:02:44.947856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947928+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:02:44.947928+0000","last_peered":"2024-09-10T16:02:44.947928+0000","last_clean":"2024-09-10T16:02:44.947928+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:02:44.947928+0000","last_undegraded":"2024-09-10T16:02:44.947928+0000","last_fullsized":"2024-09-10T16:02:44.947928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21870,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939551+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:44.939551+0000","last_peered":"2024-09-10T16:02:44.939551+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:44.939551+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20040,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191016960,"data_stored":190955539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626552,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487480,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884360,"statfs":{"total":96636764160,"available":96137584640,"internally_reserved":0,"allocated":499712,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498658990},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149833,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527016,"kb_used_data":62232,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844824,"statfs":{"total":96636764160,"available":96097099776,"internally_reserved":0,"allocated":63725568,"data_stored":63428030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953605,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:00.163 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:00.467 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:00.468 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:00.480 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-10T16:02:58.648386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20042,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1504416,"kb_used_data":187824,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281611104,"statfs":{"total":289910292480,"available":288369770496,"internally_reserved":0,"allocated":192331776,"data_stored":191388580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1348127749},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2000,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002059"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.045741+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:02:45.045741+0000","last_peered":"2024-09-10T16:02:45.045741+0000","last_clean":"2024-09-10T16:02:45.045741+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:02:45.045741+0000","last_undegraded":"2024-09-10T16:02:45.045741+0000","last_fullsized":"2024-09-10T16:02:45.045741+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20472,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.938106+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:44.938106+0000","last_peered":"2024-09-10T16:02:44.938106+0000","last_clean":"2024-09-10T16:02:44.938106+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:44.938106+0000","last_undegraded":"2024-09-10T16:02:44.938106+0000","last_fullsized":"2024-09-10T16:02:44.938106+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28770,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946705+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:44.946705+0000","last_peered":"2024-09-10T16:02:44.946705+0000","last_clean":"2024-09-10T16:02:44.946705+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:44.946705+0000","last_undegraded":"2024-09-10T16:02:44.946705+0000","last_fullsized":"2024-09-10T16:02:44.946705+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938299+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:44.938299+0000","last_peered":"2024-09-10T16:02:44.938299+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:44.938299+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21060,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937941+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:44.937941+0000","last_peered":"2024-09-10T16:02:44.937941+0000","last_clean":"2024-09-10T16:02:44.937941+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:44.937941+0000","last_undegraded":"2024-09-10T16:02:44.937941+0000","last_fullsized":"2024-09-10T16:02:44.937941+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946573+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:02:44.946573+0000","last_peered":"2024-09-10T16:02:44.946573+0000","last_clean":"2024-09-10T16:02:44.946573+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:02:44.946573+0000","last_undegraded":"2024-09-10T16:02:44.946573+0000","last_fullsized":"2024-09-10T16:02:44.946573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28861,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947768+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:44.947768+0000","last_peered":"2024-09-10T16:02:44.947768+0000","last_clean":"2024-09-10T16:02:44.947768+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:44.947768+0000","last_undegraded":"2024-09-10T16:02:44.947768+0000","last_fullsized":"2024-09-10T16:02:44.947768+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940865+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940865+0000","last_peered":"2024-09-10T16:02:44.940865+0000","last_clean":"2024-09-10T16:02:44.940865+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940865+0000","last_undegraded":"2024-09-10T16:02:44.940865+0000","last_fullsized":"2024-09-10T16:02:44.940865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21408,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938137+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:44.938137+0000","last_peered":"2024-09-10T16:02:44.938137+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:44.938137+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":116,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937964+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:02:44.937964+0000","last_peered":"2024-09-10T16:02:44.937964+0000","last_clean":"2024-09-10T16:02:44.937964+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:02:44.937964+0000","last_undegraded":"2024-09-10T16:02:44.937964+0000","last_fullsized":"2024-09-10T16:02:44.937964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28600,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941113+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941113+0000","last_peered":"2024-09-10T16:02:44.941113+0000","last_clean":"2024-09-10T16:02:44.941113+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941113+0000","last_undegraded":"2024-09-10T16:02:44.941113+0000","last_fullsized":"2024-09-10T16:02:44.941113+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947763+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:02:44.947763+0000","last_peered":"2024-09-10T16:02:44.947763+0000","last_clean":"2024-09-10T16:02:44.947763+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:02:44.947763+0000","last_undegraded":"2024-09-10T16:02:44.947763+0000","last_fullsized":"2024-09-10T16:02:44.947763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":25085,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:56.202158+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:02:56.202158+0000","last_peered":"2024-09-10T16:02:56.202158+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:56.202158+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21706,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938859+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:44.938859+0000","last_peered":"2024-09-10T16:02:44.938859+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:44.938859+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947767+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:02:44.947767+0000","last_peered":"2024-09-10T16:02:44.947767+0000","last_clean":"2024-09-10T16:02:44.947767+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:02:44.947767+0000","last_undegraded":"2024-09-10T16:02:44.947767+0000","last_fullsized":"2024-09-10T16:02:44.947767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21478,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939514+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:44.939514+0000","last_peered":"2024-09-10T16:02:44.939514+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:44.939514+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941784+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941784+0000","last_peered":"2024-09-10T16:02:44.941784+0000","last_clean":"2024-09-10T16:02:44.941784+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941784+0000","last_undegraded":"2024-09-10T16:02:44.941784+0000","last_fullsized":"2024-09-10T16:02:44.941784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21603,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939920+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:44.939920+0000","last_peered":"2024-09-10T16:02:44.939920+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:44.939920+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947856+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:02:44.947856+0000","last_peered":"2024-09-10T16:02:44.947856+0000","last_clean":"2024-09-10T16:02:44.947856+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:02:44.947856+0000","last_undegraded":"2024-09-10T16:02:44.947856+0000","last_fullsized":"2024-09-10T16:02:44.947856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947928+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:02:44.947928+0000","last_peered":"2024-09-10T16:02:44.947928+0000","last_clean":"2024-09-10T16:02:44.947928+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:02:44.947928+0000","last_undegraded":"2024-09-10T16:02:44.947928+0000","last_fullsized":"2024-09-10T16:02:44.947928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21870,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939551+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:44.939551+0000","last_peered":"2024-09-10T16:02:44.939551+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:44.939551+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20040,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191016960,"data_stored":190955539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626553,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495800,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":495291,"kb_avail":93876040,"statfs":{"total":96636764160,"available":96129064960,"internally_reserved":0,"allocated":499712,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507178670},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149834,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527016,"kb_used_data":62232,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844824,"statfs":{"total":96636764160,"available":96097099776,"internally_reserved":0,"allocated":63725568,"data_stored":63428030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953605,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:00.482 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:00.808 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:00.809 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:00.822 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-10T16:03:00.648849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20042,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1504416,"kb_used_data":187824,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281611104,"statfs":{"total":289910292480,"available":288369770496,"internally_reserved":0,"allocated":192331776,"data_stored":191388580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1348127749},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1630,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002149"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.045741+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:02:45.045741+0000","last_peered":"2024-09-10T16:02:45.045741+0000","last_clean":"2024-09-10T16:02:45.045741+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:02:45.045741+0000","last_undegraded":"2024-09-10T16:02:45.045741+0000","last_fullsized":"2024-09-10T16:02:45.045741+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20472,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.938106+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:44.938106+0000","last_peered":"2024-09-10T16:02:44.938106+0000","last_clean":"2024-09-10T16:02:44.938106+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:44.938106+0000","last_undegraded":"2024-09-10T16:02:44.938106+0000","last_fullsized":"2024-09-10T16:02:44.938106+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28770,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946705+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:44.946705+0000","last_peered":"2024-09-10T16:02:44.946705+0000","last_clean":"2024-09-10T16:02:44.946705+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:44.946705+0000","last_undegraded":"2024-09-10T16:02:44.946705+0000","last_fullsized":"2024-09-10T16:02:44.946705+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938299+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:44.938299+0000","last_peered":"2024-09-10T16:02:44.938299+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:44.938299+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21060,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937941+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:44.937941+0000","last_peered":"2024-09-10T16:02:44.937941+0000","last_clean":"2024-09-10T16:02:44.937941+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:44.937941+0000","last_undegraded":"2024-09-10T16:02:44.937941+0000","last_fullsized":"2024-09-10T16:02:44.937941+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946573+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:02:44.946573+0000","last_peered":"2024-09-10T16:02:44.946573+0000","last_clean":"2024-09-10T16:02:44.946573+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:02:44.946573+0000","last_undegraded":"2024-09-10T16:02:44.946573+0000","last_fullsized":"2024-09-10T16:02:44.946573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28861,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947768+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:44.947768+0000","last_peered":"2024-09-10T16:02:44.947768+0000","last_clean":"2024-09-10T16:02:44.947768+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:44.947768+0000","last_undegraded":"2024-09-10T16:02:44.947768+0000","last_fullsized":"2024-09-10T16:02:44.947768+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940865+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940865+0000","last_peered":"2024-09-10T16:02:44.940865+0000","last_clean":"2024-09-10T16:02:44.940865+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940865+0000","last_undegraded":"2024-09-10T16:02:44.940865+0000","last_fullsized":"2024-09-10T16:02:44.940865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21408,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938137+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:44.938137+0000","last_peered":"2024-09-10T16:02:44.938137+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:44.938137+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":116,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937964+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:02:44.937964+0000","last_peered":"2024-09-10T16:02:44.937964+0000","last_clean":"2024-09-10T16:02:44.937964+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:02:44.937964+0000","last_undegraded":"2024-09-10T16:02:44.937964+0000","last_fullsized":"2024-09-10T16:02:44.937964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28600,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941113+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941113+0000","last_peered":"2024-09-10T16:02:44.941113+0000","last_clean":"2024-09-10T16:02:44.941113+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941113+0000","last_undegraded":"2024-09-10T16:02:44.941113+0000","last_fullsized":"2024-09-10T16:02:44.941113+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947763+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:02:44.947763+0000","last_peered":"2024-09-10T16:02:44.947763+0000","last_clean":"2024-09-10T16:02:44.947763+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:02:44.947763+0000","last_undegraded":"2024-09-10T16:02:44.947763+0000","last_fullsized":"2024-09-10T16:02:44.947763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":25085,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:56.202158+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:02:56.202158+0000","last_peered":"2024-09-10T16:02:56.202158+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:56.202158+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21706,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938859+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:44.938859+0000","last_peered":"2024-09-10T16:02:44.938859+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:44.938859+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947767+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:02:44.947767+0000","last_peered":"2024-09-10T16:02:44.947767+0000","last_clean":"2024-09-10T16:02:44.947767+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:02:44.947767+0000","last_undegraded":"2024-09-10T16:02:44.947767+0000","last_fullsized":"2024-09-10T16:02:44.947767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21478,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939514+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:44.939514+0000","last_peered":"2024-09-10T16:02:44.939514+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:44.939514+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941784+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941784+0000","last_peered":"2024-09-10T16:02:44.941784+0000","last_clean":"2024-09-10T16:02:44.941784+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941784+0000","last_undegraded":"2024-09-10T16:02:44.941784+0000","last_fullsized":"2024-09-10T16:02:44.941784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21603,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939920+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:44.939920+0000","last_peered":"2024-09-10T16:02:44.939920+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:44.939920+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947856+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:02:44.947856+0000","last_peered":"2024-09-10T16:02:44.947856+0000","last_clean":"2024-09-10T16:02:44.947856+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:02:44.947856+0000","last_undegraded":"2024-09-10T16:02:44.947856+0000","last_fullsized":"2024-09-10T16:02:44.947856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947928+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:02:44.947928+0000","last_peered":"2024-09-10T16:02:44.947928+0000","last_clean":"2024-09-10T16:02:44.947928+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:02:44.947928+0000","last_undegraded":"2024-09-10T16:02:44.947928+0000","last_fullsized":"2024-09-10T16:02:44.947928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21870,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939551+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:44.939551+0000","last_peered":"2024-09-10T16:02:44.939551+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:44.939551+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20040,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191016960,"data_stored":190955539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626553,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495800,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":495291,"kb_avail":93876040,"statfs":{"total":96636764160,"available":96129064960,"internally_reserved":0,"allocated":499712,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507178670},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149834,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527016,"kb_used_data":62232,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844824,"statfs":{"total":96636764160,"available":96097099776,"internally_reserved":0,"allocated":63725568,"data_stored":63428030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953605,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:00.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:03:00.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 21115, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.941634+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:44.941634+0000', 'last_peered': '2024-09-10T16:02:44.941634+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:44.941634+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:00.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:00.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21061, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938299+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:44.938299+0000', 'last_peered': '2024-09-10T16:02:44.938299+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:44.938299+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:00.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:00.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21408, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938137+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:44.938137+0000', 'last_peered': '2024-09-10T16:02:44.938137+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:44.938137+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:00.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:03:00.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'20456", 'reported_seq': 25085, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:56.202158+0000', 'last_change': '2024-09-10T16:02:43.316268+0000', 'last_active': '2024-09-10T16:02:56.202158+0000', 'last_peered': '2024-09-10T16:02:56.202158+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:02:56.202158+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'10400", 'ondisk_log_start': "46'10400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1588, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18325, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2243, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1588}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:00.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:00.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20854", 'reported_seq': 21706, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938859+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:44.938859+0000', 'last_peered': '2024-09-10T16:02:44.938859+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:44.938859+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10800", 'ondisk_log_start': "46'10800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18046, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3478}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:00.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:00.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21478, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.939514+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:44.939514+0000', 'last_peered': '2024-09-10T16:02:44.939514+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:44.939514+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:00.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:00.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21603, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.939920+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:44.939920+0000', 'last_peered': '2024-09-10T16:02:44.939920+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:44.939920+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:00.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:00.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21870, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.939551+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:44.939551+0000', 'last_peered': '2024-09-10T16:02:44.939551+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:44.939551+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:00.825 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:01.209 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:01.209 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26608/6 objects degraded (443466.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":707,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1540521984,"bytes_avail":288369770496,"bytes_total":289910292480,"degraded_objects":26608,"degraded_total":6,"degraded_ratio":4434.666666666667,"recovering_objects_per_sec":135,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1630,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:01.222 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:01.222 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:01.535 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:01.535 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:01.548 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-10T16:03:00.648849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20042,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1504416,"kb_used_data":187824,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281611104,"statfs":{"total":289910292480,"available":288369770496,"internally_reserved":0,"allocated":192331776,"data_stored":191388580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1348127749},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1630,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002149"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.045741+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:02:45.045741+0000","last_peered":"2024-09-10T16:02:45.045741+0000","last_clean":"2024-09-10T16:02:45.045741+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:02:45.045741+0000","last_undegraded":"2024-09-10T16:02:45.045741+0000","last_fullsized":"2024-09-10T16:02:45.045741+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20472,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.938106+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:44.938106+0000","last_peered":"2024-09-10T16:02:44.938106+0000","last_clean":"2024-09-10T16:02:44.938106+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:44.938106+0000","last_undegraded":"2024-09-10T16:02:44.938106+0000","last_fullsized":"2024-09-10T16:02:44.938106+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28770,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946705+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:44.946705+0000","last_peered":"2024-09-10T16:02:44.946705+0000","last_clean":"2024-09-10T16:02:44.946705+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:44.946705+0000","last_undegraded":"2024-09-10T16:02:44.946705+0000","last_fullsized":"2024-09-10T16:02:44.946705+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938299+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:44.938299+0000","last_peered":"2024-09-10T16:02:44.938299+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:44.938299+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21060,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937941+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:44.937941+0000","last_peered":"2024-09-10T16:02:44.937941+0000","last_clean":"2024-09-10T16:02:44.937941+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:44.937941+0000","last_undegraded":"2024-09-10T16:02:44.937941+0000","last_fullsized":"2024-09-10T16:02:44.937941+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946573+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:02:44.946573+0000","last_peered":"2024-09-10T16:02:44.946573+0000","last_clean":"2024-09-10T16:02:44.946573+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:02:44.946573+0000","last_undegraded":"2024-09-10T16:02:44.946573+0000","last_fullsized":"2024-09-10T16:02:44.946573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28861,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947768+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:44.947768+0000","last_peered":"2024-09-10T16:02:44.947768+0000","last_clean":"2024-09-10T16:02:44.947768+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:44.947768+0000","last_undegraded":"2024-09-10T16:02:44.947768+0000","last_fullsized":"2024-09-10T16:02:44.947768+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940865+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940865+0000","last_peered":"2024-09-10T16:02:44.940865+0000","last_clean":"2024-09-10T16:02:44.940865+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940865+0000","last_undegraded":"2024-09-10T16:02:44.940865+0000","last_fullsized":"2024-09-10T16:02:44.940865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21408,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938137+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:44.938137+0000","last_peered":"2024-09-10T16:02:44.938137+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:44.938137+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":116,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937964+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:02:44.937964+0000","last_peered":"2024-09-10T16:02:44.937964+0000","last_clean":"2024-09-10T16:02:44.937964+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:02:44.937964+0000","last_undegraded":"2024-09-10T16:02:44.937964+0000","last_fullsized":"2024-09-10T16:02:44.937964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28600,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941113+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941113+0000","last_peered":"2024-09-10T16:02:44.941113+0000","last_clean":"2024-09-10T16:02:44.941113+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941113+0000","last_undegraded":"2024-09-10T16:02:44.941113+0000","last_fullsized":"2024-09-10T16:02:44.941113+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947763+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:02:44.947763+0000","last_peered":"2024-09-10T16:02:44.947763+0000","last_clean":"2024-09-10T16:02:44.947763+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:02:44.947763+0000","last_undegraded":"2024-09-10T16:02:44.947763+0000","last_fullsized":"2024-09-10T16:02:44.947763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":25085,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:56.202158+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:02:56.202158+0000","last_peered":"2024-09-10T16:02:56.202158+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:02:56.202158+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21706,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938859+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:44.938859+0000","last_peered":"2024-09-10T16:02:44.938859+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:44.938859+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947767+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:02:44.947767+0000","last_peered":"2024-09-10T16:02:44.947767+0000","last_clean":"2024-09-10T16:02:44.947767+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:02:44.947767+0000","last_undegraded":"2024-09-10T16:02:44.947767+0000","last_fullsized":"2024-09-10T16:02:44.947767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21478,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939514+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:44.939514+0000","last_peered":"2024-09-10T16:02:44.939514+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:44.939514+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941784+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941784+0000","last_peered":"2024-09-10T16:02:44.941784+0000","last_clean":"2024-09-10T16:02:44.941784+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941784+0000","last_undegraded":"2024-09-10T16:02:44.941784+0000","last_fullsized":"2024-09-10T16:02:44.941784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21603,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939920+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:44.939920+0000","last_peered":"2024-09-10T16:02:44.939920+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:44.939920+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947856+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:02:44.947856+0000","last_peered":"2024-09-10T16:02:44.947856+0000","last_clean":"2024-09-10T16:02:44.947856+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:02:44.947856+0000","last_undegraded":"2024-09-10T16:02:44.947856+0000","last_fullsized":"2024-09-10T16:02:44.947856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947928+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:02:44.947928+0000","last_peered":"2024-09-10T16:02:44.947928+0000","last_clean":"2024-09-10T16:02:44.947928+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:02:44.947928+0000","last_undegraded":"2024-09-10T16:02:44.947928+0000","last_fullsized":"2024-09-10T16:02:44.947928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21870,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939551+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:44.939551+0000","last_peered":"2024-09-10T16:02:44.939551+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:44.939551+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20040,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191016960,"data_stored":190955539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626553,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495800,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":495291,"kb_avail":93876040,"statfs":{"total":96636764160,"available":96129064960,"internally_reserved":0,"allocated":499712,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507178670},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149834,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527016,"kb_used_data":62232,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844824,"statfs":{"total":96636764160,"available":96097099776,"internally_reserved":0,"allocated":63725568,"data_stored":63428030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953605,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:04.550 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:04.888 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:04.888 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:04.900 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-09-10T16:03:04.649658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21001,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496096,"kb_used_data":187824,"kb_used_omap":12,"kb_used_meta":1308211,"kb_avail":281619424,"statfs":{"total":289910292480,"available":288378290176,"internally_reserved":0,"allocated":192331776,"data_stored":191388580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1339608069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1710,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002268"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.045741+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:02:45.045741+0000","last_peered":"2024-09-10T16:02:45.045741+0000","last_clean":"2024-09-10T16:02:45.045741+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:02:45.045741+0000","last_undegraded":"2024-09-10T16:02:45.045741+0000","last_fullsized":"2024-09-10T16:02:45.045741+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20472,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.938106+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:44.938106+0000","last_peered":"2024-09-10T16:02:44.938106+0000","last_clean":"2024-09-10T16:02:44.938106+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:44.938106+0000","last_undegraded":"2024-09-10T16:02:44.938106+0000","last_fullsized":"2024-09-10T16:02:44.938106+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28770,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946705+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:44.946705+0000","last_peered":"2024-09-10T16:02:44.946705+0000","last_clean":"2024-09-10T16:02:44.946705+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:44.946705+0000","last_undegraded":"2024-09-10T16:02:44.946705+0000","last_fullsized":"2024-09-10T16:02:44.946705+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938299+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:44.938299+0000","last_peered":"2024-09-10T16:02:44.938299+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:44.938299+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21060,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937941+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:44.937941+0000","last_peered":"2024-09-10T16:02:44.937941+0000","last_clean":"2024-09-10T16:02:44.937941+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:44.937941+0000","last_undegraded":"2024-09-10T16:02:44.937941+0000","last_fullsized":"2024-09-10T16:02:44.937941+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946573+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:02:44.946573+0000","last_peered":"2024-09-10T16:02:44.946573+0000","last_clean":"2024-09-10T16:02:44.946573+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:02:44.946573+0000","last_undegraded":"2024-09-10T16:02:44.946573+0000","last_fullsized":"2024-09-10T16:02:44.946573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28861,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947768+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:44.947768+0000","last_peered":"2024-09-10T16:02:44.947768+0000","last_clean":"2024-09-10T16:02:44.947768+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:44.947768+0000","last_undegraded":"2024-09-10T16:02:44.947768+0000","last_fullsized":"2024-09-10T16:02:44.947768+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940865+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940865+0000","last_peered":"2024-09-10T16:02:44.940865+0000","last_clean":"2024-09-10T16:02:44.940865+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940865+0000","last_undegraded":"2024-09-10T16:02:44.940865+0000","last_fullsized":"2024-09-10T16:02:44.940865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21408,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938137+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:44.938137+0000","last_peered":"2024-09-10T16:02:44.938137+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:44.938137+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":116,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937964+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:02:44.937964+0000","last_peered":"2024-09-10T16:02:44.937964+0000","last_clean":"2024-09-10T16:02:44.937964+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:02:44.937964+0000","last_undegraded":"2024-09-10T16:02:44.937964+0000","last_fullsized":"2024-09-10T16:02:44.937964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28600,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941113+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941113+0000","last_peered":"2024-09-10T16:02:44.941113+0000","last_clean":"2024-09-10T16:02:44.941113+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941113+0000","last_undegraded":"2024-09-10T16:02:44.941113+0000","last_fullsized":"2024-09-10T16:02:44.941113+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947763+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:02:44.947763+0000","last_peered":"2024-09-10T16:02:44.947763+0000","last_clean":"2024-09-10T16:02:44.947763+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:02:44.947763+0000","last_undegraded":"2024-09-10T16:02:44.947763+0000","last_fullsized":"2024-09-10T16:02:44.947763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":27003,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:01.201516+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:03:01.201516+0000","last_peered":"2024-09-10T16:03:01.201516+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:03:01.201516+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3202,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-629}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21706,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938859+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:44.938859+0000","last_peered":"2024-09-10T16:02:44.938859+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:44.938859+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947767+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:02:44.947767+0000","last_peered":"2024-09-10T16:02:44.947767+0000","last_clean":"2024-09-10T16:02:44.947767+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:02:44.947767+0000","last_undegraded":"2024-09-10T16:02:44.947767+0000","last_fullsized":"2024-09-10T16:02:44.947767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21478,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939514+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:44.939514+0000","last_peered":"2024-09-10T16:02:44.939514+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:44.939514+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941784+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941784+0000","last_peered":"2024-09-10T16:02:44.941784+0000","last_clean":"2024-09-10T16:02:44.941784+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941784+0000","last_undegraded":"2024-09-10T16:02:44.941784+0000","last_fullsized":"2024-09-10T16:02:44.941784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21603,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939920+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:44.939920+0000","last_peered":"2024-09-10T16:02:44.939920+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:44.939920+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947856+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:02:44.947856+0000","last_peered":"2024-09-10T16:02:44.947856+0000","last_clean":"2024-09-10T16:02:44.947856+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:02:44.947856+0000","last_undegraded":"2024-09-10T16:02:44.947856+0000","last_fullsized":"2024-09-10T16:02:44.947856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947928+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:02:44.947928+0000","last_peered":"2024-09-10T16:02:44.947928+0000","last_clean":"2024-09-10T16:02:44.947928+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:02:44.947928+0000","last_undegraded":"2024-09-10T16:02:44.947928+0000","last_fullsized":"2024-09-10T16:02:44.947928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21870,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939551+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:44.939551+0000","last_peered":"2024-09-10T16:02:44.939551+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:44.939551+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20999,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191016960,"data_stored":190955539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487480,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884360,"statfs":{"total":96636764160,"available":96137584640,"internally_reserved":0,"allocated":499712,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498658990},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149835,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527016,"kb_used_data":62232,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844824,"statfs":{"total":96636764160,"available":96097099776,"internally_reserved":0,"allocated":63725568,"data_stored":63428030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953606,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:04.902 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:05.272 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:05.272 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:05.287 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-10T16:03:05.078160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21001,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496096,"kb_used_data":187824,"kb_used_omap":12,"kb_used_meta":1308211,"kb_avail":281619424,"statfs":{"total":289910292480,"available":288378290176,"internally_reserved":0,"allocated":192331776,"data_stored":191388580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1339608069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1710,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.430420"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.045741+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:02:45.045741+0000","last_peered":"2024-09-10T16:02:45.045741+0000","last_clean":"2024-09-10T16:02:45.045741+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:02:45.045741+0000","last_undegraded":"2024-09-10T16:02:45.045741+0000","last_fullsized":"2024-09-10T16:02:45.045741+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20472,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.938106+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:44.938106+0000","last_peered":"2024-09-10T16:02:44.938106+0000","last_clean":"2024-09-10T16:02:44.938106+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:44.938106+0000","last_undegraded":"2024-09-10T16:02:44.938106+0000","last_fullsized":"2024-09-10T16:02:44.938106+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28770,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946705+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:44.946705+0000","last_peered":"2024-09-10T16:02:44.946705+0000","last_clean":"2024-09-10T16:02:44.946705+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:44.946705+0000","last_undegraded":"2024-09-10T16:02:44.946705+0000","last_fullsized":"2024-09-10T16:02:44.946705+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938299+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:44.938299+0000","last_peered":"2024-09-10T16:02:44.938299+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:44.938299+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21060,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937941+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:44.937941+0000","last_peered":"2024-09-10T16:02:44.937941+0000","last_clean":"2024-09-10T16:02:44.937941+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:44.937941+0000","last_undegraded":"2024-09-10T16:02:44.937941+0000","last_fullsized":"2024-09-10T16:02:44.937941+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946573+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:02:44.946573+0000","last_peered":"2024-09-10T16:02:44.946573+0000","last_clean":"2024-09-10T16:02:44.946573+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:02:44.946573+0000","last_undegraded":"2024-09-10T16:02:44.946573+0000","last_fullsized":"2024-09-10T16:02:44.946573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28861,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947768+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:44.947768+0000","last_peered":"2024-09-10T16:02:44.947768+0000","last_clean":"2024-09-10T16:02:44.947768+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:44.947768+0000","last_undegraded":"2024-09-10T16:02:44.947768+0000","last_fullsized":"2024-09-10T16:02:44.947768+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940865+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940865+0000","last_peered":"2024-09-10T16:02:44.940865+0000","last_clean":"2024-09-10T16:02:44.940865+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940865+0000","last_undegraded":"2024-09-10T16:02:44.940865+0000","last_fullsized":"2024-09-10T16:02:44.940865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21408,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938137+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:44.938137+0000","last_peered":"2024-09-10T16:02:44.938137+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:44.938137+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":116,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937964+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:02:44.937964+0000","last_peered":"2024-09-10T16:02:44.937964+0000","last_clean":"2024-09-10T16:02:44.937964+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:02:44.937964+0000","last_undegraded":"2024-09-10T16:02:44.937964+0000","last_fullsized":"2024-09-10T16:02:44.937964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28600,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941113+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941113+0000","last_peered":"2024-09-10T16:02:44.941113+0000","last_clean":"2024-09-10T16:02:44.941113+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941113+0000","last_undegraded":"2024-09-10T16:02:44.941113+0000","last_fullsized":"2024-09-10T16:02:44.941113+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947763+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:02:44.947763+0000","last_peered":"2024-09-10T16:02:44.947763+0000","last_clean":"2024-09-10T16:02:44.947763+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:02:44.947763+0000","last_undegraded":"2024-09-10T16:02:44.947763+0000","last_fullsized":"2024-09-10T16:02:44.947763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":27003,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:01.201516+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:03:01.201516+0000","last_peered":"2024-09-10T16:03:01.201516+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:03:01.201516+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3202,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-629}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21706,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938859+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:44.938859+0000","last_peered":"2024-09-10T16:02:44.938859+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:44.938859+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947767+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:02:44.947767+0000","last_peered":"2024-09-10T16:02:44.947767+0000","last_clean":"2024-09-10T16:02:44.947767+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:02:44.947767+0000","last_undegraded":"2024-09-10T16:02:44.947767+0000","last_fullsized":"2024-09-10T16:02:44.947767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21478,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939514+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:44.939514+0000","last_peered":"2024-09-10T16:02:44.939514+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:44.939514+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941784+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941784+0000","last_peered":"2024-09-10T16:02:44.941784+0000","last_clean":"2024-09-10T16:02:44.941784+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941784+0000","last_undegraded":"2024-09-10T16:02:44.941784+0000","last_fullsized":"2024-09-10T16:02:44.941784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21603,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939920+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:44.939920+0000","last_peered":"2024-09-10T16:02:44.939920+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:44.939920+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947856+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:02:44.947856+0000","last_peered":"2024-09-10T16:02:44.947856+0000","last_clean":"2024-09-10T16:02:44.947856+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:02:44.947856+0000","last_undegraded":"2024-09-10T16:02:44.947856+0000","last_fullsized":"2024-09-10T16:02:44.947856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947928+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:02:44.947928+0000","last_peered":"2024-09-10T16:02:44.947928+0000","last_clean":"2024-09-10T16:02:44.947928+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:02:44.947928+0000","last_undegraded":"2024-09-10T16:02:44.947928+0000","last_fullsized":"2024-09-10T16:02:44.947928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21870,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939551+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:44.939551+0000","last_peered":"2024-09-10T16:02:44.939551+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:44.939551+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20999,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191016960,"data_stored":190955539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487480,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884360,"statfs":{"total":96636764160,"available":96137584640,"internally_reserved":0,"allocated":499712,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498658990},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149835,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527016,"kb_used_data":62232,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844824,"statfs":{"total":96636764160,"available":96097099776,"internally_reserved":0,"allocated":63725568,"data_stored":63428030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953606,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:05.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:03:05.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 21115, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.941634+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:02:44.941634+0000', 'last_peered': '2024-09-10T16:02:44.941634+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:02:44.941634+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:05.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:05.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21061, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938299+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:02:44.938299+0000', 'last_peered': '2024-09-10T16:02:44.938299+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:02:44.938299+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:05.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21408, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938137+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:02:44.938137+0000', 'last_peered': '2024-09-10T16:02:44.938137+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:02:44.938137+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-10T16:03:05.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'20456", 'reported_seq': 27003, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:01.201516+0000', 'last_change': '2024-09-10T16:02:43.316268+0000', 'last_active': '2024-09-10T16:03:01.201516+0000', 'last_peered': '2024-09-10T16:03:01.201516+0000', 'last_clean': '2024-09-10T15:53:17.327041+0000', 'last_became_active': '2024-09-10T15:53:19.932600+0000', 'last_became_peered': '2024-09-10T15:53:19.932600+0000', 'last_unstale': '2024-09-10T16:03:01.201516+0000', 'last_undegraded': '2024-09-10T15:53:19.509819+0000', 'last_fullsized': '2024-09-10T15:53:19.470694+0000', 'mapping_epoch': 43, 'log_start': "46'10400", 'ondisk_log_start': "46'10400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2154", 'last_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:54.903413+0000', 'objects_scrubbed': 2125, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 979, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18325, 'num_write_kb': 32400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3202, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -629}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:05.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:05.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20854", 'reported_seq': 21706, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.938859+0000', 'last_change': '2024-09-10T15:53:19.938219+0000', 'last_active': '2024-09-10T16:02:44.938859+0000', 'last_peered': '2024-09-10T16:02:44.938859+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:02:44.938859+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10800", 'ondisk_log_start': "46'10800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3479, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18046, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 355, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3478}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:05.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21478, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.939514+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:02:44.939514+0000', 'last_peered': '2024-09-10T16:02:44.939514+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:02:44.939514+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:05.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21603, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.939920+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:02:44.939920+0000', 'last_peered': '2024-09-10T16:02:44.939920+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:02:44.939920+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.290 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:05.290 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21870, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:02:44.939551+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:02:44.939551+0000', 'last_peered': '2024-09-10T16:02:44.939551+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:02:44.939551+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:05.290 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:05.684 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:05.685 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25649/6 objects degraded (427483.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":711,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1532002304,"bytes_avail":288378290176,"bytes_total":289910292480,"degraded_objects":25649,"degraded_total":6,"degraded_ratio":4274.833333333333,"recovering_objects_per_sec":142,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1710,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-10T16:01:10.609441+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:05.697 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:05.697 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:06.007 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:06.007 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:06.020 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-10T16:03:05.078160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21001,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496096,"kb_used_data":187824,"kb_used_omap":12,"kb_used_meta":1308211,"kb_avail":281619424,"statfs":{"total":289910292480,"available":288378290176,"internally_reserved":0,"allocated":192331776,"data_stored":191388580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1339608069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1710,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.430420"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:45.045741+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:02:45.045741+0000","last_peered":"2024-09-10T16:02:45.045741+0000","last_clean":"2024-09-10T16:02:45.045741+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:02:45.045741+0000","last_undegraded":"2024-09-10T16:02:45.045741+0000","last_fullsized":"2024-09-10T16:02:45.045741+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941657+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:02:44.941657+0000","last_peered":"2024-09-10T16:02:44.941657+0000","last_clean":"2024-09-10T16:02:44.941657+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:02:44.941657+0000","last_undegraded":"2024-09-10T16:02:44.941657+0000","last_fullsized":"2024-09-10T16:02:44.941657+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21115,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.941634+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:02:44.941634+0000","last_peered":"2024-09-10T16:02:44.941634+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:02:44.941634+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941536+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:02:44.941536+0000","last_peered":"2024-09-10T16:02:44.941536+0000","last_clean":"2024-09-10T16:02:44.941536+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:02:44.941536+0000","last_undegraded":"2024-09-10T16:02:44.941536+0000","last_fullsized":"2024-09-10T16:02:44.941536+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20472,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.938106+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:02:44.938106+0000","last_peered":"2024-09-10T16:02:44.938106+0000","last_clean":"2024-09-10T16:02:44.938106+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:02:44.938106+0000","last_undegraded":"2024-09-10T16:02:44.938106+0000","last_fullsized":"2024-09-10T16:02:44.938106+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28770,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946705+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:02:44.946705+0000","last_peered":"2024-09-10T16:02:44.946705+0000","last_clean":"2024-09-10T16:02:44.946705+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:02:44.946705+0000","last_undegraded":"2024-09-10T16:02:44.946705+0000","last_fullsized":"2024-09-10T16:02:44.946705+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938299+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:02:44.938299+0000","last_peered":"2024-09-10T16:02:44.938299+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:02:44.938299+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21060,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937941+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:02:44.937941+0000","last_peered":"2024-09-10T16:02:44.937941+0000","last_clean":"2024-09-10T16:02:44.937941+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:02:44.937941+0000","last_undegraded":"2024-09-10T16:02:44.937941+0000","last_fullsized":"2024-09-10T16:02:44.937941+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.946573+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:02:44.946573+0000","last_peered":"2024-09-10T16:02:44.946573+0000","last_clean":"2024-09-10T16:02:44.946573+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:02:44.946573+0000","last_undegraded":"2024-09-10T16:02:44.946573+0000","last_fullsized":"2024-09-10T16:02:44.946573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28861,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947768+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:02:44.947768+0000","last_peered":"2024-09-10T16:02:44.947768+0000","last_clean":"2024-09-10T16:02:44.947768+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:02:44.947768+0000","last_undegraded":"2024-09-10T16:02:44.947768+0000","last_fullsized":"2024-09-10T16:02:44.947768+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.940865+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:02:44.940865+0000","last_peered":"2024-09-10T16:02:44.940865+0000","last_clean":"2024-09-10T16:02:44.940865+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:02:44.940865+0000","last_undegraded":"2024-09-10T16:02:44.940865+0000","last_fullsized":"2024-09-10T16:02:44.940865+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21408,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938137+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:02:44.938137+0000","last_peered":"2024-09-10T16:02:44.938137+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:02:44.938137+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":116,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.937964+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:02:44.937964+0000","last_peered":"2024-09-10T16:02:44.937964+0000","last_clean":"2024-09-10T16:02:44.937964+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:02:44.937964+0000","last_undegraded":"2024-09-10T16:02:44.937964+0000","last_fullsized":"2024-09-10T16:02:44.937964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28600,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941113+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:02:44.941113+0000","last_peered":"2024-09-10T16:02:44.941113+0000","last_clean":"2024-09-10T16:02:44.941113+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:02:44.941113+0000","last_undegraded":"2024-09-10T16:02:44.941113+0000","last_fullsized":"2024-09-10T16:02:44.941113+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947763+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:02:44.947763+0000","last_peered":"2024-09-10T16:02:44.947763+0000","last_clean":"2024-09-10T16:02:44.947763+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:02:44.947763+0000","last_undegraded":"2024-09-10T16:02:44.947763+0000","last_fullsized":"2024-09-10T16:02:44.947763+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":27003,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:01.201516+0000","last_change":"2024-09-10T16:02:43.316268+0000","last_active":"2024-09-10T16:03:01.201516+0000","last_peered":"2024-09-10T16:03:01.201516+0000","last_clean":"2024-09-10T15:53:17.327041+0000","last_became_active":"2024-09-10T15:53:19.932600+0000","last_became_peered":"2024-09-10T15:53:19.932600+0000","last_unstale":"2024-09-10T16:03:01.201516+0000","last_undegraded":"2024-09-10T15:53:19.509819+0000","last_fullsized":"2024-09-10T15:53:19.470694+0000","mapping_epoch":43,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3202,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-629}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21706,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.938859+0000","last_change":"2024-09-10T15:53:19.938219+0000","last_active":"2024-09-10T16:02:44.938859+0000","last_peered":"2024-09-10T16:02:44.938859+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:02:44.938859+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947767+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:02:44.947767+0000","last_peered":"2024-09-10T16:02:44.947767+0000","last_clean":"2024-09-10T16:02:44.947767+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:02:44.947767+0000","last_undegraded":"2024-09-10T16:02:44.947767+0000","last_fullsized":"2024-09-10T16:02:44.947767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21478,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939514+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:02:44.939514+0000","last_peered":"2024-09-10T16:02:44.939514+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:02:44.939514+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.941784+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:02:44.941784+0000","last_peered":"2024-09-10T16:02:44.941784+0000","last_clean":"2024-09-10T16:02:44.941784+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:02:44.941784+0000","last_undegraded":"2024-09-10T16:02:44.941784+0000","last_fullsized":"2024-09-10T16:02:44.941784+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21603,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939920+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:02:44.939920+0000","last_peered":"2024-09-10T16:02:44.939920+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:02:44.939920+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947856+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:02:44.947856+0000","last_peered":"2024-09-10T16:02:44.947856+0000","last_clean":"2024-09-10T16:02:44.947856+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:02:44.947856+0000","last_undegraded":"2024-09-10T16:02:44.947856+0000","last_fullsized":"2024-09-10T16:02:44.947856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-10T16:02:44.947928+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:02:44.947928+0000","last_peered":"2024-09-10T16:02:44.947928+0000","last_clean":"2024-09-10T16:02:44.947928+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:02:44.947928+0000","last_undegraded":"2024-09-10T16:02:44.947928+0000","last_fullsized":"2024-09-10T16:02:44.947928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21870,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:02:44.939551+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:02:44.939551+0000","last_peered":"2024-09-10T16:02:44.939551+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:02:44.939551+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20999,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191016960,"data_stored":190955539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487480,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884360,"statfs":{"total":96636764160,"available":96137584640,"internally_reserved":0,"allocated":499712,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498658990},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149835,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527016,"kb_used_data":62232,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844824,"statfs":{"total":96636764160,"available":96097099776,"internally_reserved":0,"allocated":63725568,"data_stored":63428030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953606,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481600,"kb_used_data":125104,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890240,"statfs":{"total":96636764160,"available":96143605760,"internally_reserved":0,"allocated":128106496,"data_stored":127808958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:09.022 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:09.408 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:09.408 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:09.421 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-10T16:03:08.650406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21970,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500292,"kb_used_data":187860,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281615228,"statfs":{"total":289910292480,"available":288373993472,"internally_reserved":0,"allocated":192368640,"data_stored":191402580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1343867909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1928,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1928,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001556"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21200,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086891+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:06.086891+0000","last_peered":"2024-09-10T16:03:06.086891+0000","last_clean":"2024-09-10T16:03:06.086891+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:06.086891+0000","last_undegraded":"2024-09-10T16:03:06.086891+0000","last_fullsized":"2024-09-10T16:03:06.086891+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21119,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:03:06.088239+0000","last_peered":"2024-09-10T16:03:06.088239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:06.088239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086647+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:06.086647+0000","last_peered":"2024-09-10T16:03:06.086647+0000","last_clean":"2024-09-10T16:03:06.086647+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:06.086647+0000","last_undegraded":"2024-09-10T16:03:06.086647+0000","last_fullsized":"2024-09-10T16:03:06.086647+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20476,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086217+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086217+0000","last_peered":"2024-09-10T16:03:06.086217+0000","last_clean":"2024-09-10T16:03:06.086217+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086217+0000","last_undegraded":"2024-09-10T16:03:06.086217+0000","last_fullsized":"2024-09-10T16:03:06.086217+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21065,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086797+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.086797+0000","last_peered":"2024-09-10T16:03:06.086797+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.086797+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081198+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081198+0000","last_peered":"2024-09-10T16:03:06.081198+0000","last_clean":"2024-09-10T16:03:06.081198+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081198+0000","last_undegraded":"2024-09-10T16:03:06.081198+0000","last_fullsized":"2024-09-10T16:03:06.081198+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087905+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:06.087905+0000","last_peered":"2024-09-10T16:03:06.087905+0000","last_clean":"2024-09-10T16:03:06.087905+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:06.087905+0000","last_undegraded":"2024-09-10T16:03:06.087905+0000","last_fullsized":"2024-09-10T16:03:06.087905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28604,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087967+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:06.087967+0000","last_peered":"2024-09-10T16:03:06.087967+0000","last_clean":"2024-09-10T16:03:06.087967+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:06.087967+0000","last_undegraded":"2024-09-10T16:03:06.087967+0000","last_fullsized":"2024-09-10T16:03:06.087967+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28272,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.170050+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:06.170050+0000","last_peered":"2024-09-10T16:03:06.170050+0000","last_clean":"2024-09-10T16:03:06.170050+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:06.170050+0000","last_undegraded":"2024-09-10T16:03:06.170050+0000","last_fullsized":"2024-09-10T16:03:06.170050+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":22391,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.731133+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:06.731133+0000","last_peered":"2024-09-10T16:03:06.731133+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:06.731133+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":695,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087979+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:06.087979+0000","last_peered":"2024-09-10T16:03:06.087979+0000","last_clean":"2024-09-10T16:03:06.087979+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:06.087979+0000","last_undegraded":"2024-09-10T16:03:06.087979+0000","last_fullsized":"2024-09-10T16:03:06.087979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21968,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191012864,"data_stored":190955056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626555,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491652,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880188,"statfs":{"total":96636764160,"available":96133312512,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502918830},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149836,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953607,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481608,"kb_used_data":125112,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890232,"statfs":{"total":96636764160,"available":96143597568,"internally_reserved":0,"allocated":128114688,"data_stored":127811863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:09.422 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:09.766 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:09.767 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:09.781 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-10T16:03:08.650406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21970,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500292,"kb_used_data":187860,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281615228,"statfs":{"total":289910292480,"available":288373993472,"internally_reserved":0,"allocated":192368640,"data_stored":191402580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1343867909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1928,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1928,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001556"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21200,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086891+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:06.086891+0000","last_peered":"2024-09-10T16:03:06.086891+0000","last_clean":"2024-09-10T16:03:06.086891+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:06.086891+0000","last_undegraded":"2024-09-10T16:03:06.086891+0000","last_fullsized":"2024-09-10T16:03:06.086891+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21119,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:03:06.088239+0000","last_peered":"2024-09-10T16:03:06.088239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:06.088239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086647+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:06.086647+0000","last_peered":"2024-09-10T16:03:06.086647+0000","last_clean":"2024-09-10T16:03:06.086647+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:06.086647+0000","last_undegraded":"2024-09-10T16:03:06.086647+0000","last_fullsized":"2024-09-10T16:03:06.086647+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20476,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086217+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086217+0000","last_peered":"2024-09-10T16:03:06.086217+0000","last_clean":"2024-09-10T16:03:06.086217+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086217+0000","last_undegraded":"2024-09-10T16:03:06.086217+0000","last_fullsized":"2024-09-10T16:03:06.086217+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21065,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086797+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.086797+0000","last_peered":"2024-09-10T16:03:06.086797+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.086797+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081198+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081198+0000","last_peered":"2024-09-10T16:03:06.081198+0000","last_clean":"2024-09-10T16:03:06.081198+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081198+0000","last_undegraded":"2024-09-10T16:03:06.081198+0000","last_fullsized":"2024-09-10T16:03:06.081198+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087905+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:06.087905+0000","last_peered":"2024-09-10T16:03:06.087905+0000","last_clean":"2024-09-10T16:03:06.087905+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:06.087905+0000","last_undegraded":"2024-09-10T16:03:06.087905+0000","last_fullsized":"2024-09-10T16:03:06.087905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28604,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087967+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:06.087967+0000","last_peered":"2024-09-10T16:03:06.087967+0000","last_clean":"2024-09-10T16:03:06.087967+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:06.087967+0000","last_undegraded":"2024-09-10T16:03:06.087967+0000","last_fullsized":"2024-09-10T16:03:06.087967+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28272,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.170050+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:06.170050+0000","last_peered":"2024-09-10T16:03:06.170050+0000","last_clean":"2024-09-10T16:03:06.170050+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:06.170050+0000","last_undegraded":"2024-09-10T16:03:06.170050+0000","last_fullsized":"2024-09-10T16:03:06.170050+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":22391,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.731133+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:06.731133+0000","last_peered":"2024-09-10T16:03:06.731133+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:06.731133+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":695,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087979+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:06.087979+0000","last_peered":"2024-09-10T16:03:06.087979+0000","last_clean":"2024-09-10T16:03:06.087979+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:06.087979+0000","last_undegraded":"2024-09-10T16:03:06.087979+0000","last_fullsized":"2024-09-10T16:03:06.087979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21968,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191012864,"data_stored":190955056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626555,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491652,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880188,"statfs":{"total":96636764160,"available":96133312512,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502918830},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149836,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953607,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481608,"kb_used_data":125112,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890232,"statfs":{"total":96636764160,"available":96143597568,"internally_reserved":0,"allocated":128114688,"data_stored":127811863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:09.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:03:09.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 21119, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.088239+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:03:06.088239+0000', 'last_peered': '2024-09-10T16:03:06.088239+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:03:06.088239+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:09.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:09.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21065, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.086797+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:06.086797+0000', 'last_peered': '2024-09-10T16:03:06.086797+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:06.086797+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:09.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:09.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21412, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.085461+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:06.085461+0000', 'last_peered': '2024-09-10T16:03:06.085461+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:06.085461+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:09.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:09.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20854", 'reported_seq': 22391, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.731133+0000', 'last_change': '2024-09-10T16:03:04.326890+0000', 'last_active': '2024-09-10T16:03:06.731133+0000', 'last_peered': '2024-09-10T16:03:06.731133+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:03:06.731133+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10800", 'ondisk_log_start': "46'10800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18046, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 695, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:09.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:09.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21483, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.086596+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:06.086596+0000', 'last_peered': '2024-09-10T16:03:06.086596+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:06.086596+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:09.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:09.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21607, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.085558+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:03:06.085558+0000', 'last_peered': '2024-09-10T16:03:06.085558+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:03:06.085558+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:09.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:09.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21874, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.086306+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:06.086306+0000', 'last_peered': '2024-09-10T16:03:06.086306+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:06.086306+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:09.784 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:10.249 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24680/6 objects degraded (411333.333%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":716,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1536299008,"bytes_avail":288373993472,"bytes_total":289910292480,"degraded_objects":24680,"degraded_total":6,"degraded_ratio":4113.333333333333,"recovering_objects_per_sec":240,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1928,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-09-10T16:03:08.650481+0000","services":{"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:03:10.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:10.264 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:10.665 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:10.666 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:10.678 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-10T16:03:10.650754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21970,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500292,"kb_used_data":187860,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281615228,"statfs":{"total":289910292480,"available":288373993472,"internally_reserved":0,"allocated":192368640,"data_stored":191402580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1343867909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-969,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":969,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001575"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21200,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086891+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:06.086891+0000","last_peered":"2024-09-10T16:03:06.086891+0000","last_clean":"2024-09-10T16:03:06.086891+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:06.086891+0000","last_undegraded":"2024-09-10T16:03:06.086891+0000","last_fullsized":"2024-09-10T16:03:06.086891+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21119,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:03:06.088239+0000","last_peered":"2024-09-10T16:03:06.088239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:06.088239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086647+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:06.086647+0000","last_peered":"2024-09-10T16:03:06.086647+0000","last_clean":"2024-09-10T16:03:06.086647+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:06.086647+0000","last_undegraded":"2024-09-10T16:03:06.086647+0000","last_fullsized":"2024-09-10T16:03:06.086647+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20476,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086217+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086217+0000","last_peered":"2024-09-10T16:03:06.086217+0000","last_clean":"2024-09-10T16:03:06.086217+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086217+0000","last_undegraded":"2024-09-10T16:03:06.086217+0000","last_fullsized":"2024-09-10T16:03:06.086217+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21065,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086797+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.086797+0000","last_peered":"2024-09-10T16:03:06.086797+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.086797+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081198+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081198+0000","last_peered":"2024-09-10T16:03:06.081198+0000","last_clean":"2024-09-10T16:03:06.081198+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081198+0000","last_undegraded":"2024-09-10T16:03:06.081198+0000","last_fullsized":"2024-09-10T16:03:06.081198+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087905+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:06.087905+0000","last_peered":"2024-09-10T16:03:06.087905+0000","last_clean":"2024-09-10T16:03:06.087905+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:06.087905+0000","last_undegraded":"2024-09-10T16:03:06.087905+0000","last_fullsized":"2024-09-10T16:03:06.087905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28604,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087967+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:06.087967+0000","last_peered":"2024-09-10T16:03:06.087967+0000","last_clean":"2024-09-10T16:03:06.087967+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:06.087967+0000","last_undegraded":"2024-09-10T16:03:06.087967+0000","last_fullsized":"2024-09-10T16:03:06.087967+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28272,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.170050+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:06.170050+0000","last_peered":"2024-09-10T16:03:06.170050+0000","last_clean":"2024-09-10T16:03:06.170050+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:06.170050+0000","last_undegraded":"2024-09-10T16:03:06.170050+0000","last_fullsized":"2024-09-10T16:03:06.170050+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":22391,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.731133+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:06.731133+0000","last_peered":"2024-09-10T16:03:06.731133+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:06.731133+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":695,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087979+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:06.087979+0000","last_peered":"2024-09-10T16:03:06.087979+0000","last_clean":"2024-09-10T16:03:06.087979+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:06.087979+0000","last_undegraded":"2024-09-10T16:03:06.087979+0000","last_fullsized":"2024-09-10T16:03:06.087979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21968,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191012864,"data_stored":190955056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626555,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491652,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880188,"statfs":{"total":96636764160,"available":96133312512,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502918830},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149836,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953607,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481608,"kb_used_data":125112,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890232,"statfs":{"total":96636764160,"available":96143597568,"internally_reserved":0,"allocated":128114688,"data_stored":127811863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:13.681 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:14.014 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:14.015 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:14.028 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-10T16:03:12.651172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23042,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491980,"kb_used_data":187868,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623540,"statfs":{"total":289910292480,"available":288382504960,"internally_reserved":0,"allocated":192376832,"data_stored":191405464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001514"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21200,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086891+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:06.086891+0000","last_peered":"2024-09-10T16:03:06.086891+0000","last_clean":"2024-09-10T16:03:06.086891+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:06.086891+0000","last_undegraded":"2024-09-10T16:03:06.086891+0000","last_fullsized":"2024-09-10T16:03:06.086891+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21119,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:03:06.088239+0000","last_peered":"2024-09-10T16:03:06.088239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:06.088239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086647+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:06.086647+0000","last_peered":"2024-09-10T16:03:06.086647+0000","last_clean":"2024-09-10T16:03:06.086647+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:06.086647+0000","last_undegraded":"2024-09-10T16:03:06.086647+0000","last_fullsized":"2024-09-10T16:03:06.086647+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20476,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086217+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086217+0000","last_peered":"2024-09-10T16:03:06.086217+0000","last_clean":"2024-09-10T16:03:06.086217+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086217+0000","last_undegraded":"2024-09-10T16:03:06.086217+0000","last_fullsized":"2024-09-10T16:03:06.086217+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21065,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086797+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.086797+0000","last_peered":"2024-09-10T16:03:06.086797+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.086797+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081198+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081198+0000","last_peered":"2024-09-10T16:03:06.081198+0000","last_clean":"2024-09-10T16:03:06.081198+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081198+0000","last_undegraded":"2024-09-10T16:03:06.081198+0000","last_fullsized":"2024-09-10T16:03:06.081198+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087905+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:06.087905+0000","last_peered":"2024-09-10T16:03:06.087905+0000","last_clean":"2024-09-10T16:03:06.087905+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:06.087905+0000","last_undegraded":"2024-09-10T16:03:06.087905+0000","last_fullsized":"2024-09-10T16:03:06.087905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28604,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087967+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:06.087967+0000","last_peered":"2024-09-10T16:03:06.087967+0000","last_clean":"2024-09-10T16:03:06.087967+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:06.087967+0000","last_undegraded":"2024-09-10T16:03:06.087967+0000","last_fullsized":"2024-09-10T16:03:06.087967+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28272,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.170050+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:06.170050+0000","last_peered":"2024-09-10T16:03:06.170050+0000","last_clean":"2024-09-10T16:03:06.170050+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:06.170050+0000","last_undegraded":"2024-09-10T16:03:06.170050+0000","last_fullsized":"2024-09-10T16:03:06.170050+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":24535,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:11.734377+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:11.734377+0000","last_peered":"2024-09-10T16:03:11.734377+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:11.734377+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1767,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2067}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087979+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:06.087979+0000","last_peered":"2024-09-10T16:03:06.087979+0000","last_clean":"2024-09-10T16:03:06.087979+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:06.087979+0000","last_undegraded":"2024-09-10T16:03:06.087979+0000","last_fullsized":"2024-09-10T16:03:06.087979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23040,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191012864,"data_stored":190955056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626556,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953608,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481616,"kb_used_data":125120,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890224,"statfs":{"total":96636764160,"available":96143589376,"internally_reserved":0,"allocated":128122880,"data_stored":127814747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:14.031 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:14.356 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:14.357 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:14.369 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-10T16:03:12.651172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23042,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491980,"kb_used_data":187868,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623540,"statfs":{"total":289910292480,"available":288382504960,"internally_reserved":0,"allocated":192376832,"data_stored":191405464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001514"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21200,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086891+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:06.086891+0000","last_peered":"2024-09-10T16:03:06.086891+0000","last_clean":"2024-09-10T16:03:06.086891+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:06.086891+0000","last_undegraded":"2024-09-10T16:03:06.086891+0000","last_fullsized":"2024-09-10T16:03:06.086891+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21119,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:03:06.088239+0000","last_peered":"2024-09-10T16:03:06.088239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:06.088239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086647+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:06.086647+0000","last_peered":"2024-09-10T16:03:06.086647+0000","last_clean":"2024-09-10T16:03:06.086647+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:06.086647+0000","last_undegraded":"2024-09-10T16:03:06.086647+0000","last_fullsized":"2024-09-10T16:03:06.086647+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20476,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086217+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086217+0000","last_peered":"2024-09-10T16:03:06.086217+0000","last_clean":"2024-09-10T16:03:06.086217+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086217+0000","last_undegraded":"2024-09-10T16:03:06.086217+0000","last_fullsized":"2024-09-10T16:03:06.086217+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21065,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086797+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.086797+0000","last_peered":"2024-09-10T16:03:06.086797+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.086797+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081198+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081198+0000","last_peered":"2024-09-10T16:03:06.081198+0000","last_clean":"2024-09-10T16:03:06.081198+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081198+0000","last_undegraded":"2024-09-10T16:03:06.081198+0000","last_fullsized":"2024-09-10T16:03:06.081198+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087905+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:06.087905+0000","last_peered":"2024-09-10T16:03:06.087905+0000","last_clean":"2024-09-10T16:03:06.087905+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:06.087905+0000","last_undegraded":"2024-09-10T16:03:06.087905+0000","last_fullsized":"2024-09-10T16:03:06.087905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28604,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087967+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:06.087967+0000","last_peered":"2024-09-10T16:03:06.087967+0000","last_clean":"2024-09-10T16:03:06.087967+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:06.087967+0000","last_undegraded":"2024-09-10T16:03:06.087967+0000","last_fullsized":"2024-09-10T16:03:06.087967+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28272,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.170050+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:06.170050+0000","last_peered":"2024-09-10T16:03:06.170050+0000","last_clean":"2024-09-10T16:03:06.170050+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:06.170050+0000","last_undegraded":"2024-09-10T16:03:06.170050+0000","last_fullsized":"2024-09-10T16:03:06.170050+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":24535,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:11.734377+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:11.734377+0000","last_peered":"2024-09-10T16:03:11.734377+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:11.734377+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1767,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2067}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087979+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:06.087979+0000","last_peered":"2024-09-10T16:03:06.087979+0000","last_clean":"2024-09-10T16:03:06.087979+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:06.087979+0000","last_undegraded":"2024-09-10T16:03:06.087979+0000","last_fullsized":"2024-09-10T16:03:06.087979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23040,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191012864,"data_stored":190955056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626556,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953608,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481616,"kb_used_data":125120,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890224,"statfs":{"total":96636764160,"available":96143589376,"internally_reserved":0,"allocated":128122880,"data_stored":127814747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:14.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:03:14.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 21119, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.088239+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:03:06.088239+0000', 'last_peered': '2024-09-10T16:03:06.088239+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:03:06.088239+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:14.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:14.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21065, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.086797+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:06.086797+0000', 'last_peered': '2024-09-10T16:03:06.086797+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:06.086797+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:14.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:14.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21412, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.085461+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:06.085461+0000', 'last_peered': '2024-09-10T16:03:06.085461+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:06.085461+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:14.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:14.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20854", 'reported_seq': 24535, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:11.734377+0000', 'last_change': '2024-09-10T16:03:04.326890+0000', 'last_active': '2024-09-10T16:03:11.734377+0000', 'last_peered': '2024-09-10T16:03:11.734377+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:03:11.734377+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10800", 'ondisk_log_start': "46'10800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18046, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1767, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2067}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:14.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:14.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21483, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.086596+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:06.086596+0000', 'last_peered': '2024-09-10T16:03:06.086596+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:06.086596+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:14.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:14.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21607, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.085558+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:03:06.085558+0000', 'last_peered': '2024-09-10T16:03:06.085558+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:03:06.085558+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:14.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:14.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21874, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.086306+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:06.086306+0000', 'last_peered': '2024-09-10T16:03:06.086306+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:06.086306+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:14.372 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:14.752 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:14.752 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23608/6 objects degraded (393466.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":720,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1527787520,"bytes_avail":288382504960,"bytes_total":289910292480,"degraded_objects":23608,"degraded_total":6,"degraded_ratio":3934.6666666666661,"recovering_objects_per_sec":255,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2041,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-09-10T16:03:08.650481+0000","services":{"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:03:14.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:14.766 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:15.058 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:15.070 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-10T16:03:14.651531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23042,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491980,"kb_used_data":187868,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623540,"statfs":{"total":289910292480,"available":288382504960,"internally_reserved":0,"allocated":192376832,"data_stored":191405464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.573371"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21200,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086891+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:06.086891+0000","last_peered":"2024-09-10T16:03:06.086891+0000","last_clean":"2024-09-10T16:03:06.086891+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:06.086891+0000","last_undegraded":"2024-09-10T16:03:06.086891+0000","last_fullsized":"2024-09-10T16:03:06.086891+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21119,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:03:06.088239+0000","last_peered":"2024-09-10T16:03:06.088239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:06.088239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086647+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:06.086647+0000","last_peered":"2024-09-10T16:03:06.086647+0000","last_clean":"2024-09-10T16:03:06.086647+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:06.086647+0000","last_undegraded":"2024-09-10T16:03:06.086647+0000","last_fullsized":"2024-09-10T16:03:06.086647+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20476,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086217+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086217+0000","last_peered":"2024-09-10T16:03:06.086217+0000","last_clean":"2024-09-10T16:03:06.086217+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086217+0000","last_undegraded":"2024-09-10T16:03:06.086217+0000","last_fullsized":"2024-09-10T16:03:06.086217+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21065,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086797+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.086797+0000","last_peered":"2024-09-10T16:03:06.086797+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.086797+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081198+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081198+0000","last_peered":"2024-09-10T16:03:06.081198+0000","last_clean":"2024-09-10T16:03:06.081198+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081198+0000","last_undegraded":"2024-09-10T16:03:06.081198+0000","last_fullsized":"2024-09-10T16:03:06.081198+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087905+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:06.087905+0000","last_peered":"2024-09-10T16:03:06.087905+0000","last_clean":"2024-09-10T16:03:06.087905+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:06.087905+0000","last_undegraded":"2024-09-10T16:03:06.087905+0000","last_fullsized":"2024-09-10T16:03:06.087905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28604,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087967+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:06.087967+0000","last_peered":"2024-09-10T16:03:06.087967+0000","last_clean":"2024-09-10T16:03:06.087967+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:06.087967+0000","last_undegraded":"2024-09-10T16:03:06.087967+0000","last_fullsized":"2024-09-10T16:03:06.087967+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28272,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.170050+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:06.170050+0000","last_peered":"2024-09-10T16:03:06.170050+0000","last_clean":"2024-09-10T16:03:06.170050+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:06.170050+0000","last_undegraded":"2024-09-10T16:03:06.170050+0000","last_fullsized":"2024-09-10T16:03:06.170050+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":24535,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:11.734377+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:11.734377+0000","last_peered":"2024-09-10T16:03:11.734377+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:11.734377+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1767,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2067}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087979+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:06.087979+0000","last_peered":"2024-09-10T16:03:06.087979+0000","last_clean":"2024-09-10T16:03:06.087979+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:06.087979+0000","last_undegraded":"2024-09-10T16:03:06.087979+0000","last_fullsized":"2024-09-10T16:03:06.087979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23040,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191012864,"data_stored":190955056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626556,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953608,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481616,"kb_used_data":125120,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890224,"statfs":{"total":96636764160,"available":96143589376,"internally_reserved":0,"allocated":128122880,"data_stored":127814747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:18.072 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:18.422 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:18.422 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:18.434 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-09-10T16:03:16.651949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23042,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491980,"kb_used_data":187868,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623540,"statfs":{"total":289910292480,"available":288382504960,"internally_reserved":0,"allocated":192376832,"data_stored":191405464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.571624"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21200,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086891+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:06.086891+0000","last_peered":"2024-09-10T16:03:06.086891+0000","last_clean":"2024-09-10T16:03:06.086891+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:06.086891+0000","last_undegraded":"2024-09-10T16:03:06.086891+0000","last_fullsized":"2024-09-10T16:03:06.086891+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21119,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:03:06.088239+0000","last_peered":"2024-09-10T16:03:06.088239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:06.088239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086647+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:06.086647+0000","last_peered":"2024-09-10T16:03:06.086647+0000","last_clean":"2024-09-10T16:03:06.086647+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:06.086647+0000","last_undegraded":"2024-09-10T16:03:06.086647+0000","last_fullsized":"2024-09-10T16:03:06.086647+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20476,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086217+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086217+0000","last_peered":"2024-09-10T16:03:06.086217+0000","last_clean":"2024-09-10T16:03:06.086217+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086217+0000","last_undegraded":"2024-09-10T16:03:06.086217+0000","last_fullsized":"2024-09-10T16:03:06.086217+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21065,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086797+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.086797+0000","last_peered":"2024-09-10T16:03:06.086797+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.086797+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081198+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081198+0000","last_peered":"2024-09-10T16:03:06.081198+0000","last_clean":"2024-09-10T16:03:06.081198+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081198+0000","last_undegraded":"2024-09-10T16:03:06.081198+0000","last_fullsized":"2024-09-10T16:03:06.081198+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087905+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:06.087905+0000","last_peered":"2024-09-10T16:03:06.087905+0000","last_clean":"2024-09-10T16:03:06.087905+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:06.087905+0000","last_undegraded":"2024-09-10T16:03:06.087905+0000","last_fullsized":"2024-09-10T16:03:06.087905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28604,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087967+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:06.087967+0000","last_peered":"2024-09-10T16:03:06.087967+0000","last_clean":"2024-09-10T16:03:06.087967+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:06.087967+0000","last_undegraded":"2024-09-10T16:03:06.087967+0000","last_fullsized":"2024-09-10T16:03:06.087967+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28272,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.170050+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:06.170050+0000","last_peered":"2024-09-10T16:03:06.170050+0000","last_clean":"2024-09-10T16:03:06.170050+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:06.170050+0000","last_undegraded":"2024-09-10T16:03:06.170050+0000","last_fullsized":"2024-09-10T16:03:06.170050+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":24535,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:11.734377+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:11.734377+0000","last_peered":"2024-09-10T16:03:11.734377+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:11.734377+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1767,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2067}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087979+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:06.087979+0000","last_peered":"2024-09-10T16:03:06.087979+0000","last_clean":"2024-09-10T16:03:06.087979+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:06.087979+0000","last_undegraded":"2024-09-10T16:03:06.087979+0000","last_fullsized":"2024-09-10T16:03:06.087979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23040,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191012864,"data_stored":190955056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626556,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953609,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481616,"kb_used_data":125120,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890224,"statfs":{"total":96636764160,"available":96143589376,"internally_reserved":0,"allocated":128122880,"data_stored":127814747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:18.435 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:18.736 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:18.737 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:18.749 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-10T16:03:18.652274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22807,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23843,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500300,"kb_used_data":187868,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281615220,"statfs":{"total":289910292480,"available":288373985280,"internally_reserved":0,"allocated":192376832,"data_stored":191405464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1343867909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2213,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002309"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21200,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086891+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:06.086891+0000","last_peered":"2024-09-10T16:03:06.086891+0000","last_clean":"2024-09-10T16:03:06.086891+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:06.086891+0000","last_undegraded":"2024-09-10T16:03:06.086891+0000","last_fullsized":"2024-09-10T16:03:06.086891+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21119,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:03:06.088239+0000","last_peered":"2024-09-10T16:03:06.088239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:06.088239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086647+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:06.086647+0000","last_peered":"2024-09-10T16:03:06.086647+0000","last_clean":"2024-09-10T16:03:06.086647+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:06.086647+0000","last_undegraded":"2024-09-10T16:03:06.086647+0000","last_fullsized":"2024-09-10T16:03:06.086647+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20477,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086253+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086253+0000","last_peered":"2024-09-10T16:03:06.086253+0000","last_clean":"2024-09-10T16:03:06.086253+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086253+0000","last_undegraded":"2024-09-10T16:03:06.086253+0000","last_fullsized":"2024-09-10T16:03:06.086253+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21066,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088438+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.088438+0000","last_peered":"2024-09-10T16:03:06.088438+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.088438+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081198+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081198+0000","last_peered":"2024-09-10T16:03:06.081198+0000","last_clean":"2024-09-10T16:03:06.081198+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081198+0000","last_undegraded":"2024-09-10T16:03:06.081198+0000","last_fullsized":"2024-09-10T16:03:06.081198+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087905+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:06.087905+0000","last_peered":"2024-09-10T16:03:06.087905+0000","last_clean":"2024-09-10T16:03:06.087905+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:06.087905+0000","last_undegraded":"2024-09-10T16:03:06.087905+0000","last_fullsized":"2024-09-10T16:03:06.087905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28604,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087967+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:06.087967+0000","last_peered":"2024-09-10T16:03:06.087967+0000","last_clean":"2024-09-10T16:03:06.087967+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:06.087967+0000","last_undegraded":"2024-09-10T16:03:06.087967+0000","last_fullsized":"2024-09-10T16:03:06.087967+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28272,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.170050+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:06.170050+0000","last_peered":"2024-09-10T16:03:06.170050+0000","last_clean":"2024-09-10T16:03:06.170050+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:06.170050+0000","last_undegraded":"2024-09-10T16:03:06.170050+0000","last_fullsized":"2024-09-10T16:03:06.170050+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":26138,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:16.736330+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:16.736330+0000","last_peered":"2024-09-10T16:03:16.736330+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:16.736330+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1266,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087979+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:06.087979+0000","last_peered":"2024-09-10T16:03:06.087979+0000","last_clean":"2024-09-10T16:03:06.087979+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:06.087979+0000","last_undegraded":"2024-09-10T16:03:06.087979+0000","last_fullsized":"2024-09-10T16:03:06.087979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22807,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23841,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191012864,"data_stored":190955056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626557,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491652,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880188,"statfs":{"total":96636764160,"available":96133312512,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502918830},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149838,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953609,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481616,"kb_used_data":125120,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890224,"statfs":{"total":96636764160,"available":96143589376,"internally_reserved":0,"allocated":128122880,"data_stored":127814747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:18.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:03:18.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 21119, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.088239+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:03:06.088239+0000', 'last_peered': '2024-09-10T16:03:06.088239+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:03:06.088239+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:18.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:18.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21066, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.088438+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:06.088438+0000', 'last_peered': '2024-09-10T16:03:06.088438+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:06.088438+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:18.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:18.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21412, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.085461+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:06.085461+0000', 'last_peered': '2024-09-10T16:03:06.085461+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:06.085461+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:18.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:18.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20854", 'reported_seq': 26138, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:16.736330+0000', 'last_change': '2024-09-10T16:03:04.326890+0000', 'last_active': '2024-09-10T16:03:16.736330+0000', 'last_peered': '2024-09-10T16:03:16.736330+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:03:16.736330+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10800", 'ondisk_log_start': "46'10800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1266, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18046, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2568, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1266}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:18.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:18.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21483, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.086596+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:06.086596+0000', 'last_peered': '2024-09-10T16:03:06.086596+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:06.086596+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:18.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:18.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21607, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.085558+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:03:06.085558+0000', 'last_peered': '2024-09-10T16:03:06.085558+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:03:06.085558+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:18.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:18.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21874, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.086306+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:06.086306+0000', 'last_peered': '2024-09-10T16:03:06.086306+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:06.086306+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:18.752 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:19.162 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:19.162 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22807/6 objects degraded (380116.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":725,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1536307200,"bytes_avail":288373985280,"bytes_total":289910292480,"degraded_objects":22807,"degraded_total":6,"degraded_ratio":3801.1666666666661,"recovering_objects_per_sec":184,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2213,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-09-10T16:03:08.650481+0000","services":{"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:03:19.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:19.175 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:19.480 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:19.480 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:19.493 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-10T16:03:18.652274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22807,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23843,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500300,"kb_used_data":187868,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281615220,"statfs":{"total":289910292480,"available":288373985280,"internally_reserved":0,"allocated":192376832,"data_stored":191405464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1343867909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2213,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002309"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21200,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086891+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:06.086891+0000","last_peered":"2024-09-10T16:03:06.086891+0000","last_clean":"2024-09-10T16:03:06.086891+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:06.086891+0000","last_undegraded":"2024-09-10T16:03:06.086891+0000","last_fullsized":"2024-09-10T16:03:06.086891+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21119,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:03:06.088239+0000","last_peered":"2024-09-10T16:03:06.088239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:06.088239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086647+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:06.086647+0000","last_peered":"2024-09-10T16:03:06.086647+0000","last_clean":"2024-09-10T16:03:06.086647+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:06.086647+0000","last_undegraded":"2024-09-10T16:03:06.086647+0000","last_fullsized":"2024-09-10T16:03:06.086647+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20477,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086253+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086253+0000","last_peered":"2024-09-10T16:03:06.086253+0000","last_clean":"2024-09-10T16:03:06.086253+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086253+0000","last_undegraded":"2024-09-10T16:03:06.086253+0000","last_fullsized":"2024-09-10T16:03:06.086253+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21066,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088438+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.088438+0000","last_peered":"2024-09-10T16:03:06.088438+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.088438+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081198+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081198+0000","last_peered":"2024-09-10T16:03:06.081198+0000","last_clean":"2024-09-10T16:03:06.081198+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081198+0000","last_undegraded":"2024-09-10T16:03:06.081198+0000","last_fullsized":"2024-09-10T16:03:06.081198+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087905+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:06.087905+0000","last_peered":"2024-09-10T16:03:06.087905+0000","last_clean":"2024-09-10T16:03:06.087905+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:06.087905+0000","last_undegraded":"2024-09-10T16:03:06.087905+0000","last_fullsized":"2024-09-10T16:03:06.087905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28604,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087967+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:06.087967+0000","last_peered":"2024-09-10T16:03:06.087967+0000","last_clean":"2024-09-10T16:03:06.087967+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:06.087967+0000","last_undegraded":"2024-09-10T16:03:06.087967+0000","last_fullsized":"2024-09-10T16:03:06.087967+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28272,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.170050+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:06.170050+0000","last_peered":"2024-09-10T16:03:06.170050+0000","last_clean":"2024-09-10T16:03:06.170050+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:06.170050+0000","last_undegraded":"2024-09-10T16:03:06.170050+0000","last_fullsized":"2024-09-10T16:03:06.170050+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":26138,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:16.736330+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:16.736330+0000","last_peered":"2024-09-10T16:03:16.736330+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:16.736330+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1266,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087979+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:06.087979+0000","last_peered":"2024-09-10T16:03:06.087979+0000","last_clean":"2024-09-10T16:03:06.087979+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:06.087979+0000","last_undegraded":"2024-09-10T16:03:06.087979+0000","last_fullsized":"2024-09-10T16:03:06.087979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22807,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23841,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191012864,"data_stored":190955056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626557,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491652,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880188,"statfs":{"total":96636764160,"available":96133312512,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502918830},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149838,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953609,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481616,"kb_used_data":125120,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890224,"statfs":{"total":96636764160,"available":96143589376,"internally_reserved":0,"allocated":128122880,"data_stored":127814747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:22.496 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:22.826 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:22.826 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:22.840 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-10T16:03:22.653022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24981,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491980,"kb_used_data":187868,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623540,"statfs":{"total":289910292480,"available":288382504960,"internally_reserved":0,"allocated":192376832,"data_stored":191405464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3011,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002268"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21200,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086891+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:06.086891+0000","last_peered":"2024-09-10T16:03:06.086891+0000","last_clean":"2024-09-10T16:03:06.086891+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:06.086891+0000","last_undegraded":"2024-09-10T16:03:06.086891+0000","last_fullsized":"2024-09-10T16:03:06.086891+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21119,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:03:06.088239+0000","last_peered":"2024-09-10T16:03:06.088239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:06.088239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086647+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:06.086647+0000","last_peered":"2024-09-10T16:03:06.086647+0000","last_clean":"2024-09-10T16:03:06.086647+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:06.086647+0000","last_undegraded":"2024-09-10T16:03:06.086647+0000","last_fullsized":"2024-09-10T16:03:06.086647+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20477,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086253+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086253+0000","last_peered":"2024-09-10T16:03:06.086253+0000","last_clean":"2024-09-10T16:03:06.086253+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086253+0000","last_undegraded":"2024-09-10T16:03:06.086253+0000","last_fullsized":"2024-09-10T16:03:06.086253+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21066,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088438+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.088438+0000","last_peered":"2024-09-10T16:03:06.088438+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.088438+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081248+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081248+0000","last_peered":"2024-09-10T16:03:06.081248+0000","last_clean":"2024-09-10T16:03:06.081248+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081248+0000","last_undegraded":"2024-09-10T16:03:06.081248+0000","last_fullsized":"2024-09-10T16:03:06.081248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087905+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:06.087905+0000","last_peered":"2024-09-10T16:03:06.087905+0000","last_clean":"2024-09-10T16:03:06.087905+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:06.087905+0000","last_undegraded":"2024-09-10T16:03:06.087905+0000","last_fullsized":"2024-09-10T16:03:06.087905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28604,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087967+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:06.087967+0000","last_peered":"2024-09-10T16:03:06.087967+0000","last_clean":"2024-09-10T16:03:06.087967+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:06.087967+0000","last_undegraded":"2024-09-10T16:03:06.087967+0000","last_fullsized":"2024-09-10T16:03:06.087967+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28272,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.170050+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:06.170050+0000","last_peered":"2024-09-10T16:03:06.170050+0000","last_clean":"2024-09-10T16:03:06.170050+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:06.170050+0000","last_undegraded":"2024-09-10T16:03:06.170050+0000","last_fullsized":"2024-09-10T16:03:06.170050+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":28414,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:21.735928+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:21.735928+0000","last_peered":"2024-09-10T16:03:21.735928+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:21.735928+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3706,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087979+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:06.087979+0000","last_peered":"2024-09-10T16:03:06.087979+0000","last_clean":"2024-09-10T16:03:06.087979+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:06.087979+0000","last_undegraded":"2024-09-10T16:03:06.087979+0000","last_fullsized":"2024-09-10T16:03:06.087979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24979,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953610,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481616,"kb_used_data":125120,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890224,"statfs":{"total":96636764160,"available":96143589376,"internally_reserved":0,"allocated":128122880,"data_stored":127814747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:22.841 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:23.157 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:23.170 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-10T16:03:23.133997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24981,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491980,"kb_used_data":187868,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623540,"statfs":{"total":289910292480,"available":288382504960,"internally_reserved":0,"allocated":192376832,"data_stored":191405464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1939,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1939,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.482825"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21200,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086891+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:06.086891+0000","last_peered":"2024-09-10T16:03:06.086891+0000","last_clean":"2024-09-10T16:03:06.086891+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:06.086891+0000","last_undegraded":"2024-09-10T16:03:06.086891+0000","last_fullsized":"2024-09-10T16:03:06.086891+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21119,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:03:06.088239+0000","last_peered":"2024-09-10T16:03:06.088239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:06.088239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086647+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:06.086647+0000","last_peered":"2024-09-10T16:03:06.086647+0000","last_clean":"2024-09-10T16:03:06.086647+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:06.086647+0000","last_undegraded":"2024-09-10T16:03:06.086647+0000","last_fullsized":"2024-09-10T16:03:06.086647+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20477,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086253+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086253+0000","last_peered":"2024-09-10T16:03:06.086253+0000","last_clean":"2024-09-10T16:03:06.086253+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086253+0000","last_undegraded":"2024-09-10T16:03:06.086253+0000","last_fullsized":"2024-09-10T16:03:06.086253+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21066,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088438+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.088438+0000","last_peered":"2024-09-10T16:03:06.088438+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.088438+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081248+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081248+0000","last_peered":"2024-09-10T16:03:06.081248+0000","last_clean":"2024-09-10T16:03:06.081248+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081248+0000","last_undegraded":"2024-09-10T16:03:06.081248+0000","last_fullsized":"2024-09-10T16:03:06.081248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087905+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:06.087905+0000","last_peered":"2024-09-10T16:03:06.087905+0000","last_clean":"2024-09-10T16:03:06.087905+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:06.087905+0000","last_undegraded":"2024-09-10T16:03:06.087905+0000","last_fullsized":"2024-09-10T16:03:06.087905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28604,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087967+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:06.087967+0000","last_peered":"2024-09-10T16:03:06.087967+0000","last_clean":"2024-09-10T16:03:06.087967+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:06.087967+0000","last_undegraded":"2024-09-10T16:03:06.087967+0000","last_fullsized":"2024-09-10T16:03:06.087967+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28272,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.170050+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:06.170050+0000","last_peered":"2024-09-10T16:03:06.170050+0000","last_clean":"2024-09-10T16:03:06.170050+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:06.170050+0000","last_undegraded":"2024-09-10T16:03:06.170050+0000","last_fullsized":"2024-09-10T16:03:06.170050+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":28414,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:21.735928+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:21.735928+0000","last_peered":"2024-09-10T16:03:21.735928+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:21.735928+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3706,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087979+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:06.087979+0000","last_peered":"2024-09-10T16:03:06.087979+0000","last_clean":"2024-09-10T16:03:06.087979+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:06.087979+0000","last_undegraded":"2024-09-10T16:03:06.087979+0000","last_fullsized":"2024-09-10T16:03:06.087979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24979,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953610,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481616,"kb_used_data":125120,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890224,"statfs":{"total":96636764160,"available":96143589376,"internally_reserved":0,"allocated":128122880,"data_stored":127814747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:23.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:03:23.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 21119, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.088239+0000', 'last_change': '2024-09-10T15:53:19.970364+0000', 'last_active': '2024-09-10T16:03:06.088239+0000', 'last_peered': '2024-09-10T16:03:06.088239+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:03:06.088239+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:23.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:23.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21066, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.088438+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:06.088438+0000', 'last_peered': '2024-09-10T16:03:06.088438+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:06.088438+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:23.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:23.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21412, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.085461+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:06.085461+0000', 'last_peered': '2024-09-10T16:03:06.085461+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:06.085461+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:23.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:23.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20854", 'reported_seq': 28414, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:21.735928+0000', 'last_change': '2024-09-10T16:03:04.326890+0000', 'last_active': '2024-09-10T16:03:21.735928+0000', 'last_peered': '2024-09-10T16:03:21.735928+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:03:21.735928+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10800", 'ondisk_log_start': "46'10800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 128, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18046, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3706, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -128}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:23.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:23.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21483, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.086596+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:06.086596+0000', 'last_peered': '2024-09-10T16:03:06.086596+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:06.086596+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:23.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:23.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21607, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.085558+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:03:06.085558+0000', 'last_peered': '2024-09-10T16:03:06.085558+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:03:06.085558+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:23.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:23.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21874, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.086306+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:06.086306+0000', 'last_peered': '2024-09-10T16:03:06.086306+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:06.086306+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:23.173 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:23.550 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21669/6 objects degraded (361150.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":729,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1527787520,"bytes_avail":288382504960,"bytes_total":289910292480,"degraded_objects":21669,"degraded_total":6,"degraded_ratio":3611.5,"recovering_objects_per_sec":250,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":3011,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-09-10T16:03:08.650481+0000","services":{"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:03:23.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:23.566 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:23.878 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:23.878 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:23.892 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-10T16:03:23.133997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24981,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491980,"kb_used_data":187868,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623540,"statfs":{"total":289910292480,"available":288382504960,"internally_reserved":0,"allocated":192376832,"data_stored":191405464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1939,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1939,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.482825"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21200,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086891+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:06.086891+0000","last_peered":"2024-09-10T16:03:06.086891+0000","last_clean":"2024-09-10T16:03:06.086891+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:06.086891+0000","last_undegraded":"2024-09-10T16:03:06.086891+0000","last_fullsized":"2024-09-10T16:03:06.086891+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21119,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088239+0000","last_change":"2024-09-10T15:53:19.970364+0000","last_active":"2024-09-10T16:03:06.088239+0000","last_peered":"2024-09-10T16:03:06.088239+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:06.088239+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20780,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086647+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:06.086647+0000","last_peered":"2024-09-10T16:03:06.086647+0000","last_clean":"2024-09-10T16:03:06.086647+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:06.086647+0000","last_undegraded":"2024-09-10T16:03:06.086647+0000","last_fullsized":"2024-09-10T16:03:06.086647+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20477,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086253+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086253+0000","last_peered":"2024-09-10T16:03:06.086253+0000","last_clean":"2024-09-10T16:03:06.086253+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086253+0000","last_undegraded":"2024-09-10T16:03:06.086253+0000","last_fullsized":"2024-09-10T16:03:06.086253+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21066,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088438+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.088438+0000","last_peered":"2024-09-10T16:03:06.088438+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.088438+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081248+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081248+0000","last_peered":"2024-09-10T16:03:06.081248+0000","last_clean":"2024-09-10T16:03:06.081248+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081248+0000","last_undegraded":"2024-09-10T16:03:06.081248+0000","last_fullsized":"2024-09-10T16:03:06.081248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087905+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:06.087905+0000","last_peered":"2024-09-10T16:03:06.087905+0000","last_clean":"2024-09-10T16:03:06.087905+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:06.087905+0000","last_undegraded":"2024-09-10T16:03:06.087905+0000","last_fullsized":"2024-09-10T16:03:06.087905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28604,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087967+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:06.087967+0000","last_peered":"2024-09-10T16:03:06.087967+0000","last_clean":"2024-09-10T16:03:06.087967+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:06.087967+0000","last_undegraded":"2024-09-10T16:03:06.087967+0000","last_fullsized":"2024-09-10T16:03:06.087967+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28272,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.170050+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:06.170050+0000","last_peered":"2024-09-10T16:03:06.170050+0000","last_clean":"2024-09-10T16:03:06.170050+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:06.170050+0000","last_undegraded":"2024-09-10T16:03:06.170050+0000","last_fullsized":"2024-09-10T16:03:06.170050+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":28414,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:21.735928+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:21.735928+0000","last_peered":"2024-09-10T16:03:21.735928+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:21.735928+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3706,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087979+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:06.087979+0000","last_peered":"2024-09-10T16:03:06.087979+0000","last_clean":"2024-09-10T16:03:06.087979+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:06.087979+0000","last_undegraded":"2024-09-10T16:03:06.087979+0000","last_fullsized":"2024-09-10T16:03:06.087979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21669,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24979,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953610,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481616,"kb_used_data":125120,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890224,"statfs":{"total":96636764160,"available":96143589376,"internally_reserved":0,"allocated":128122880,"data_stored":127814747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:26.894 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:27.193 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:27.193 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:27.205 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-09-10T16:03:26.653803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25485,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491996,"kb_used_data":187884,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623524,"statfs":{"total":289910292480,"available":288382488576,"internally_reserved":0,"allocated":192393216,"data_stored":191411203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1642,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001528"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":22132,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:26.202126+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:26.202126+0000","last_peered":"2024-09-10T16:03:26.202126+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:26.202126+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":677,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20784,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146854+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146854+0000","last_peered":"2024-09-10T16:03:24.146854+0000","last_clean":"2024-09-10T16:03:24.146854+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146854+0000","last_undegraded":"2024-09-10T16:03:24.146854+0000","last_fullsized":"2024-09-10T16:03:24.146854+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20477,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086253+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086253+0000","last_peered":"2024-09-10T16:03:06.086253+0000","last_clean":"2024-09-10T16:03:06.086253+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086253+0000","last_undegraded":"2024-09-10T16:03:06.086253+0000","last_fullsized":"2024-09-10T16:03:06.086253+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21066,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088438+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.088438+0000","last_peered":"2024-09-10T16:03:06.088438+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.088438+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081248+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081248+0000","last_peered":"2024-09-10T16:03:06.081248+0000","last_clean":"2024-09-10T16:03:06.081248+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081248+0000","last_undegraded":"2024-09-10T16:03:06.081248+0000","last_fullsized":"2024-09-10T16:03:06.081248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":28414,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:21.735928+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:21.735928+0000","last_peered":"2024-09-10T16:03:21.735928+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:21.735928+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3706,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25483,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953611,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:27.207 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:27.509 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:27.510 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:27.521 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-09-10T16:03:26.653803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25485,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491996,"kb_used_data":187884,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623524,"statfs":{"total":289910292480,"available":288382488576,"internally_reserved":0,"allocated":192393216,"data_stored":191411203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1642,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001528"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":22132,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:26.202126+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:26.202126+0000","last_peered":"2024-09-10T16:03:26.202126+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:26.202126+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":677,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20784,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146854+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146854+0000","last_peered":"2024-09-10T16:03:24.146854+0000","last_clean":"2024-09-10T16:03:24.146854+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146854+0000","last_undegraded":"2024-09-10T16:03:24.146854+0000","last_fullsized":"2024-09-10T16:03:24.146854+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20477,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086253+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086253+0000","last_peered":"2024-09-10T16:03:06.086253+0000","last_clean":"2024-09-10T16:03:06.086253+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086253+0000","last_undegraded":"2024-09-10T16:03:06.086253+0000","last_fullsized":"2024-09-10T16:03:06.086253+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21066,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088438+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.088438+0000","last_peered":"2024-09-10T16:03:06.088438+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.088438+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081248+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081248+0000","last_peered":"2024-09-10T16:03:06.081248+0000","last_clean":"2024-09-10T16:03:06.081248+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081248+0000","last_undegraded":"2024-09-10T16:03:06.081248+0000","last_fullsized":"2024-09-10T16:03:06.081248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":28414,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:21.735928+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:21.735928+0000","last_peered":"2024-09-10T16:03:21.735928+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:21.735928+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3706,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25483,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953611,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:27.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:03:27.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 22132, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:26.202126+0000', 'last_change': '2024-09-10T16:03:22.157387+0000', 'last_active': '2024-09-10T16:03:26.202126+0000', 'last_peered': '2024-09-10T16:03:26.202126+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:03:26.202126+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 677, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3126}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:27.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:27.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21066, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.088438+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:06.088438+0000', 'last_peered': '2024-09-10T16:03:06.088438+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:06.088438+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:27.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21412, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.085461+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:06.085461+0000', 'last_peered': '2024-09-10T16:03:06.085461+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:06.085461+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-10T16:03:27.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'20854", 'reported_seq': 28414, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:21.735928+0000', 'last_change': '2024-09-10T16:03:04.326890+0000', 'last_active': '2024-09-10T16:03:21.735928+0000', 'last_peered': '2024-09-10T16:03:21.735928+0000', 'last_clean': '2024-09-10T15:53:17.295937+0000', 'last_became_active': '2024-09-10T15:53:19.932049+0000', 'last_became_peered': '2024-09-10T15:53:19.932049+0000', 'last_unstale': '2024-09-10T16:03:21.735928+0000', 'last_undegraded': '2024-09-10T15:53:19.575645+0000', 'last_fullsized': '2024-09-10T15:53:19.550077+0000', 'mapping_epoch': 43, 'log_start': "46'10800", 'ondisk_log_start': "46'10800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2845", 'last_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:02.560275+0000', 'objects_scrubbed': 2806, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 964, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 128, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18046, 'num_write_kb': 30480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3706, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -128}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:27.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21483, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.086596+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:06.086596+0000', 'last_peered': '2024-09-10T16:03:06.086596+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:06.086596+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:27.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21607, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.085558+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:03:06.085558+0000', 'last_peered': '2024-09-10T16:03:06.085558+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:03:06.085558+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:27.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21874, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:06.086306+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:06.086306+0000', 'last_peered': '2024-09-10T16:03:06.086306+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:06.086306+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:27.524 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:27.964 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:27.964 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21165/6 objects degraded (352750.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":733,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1527803904,"bytes_avail":288382488576,"bytes_total":289910292480,"degraded_objects":21165,"degraded_total":6,"degraded_ratio":3527.5,"recovering_objects_per_sec":205,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1642,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-09-10T16:03:08.650481+0000","services":{"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T16:03:27.977 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:27.977 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:28.283 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:28.284 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:28.298 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-09-10T16:03:26.653803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25485,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491996,"kb_used_data":187884,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623524,"statfs":{"total":289910292480,"available":288382488576,"internally_reserved":0,"allocated":192393216,"data_stored":191411203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1642,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001528"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28819,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.087623+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:06.087623+0000","last_peered":"2024-09-10T16:03:06.087623+0000","last_clean":"2024-09-10T16:03:06.087623+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:06.087623+0000","last_undegraded":"2024-09-10T16:03:06.087623+0000","last_fullsized":"2024-09-10T16:03:06.087623+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":22132,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:26.202126+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:26.202126+0000","last_peered":"2024-09-10T16:03:26.202126+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:26.202126+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":677,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20784,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146854+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146854+0000","last_peered":"2024-09-10T16:03:24.146854+0000","last_clean":"2024-09-10T16:03:24.146854+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146854+0000","last_undegraded":"2024-09-10T16:03:24.146854+0000","last_fullsized":"2024-09-10T16:03:24.146854+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20477,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086253+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:06.086253+0000","last_peered":"2024-09-10T16:03:06.086253+0000","last_clean":"2024-09-10T16:03:06.086253+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:06.086253+0000","last_undegraded":"2024-09-10T16:03:06.086253+0000","last_fullsized":"2024-09-10T16:03:06.086253+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28774,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085884+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:06.085884+0000","last_peered":"2024-09-10T16:03:06.085884+0000","last_clean":"2024-09-10T16:03:06.085884+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:06.085884+0000","last_undegraded":"2024-09-10T16:03:06.085884+0000","last_fullsized":"2024-09-10T16:03:06.085884+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21066,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.088438+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:06.088438+0000","last_peered":"2024-09-10T16:03:06.088438+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:06.088438+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21064,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.086126+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:06.086126+0000","last_peered":"2024-09-10T16:03:06.086126+0000","last_clean":"2024-09-10T16:03:06.086126+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:06.086126+0000","last_undegraded":"2024-09-10T16:03:06.086126+0000","last_fullsized":"2024-09-10T16:03:06.086126+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081248+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:06.081248+0000","last_peered":"2024-09-10T16:03:06.081248+0000","last_clean":"2024-09-10T16:03:06.081248+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:06.081248+0000","last_undegraded":"2024-09-10T16:03:06.081248+0000","last_fullsized":"2024-09-10T16:03:06.081248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28865,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085907+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:06.085907+0000","last_peered":"2024-09-10T16:03:06.085907+0000","last_clean":"2024-09-10T16:03:06.085907+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:06.085907+0000","last_undegraded":"2024-09-10T16:03:06.085907+0000","last_fullsized":"2024-09-10T16:03:06.085907+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21412,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085461+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:06.085461+0000","last_peered":"2024-09-10T16:03:06.085461+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:06.085461+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.081479+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:06.081479+0000","last_peered":"2024-09-10T16:03:06.081479+0000","last_clean":"2024-09-10T16:03:06.081479+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:06.081479+0000","last_undegraded":"2024-09-10T16:03:06.081479+0000","last_fullsized":"2024-09-10T16:03:06.081479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":97,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085440+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:06.085440+0000","last_peered":"2024-09-10T16:03:06.085440+0000","last_clean":"2024-09-10T16:03:06.085440+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:06.085440+0000","last_undegraded":"2024-09-10T16:03:06.085440+0000","last_fullsized":"2024-09-10T16:03:06.085440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":28414,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:21.735928+0000","last_change":"2024-09-10T16:03:04.326890+0000","last_active":"2024-09-10T16:03:21.735928+0000","last_peered":"2024-09-10T16:03:21.735928+0000","last_clean":"2024-09-10T15:53:17.295937+0000","last_became_active":"2024-09-10T15:53:19.932049+0000","last_became_peered":"2024-09-10T15:53:19.932049+0000","last_unstale":"2024-09-10T16:03:21.735928+0000","last_undegraded":"2024-09-10T15:53:19.575645+0000","last_fullsized":"2024-09-10T15:53:19.550077+0000","mapping_epoch":43,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3706,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085591+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:06.085591+0000","last_peered":"2024-09-10T16:03:06.085591+0000","last_clean":"2024-09-10T16:03:06.085591+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:06.085591+0000","last_undegraded":"2024-09-10T16:03:06.085591+0000","last_fullsized":"2024-09-10T16:03:06.085591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21483,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086596+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:06.086596+0000","last_peered":"2024-09-10T16:03:06.086596+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:06.086596+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21607,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.085558+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:06.085558+0000","last_peered":"2024-09-10T16:03:06.085558+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:06.085558+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085793+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:06.085793+0000","last_peered":"2024-09-10T16:03:06.085793+0000","last_clean":"2024-09-10T16:03:06.085793+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:06.085793+0000","last_undegraded":"2024-09-10T16:03:06.085793+0000","last_fullsized":"2024-09-10T16:03:06.085793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-10T16:03:06.085866+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:06.085866+0000","last_peered":"2024-09-10T16:03:06.085866+0000","last_clean":"2024-09-10T16:03:06.085866+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:06.085866+0000","last_undegraded":"2024-09-10T16:03:06.085866+0000","last_fullsized":"2024-09-10T16:03:06.085866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21874,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:06.086306+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:06.086306+0000","last_peered":"2024-09-10T16:03:06.086306+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:06.086306+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25483,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":512000,"data_stored":156898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527032,"kb_used_data":62248,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844808,"statfs":{"total":96636764160,"available":96097083392,"internally_reserved":0,"allocated":63741952,"data_stored":63433819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953611,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:31.300 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:31.608 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:31.608 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:31.620 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-10T16:03:30.654472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25613,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500344,"kb_used_data":187912,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281615176,"statfs":{"total":289910292480,"available":288373940224,"internally_reserved":0,"allocated":192421888,"data_stored":191422198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1343867909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001449"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28824,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.138148+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:24.138148+0000","last_peered":"2024-09-10T16:03:24.138148+0000","last_clean":"2024-09-10T16:03:24.138148+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:24.138148+0000","last_undegraded":"2024-09-10T16:03:24.138148+0000","last_fullsized":"2024-09-10T16:03:24.138148+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":22132,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:26.202126+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:26.202126+0000","last_peered":"2024-09-10T16:03:26.202126+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:26.202126+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":677,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20784,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146854+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146854+0000","last_peered":"2024-09-10T16:03:24.146854+0000","last_clean":"2024-09-10T16:03:24.146854+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146854+0000","last_undegraded":"2024-09-10T16:03:24.146854+0000","last_fullsized":"2024-09-10T16:03:24.146854+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135509+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:24.135509+0000","last_peered":"2024-09-10T16:03:24.135509+0000","last_clean":"2024-09-10T16:03:24.135509+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:24.135509+0000","last_undegraded":"2024-09-10T16:03:24.135509+0000","last_fullsized":"2024-09-10T16:03:24.135509+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28778,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135429+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:24.135429+0000","last_peered":"2024-09-10T16:03:24.135429+0000","last_clean":"2024-09-10T16:03:24.135429+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:24.135429+0000","last_undegraded":"2024-09-10T16:03:24.135429+0000","last_fullsized":"2024-09-10T16:03:24.135429+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:24.135450+0000","last_peered":"2024-09-10T16:03:24.135450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:24.135450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21068,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135417+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:24.135417+0000","last_peered":"2024-09-10T16:03:24.135417+0000","last_clean":"2024-09-10T16:03:24.135417+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:24.135417+0000","last_undegraded":"2024-09-10T16:03:24.135417+0000","last_fullsized":"2024-09-10T16:03:24.135417+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135158+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:24.135158+0000","last_peered":"2024-09-10T16:03:24.135158+0000","last_clean":"2024-09-10T16:03:24.135158+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:24.135158+0000","last_undegraded":"2024-09-10T16:03:24.135158+0000","last_fullsized":"2024-09-10T16:03:24.135158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28869,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135532+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:24.135532+0000","last_peered":"2024-09-10T16:03:24.135532+0000","last_clean":"2024-09-10T16:03:24.135532+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:24.135532+0000","last_undegraded":"2024-09-10T16:03:24.135532+0000","last_fullsized":"2024-09-10T16:03:24.135532+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21416,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135875+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:24.135875+0000","last_peered":"2024-09-10T16:03:24.135875+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:24.135875+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135720+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:24.135720+0000","last_peered":"2024-09-10T16:03:24.135720+0000","last_clean":"2024-09-10T16:03:24.135720+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:24.135720+0000","last_undegraded":"2024-09-10T16:03:24.135720+0000","last_fullsized":"2024-09-10T16:03:24.135720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":101,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135526+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:24.135526+0000","last_peered":"2024-09-10T16:03:24.135526+0000","last_clean":"2024-09-10T16:03:24.135526+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:24.135526+0000","last_undegraded":"2024-09-10T16:03:24.135526+0000","last_fullsized":"2024-09-10T16:03:24.135526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21714,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.239868+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:24.239868+0000","last_peered":"2024-09-10T16:03:24.239868+0000","last_clean":"2024-09-10T16:03:24.239868+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:24.239868+0000","last_undegraded":"2024-09-10T16:03:24.239868+0000","last_fullsized":"2024-09-10T16:03:24.239868+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135778+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:24.135778+0000","last_peered":"2024-09-10T16:03:24.135778+0000","last_clean":"2024-09-10T16:03:24.135778+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:24.135778+0000","last_undegraded":"2024-09-10T16:03:24.135778+0000","last_fullsized":"2024-09-10T16:03:24.135778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21487,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.137945+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:24.137945+0000","last_peered":"2024-09-10T16:03:24.137945+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:24.137945+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.136096+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:24.136096+0000","last_peered":"2024-09-10T16:03:24.136096+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:24.136096+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135300+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:24.135300+0000","last_peered":"2024-09-10T16:03:24.135300+0000","last_clean":"2024-09-10T16:03:24.135300+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:24.135300+0000","last_undegraded":"2024-09-10T16:03:24.135300+0000","last_fullsized":"2024-09-10T16:03:24.135300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135919+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:24.135919+0000","last_peered":"2024-09-10T16:03:24.135919+0000","last_clean":"2024-09-10T16:03:24.135919+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:24.135919+0000","last_undegraded":"2024-09-10T16:03:24.135919+0000","last_fullsized":"2024-09-10T16:03:24.135919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21878,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135573+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:24.135573+0000","last_peered":"2024-09-10T16:03:24.135573+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:24.135573+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25611,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626559,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491664,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880176,"statfs":{"total":96636764160,"available":96133300224,"internally_reserved":0,"allocated":524288,"data_stored":162154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502918830},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149840,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527048,"kb_used_data":62264,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844792,"statfs":{"total":96636764160,"available":96097067008,"internally_reserved":0,"allocated":63758336,"data_stored":63439558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953611,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:31.621 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:31.939 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:31.951 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-10T16:03:30.654472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25613,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500344,"kb_used_data":187912,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281615176,"statfs":{"total":289910292480,"available":288373940224,"internally_reserved":0,"allocated":192421888,"data_stored":191422198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1343867909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001449"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28824,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.138148+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:24.138148+0000","last_peered":"2024-09-10T16:03:24.138148+0000","last_clean":"2024-09-10T16:03:24.138148+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:24.138148+0000","last_undegraded":"2024-09-10T16:03:24.138148+0000","last_fullsized":"2024-09-10T16:03:24.138148+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":22132,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:26.202126+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:26.202126+0000","last_peered":"2024-09-10T16:03:26.202126+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:26.202126+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":677,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20784,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146854+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146854+0000","last_peered":"2024-09-10T16:03:24.146854+0000","last_clean":"2024-09-10T16:03:24.146854+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146854+0000","last_undegraded":"2024-09-10T16:03:24.146854+0000","last_fullsized":"2024-09-10T16:03:24.146854+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135509+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:24.135509+0000","last_peered":"2024-09-10T16:03:24.135509+0000","last_clean":"2024-09-10T16:03:24.135509+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:24.135509+0000","last_undegraded":"2024-09-10T16:03:24.135509+0000","last_fullsized":"2024-09-10T16:03:24.135509+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28778,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135429+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:24.135429+0000","last_peered":"2024-09-10T16:03:24.135429+0000","last_clean":"2024-09-10T16:03:24.135429+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:24.135429+0000","last_undegraded":"2024-09-10T16:03:24.135429+0000","last_fullsized":"2024-09-10T16:03:24.135429+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:24.135450+0000","last_peered":"2024-09-10T16:03:24.135450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:24.135450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21068,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135417+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:24.135417+0000","last_peered":"2024-09-10T16:03:24.135417+0000","last_clean":"2024-09-10T16:03:24.135417+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:24.135417+0000","last_undegraded":"2024-09-10T16:03:24.135417+0000","last_fullsized":"2024-09-10T16:03:24.135417+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135158+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:24.135158+0000","last_peered":"2024-09-10T16:03:24.135158+0000","last_clean":"2024-09-10T16:03:24.135158+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:24.135158+0000","last_undegraded":"2024-09-10T16:03:24.135158+0000","last_fullsized":"2024-09-10T16:03:24.135158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28869,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135532+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:24.135532+0000","last_peered":"2024-09-10T16:03:24.135532+0000","last_clean":"2024-09-10T16:03:24.135532+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:24.135532+0000","last_undegraded":"2024-09-10T16:03:24.135532+0000","last_fullsized":"2024-09-10T16:03:24.135532+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21416,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135875+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:24.135875+0000","last_peered":"2024-09-10T16:03:24.135875+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:24.135875+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135720+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:24.135720+0000","last_peered":"2024-09-10T16:03:24.135720+0000","last_clean":"2024-09-10T16:03:24.135720+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:24.135720+0000","last_undegraded":"2024-09-10T16:03:24.135720+0000","last_fullsized":"2024-09-10T16:03:24.135720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":101,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135526+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:24.135526+0000","last_peered":"2024-09-10T16:03:24.135526+0000","last_clean":"2024-09-10T16:03:24.135526+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:24.135526+0000","last_undegraded":"2024-09-10T16:03:24.135526+0000","last_fullsized":"2024-09-10T16:03:24.135526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21714,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.239868+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:24.239868+0000","last_peered":"2024-09-10T16:03:24.239868+0000","last_clean":"2024-09-10T16:03:24.239868+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:24.239868+0000","last_undegraded":"2024-09-10T16:03:24.239868+0000","last_fullsized":"2024-09-10T16:03:24.239868+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135778+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:24.135778+0000","last_peered":"2024-09-10T16:03:24.135778+0000","last_clean":"2024-09-10T16:03:24.135778+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:24.135778+0000","last_undegraded":"2024-09-10T16:03:24.135778+0000","last_fullsized":"2024-09-10T16:03:24.135778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21487,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.137945+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:24.137945+0000","last_peered":"2024-09-10T16:03:24.137945+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:24.137945+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.136096+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:24.136096+0000","last_peered":"2024-09-10T16:03:24.136096+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:24.136096+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135300+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:24.135300+0000","last_peered":"2024-09-10T16:03:24.135300+0000","last_clean":"2024-09-10T16:03:24.135300+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:24.135300+0000","last_undegraded":"2024-09-10T16:03:24.135300+0000","last_fullsized":"2024-09-10T16:03:24.135300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135919+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:24.135919+0000","last_peered":"2024-09-10T16:03:24.135919+0000","last_clean":"2024-09-10T16:03:24.135919+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:24.135919+0000","last_undegraded":"2024-09-10T16:03:24.135919+0000","last_fullsized":"2024-09-10T16:03:24.135919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21878,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135573+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:24.135573+0000","last_peered":"2024-09-10T16:03:24.135573+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:24.135573+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25611,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626559,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491664,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880176,"statfs":{"total":96636764160,"available":96133300224,"internally_reserved":0,"allocated":524288,"data_stored":162154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502918830},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149840,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527048,"kb_used_data":62264,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844792,"statfs":{"total":96636764160,"available":96097067008,"internally_reserved":0,"allocated":63758336,"data_stored":63439558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953611,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:31.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:03:31.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 22132, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:26.202126+0000', 'last_change': '2024-09-10T16:03:22.157387+0000', 'last_active': '2024-09-10T16:03:26.202126+0000', 'last_peered': '2024-09-10T16:03:26.202126+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:03:26.202126+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 677, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3126}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:31.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:31.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21070, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.135450+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:24.135450+0000', 'last_peered': '2024-09-10T16:03:24.135450+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:24.135450+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:31.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:31.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21416, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.135875+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:24.135875+0000', 'last_peered': '2024-09-10T16:03:24.135875+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:24.135875+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:31.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:31.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21487, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.137945+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:24.137945+0000', 'last_peered': '2024-09-10T16:03:24.137945+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:24.137945+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:31.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:31.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21611, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.136096+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:03:24.136096+0000', 'last_peered': '2024-09-10T16:03:24.136096+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:03:24.136096+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:31.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:31.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21878, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.135573+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:24.135573+0000', 'last_peered': '2024-09-10T16:03:24.135573+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:24.135573+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:31.954 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:32.335 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:32.336 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21037/6 objects degraded (350616.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":738,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1536352256,"bytes_avail":288373940224,"bytes_total":289910292480,"degraded_objects":21037,"degraded_total":6,"degraded_ratio":3506.1666666666661,"recovering_objects_per_sec":78,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":632,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:32.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:32.350 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:32.648 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:32.648 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:32.662 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-10T16:03:30.654472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25613,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500344,"kb_used_data":187912,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281615176,"statfs":{"total":289910292480,"available":288373940224,"internally_reserved":0,"allocated":192421888,"data_stored":191422198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1343867909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001449"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28824,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.138148+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:24.138148+0000","last_peered":"2024-09-10T16:03:24.138148+0000","last_clean":"2024-09-10T16:03:24.138148+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:24.138148+0000","last_undegraded":"2024-09-10T16:03:24.138148+0000","last_fullsized":"2024-09-10T16:03:24.138148+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":22132,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:26.202126+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:26.202126+0000","last_peered":"2024-09-10T16:03:26.202126+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:26.202126+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":677,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20784,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146854+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146854+0000","last_peered":"2024-09-10T16:03:24.146854+0000","last_clean":"2024-09-10T16:03:24.146854+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146854+0000","last_undegraded":"2024-09-10T16:03:24.146854+0000","last_fullsized":"2024-09-10T16:03:24.146854+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135509+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:24.135509+0000","last_peered":"2024-09-10T16:03:24.135509+0000","last_clean":"2024-09-10T16:03:24.135509+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:24.135509+0000","last_undegraded":"2024-09-10T16:03:24.135509+0000","last_fullsized":"2024-09-10T16:03:24.135509+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28778,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135429+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:24.135429+0000","last_peered":"2024-09-10T16:03:24.135429+0000","last_clean":"2024-09-10T16:03:24.135429+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:24.135429+0000","last_undegraded":"2024-09-10T16:03:24.135429+0000","last_fullsized":"2024-09-10T16:03:24.135429+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:24.135450+0000","last_peered":"2024-09-10T16:03:24.135450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:24.135450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21068,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135417+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:24.135417+0000","last_peered":"2024-09-10T16:03:24.135417+0000","last_clean":"2024-09-10T16:03:24.135417+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:24.135417+0000","last_undegraded":"2024-09-10T16:03:24.135417+0000","last_fullsized":"2024-09-10T16:03:24.135417+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135158+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:24.135158+0000","last_peered":"2024-09-10T16:03:24.135158+0000","last_clean":"2024-09-10T16:03:24.135158+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:24.135158+0000","last_undegraded":"2024-09-10T16:03:24.135158+0000","last_fullsized":"2024-09-10T16:03:24.135158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28869,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135532+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:24.135532+0000","last_peered":"2024-09-10T16:03:24.135532+0000","last_clean":"2024-09-10T16:03:24.135532+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:24.135532+0000","last_undegraded":"2024-09-10T16:03:24.135532+0000","last_fullsized":"2024-09-10T16:03:24.135532+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21416,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135875+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:24.135875+0000","last_peered":"2024-09-10T16:03:24.135875+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:24.135875+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135720+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:24.135720+0000","last_peered":"2024-09-10T16:03:24.135720+0000","last_clean":"2024-09-10T16:03:24.135720+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:24.135720+0000","last_undegraded":"2024-09-10T16:03:24.135720+0000","last_fullsized":"2024-09-10T16:03:24.135720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":101,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135526+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:24.135526+0000","last_peered":"2024-09-10T16:03:24.135526+0000","last_clean":"2024-09-10T16:03:24.135526+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:24.135526+0000","last_undegraded":"2024-09-10T16:03:24.135526+0000","last_fullsized":"2024-09-10T16:03:24.135526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21714,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.239868+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:24.239868+0000","last_peered":"2024-09-10T16:03:24.239868+0000","last_clean":"2024-09-10T16:03:24.239868+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:24.239868+0000","last_undegraded":"2024-09-10T16:03:24.239868+0000","last_fullsized":"2024-09-10T16:03:24.239868+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135778+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:24.135778+0000","last_peered":"2024-09-10T16:03:24.135778+0000","last_clean":"2024-09-10T16:03:24.135778+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:24.135778+0000","last_undegraded":"2024-09-10T16:03:24.135778+0000","last_fullsized":"2024-09-10T16:03:24.135778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21487,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.137945+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:24.137945+0000","last_peered":"2024-09-10T16:03:24.137945+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:24.137945+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.136096+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:24.136096+0000","last_peered":"2024-09-10T16:03:24.136096+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:24.136096+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135300+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:24.135300+0000","last_peered":"2024-09-10T16:03:24.135300+0000","last_clean":"2024-09-10T16:03:24.135300+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:24.135300+0000","last_undegraded":"2024-09-10T16:03:24.135300+0000","last_fullsized":"2024-09-10T16:03:24.135300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135919+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:24.135919+0000","last_peered":"2024-09-10T16:03:24.135919+0000","last_clean":"2024-09-10T16:03:24.135919+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:24.135919+0000","last_undegraded":"2024-09-10T16:03:24.135919+0000","last_fullsized":"2024-09-10T16:03:24.135919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21878,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135573+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:24.135573+0000","last_peered":"2024-09-10T16:03:24.135573+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:24.135573+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25611,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626559,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491664,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880176,"statfs":{"total":96636764160,"available":96133300224,"internally_reserved":0,"allocated":524288,"data_stored":162154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502918830},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149840,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527048,"kb_used_data":62264,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844792,"statfs":{"total":96636764160,"available":96097067008,"internally_reserved":0,"allocated":63758336,"data_stored":63439558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953611,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:35.663 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:35.959 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:35.960 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:35.972 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-10T16:03:34.655138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26249,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487864,"kb_used_data":187912,"kb_used_omap":12,"kb_used_meta":1299891,"kb_avail":281627656,"statfs":{"total":289910292480,"available":288386719744,"internally_reserved":0,"allocated":192421888,"data_stored":191422198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1331088389},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.522668"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28824,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.138148+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:24.138148+0000","last_peered":"2024-09-10T16:03:24.138148+0000","last_clean":"2024-09-10T16:03:24.138148+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:24.138148+0000","last_undegraded":"2024-09-10T16:03:24.138148+0000","last_fullsized":"2024-09-10T16:03:24.138148+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":23404,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:31.201841+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:31.201841+0000","last_peered":"2024-09-10T16:03:31.201841+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:31.201841+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1313,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2490}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20784,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146854+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146854+0000","last_peered":"2024-09-10T16:03:24.146854+0000","last_clean":"2024-09-10T16:03:24.146854+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146854+0000","last_undegraded":"2024-09-10T16:03:24.146854+0000","last_fullsized":"2024-09-10T16:03:24.146854+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135509+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:24.135509+0000","last_peered":"2024-09-10T16:03:24.135509+0000","last_clean":"2024-09-10T16:03:24.135509+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:24.135509+0000","last_undegraded":"2024-09-10T16:03:24.135509+0000","last_fullsized":"2024-09-10T16:03:24.135509+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28778,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135429+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:24.135429+0000","last_peered":"2024-09-10T16:03:24.135429+0000","last_clean":"2024-09-10T16:03:24.135429+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:24.135429+0000","last_undegraded":"2024-09-10T16:03:24.135429+0000","last_fullsized":"2024-09-10T16:03:24.135429+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:24.135450+0000","last_peered":"2024-09-10T16:03:24.135450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:24.135450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21068,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135417+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:24.135417+0000","last_peered":"2024-09-10T16:03:24.135417+0000","last_clean":"2024-09-10T16:03:24.135417+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:24.135417+0000","last_undegraded":"2024-09-10T16:03:24.135417+0000","last_fullsized":"2024-09-10T16:03:24.135417+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135158+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:24.135158+0000","last_peered":"2024-09-10T16:03:24.135158+0000","last_clean":"2024-09-10T16:03:24.135158+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:24.135158+0000","last_undegraded":"2024-09-10T16:03:24.135158+0000","last_fullsized":"2024-09-10T16:03:24.135158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28869,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135532+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:24.135532+0000","last_peered":"2024-09-10T16:03:24.135532+0000","last_clean":"2024-09-10T16:03:24.135532+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:24.135532+0000","last_undegraded":"2024-09-10T16:03:24.135532+0000","last_fullsized":"2024-09-10T16:03:24.135532+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21416,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135875+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:24.135875+0000","last_peered":"2024-09-10T16:03:24.135875+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:24.135875+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135720+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:24.135720+0000","last_peered":"2024-09-10T16:03:24.135720+0000","last_clean":"2024-09-10T16:03:24.135720+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:24.135720+0000","last_undegraded":"2024-09-10T16:03:24.135720+0000","last_fullsized":"2024-09-10T16:03:24.135720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":101,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135526+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:24.135526+0000","last_peered":"2024-09-10T16:03:24.135526+0000","last_clean":"2024-09-10T16:03:24.135526+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:24.135526+0000","last_undegraded":"2024-09-10T16:03:24.135526+0000","last_fullsized":"2024-09-10T16:03:24.135526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21714,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.239868+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:24.239868+0000","last_peered":"2024-09-10T16:03:24.239868+0000","last_clean":"2024-09-10T16:03:24.239868+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:24.239868+0000","last_undegraded":"2024-09-10T16:03:24.239868+0000","last_fullsized":"2024-09-10T16:03:24.239868+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135778+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:24.135778+0000","last_peered":"2024-09-10T16:03:24.135778+0000","last_clean":"2024-09-10T16:03:24.135778+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:24.135778+0000","last_undegraded":"2024-09-10T16:03:24.135778+0000","last_fullsized":"2024-09-10T16:03:24.135778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21487,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.137945+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:24.137945+0000","last_peered":"2024-09-10T16:03:24.137945+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:24.137945+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.136096+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:24.136096+0000","last_peered":"2024-09-10T16:03:24.136096+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:24.136096+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135300+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:24.135300+0000","last_peered":"2024-09-10T16:03:24.135300+0000","last_clean":"2024-09-10T16:03:24.135300+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:24.135300+0000","last_undegraded":"2024-09-10T16:03:24.135300+0000","last_fullsized":"2024-09-10T16:03:24.135300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135919+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:24.135919+0000","last_peered":"2024-09-10T16:03:24.135919+0000","last_clean":"2024-09-10T16:03:24.135919+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:24.135919+0000","last_undegraded":"2024-09-10T16:03:24.135919+0000","last_fullsized":"2024-09-10T16:03:24.135919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21878,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135573+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:24.135573+0000","last_peered":"2024-09-10T16:03:24.135573+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:24.135573+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26247,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626560,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479184,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892656,"statfs":{"total":96636764160,"available":96146079744,"internally_reserved":0,"allocated":524288,"data_stored":162154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490139310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149841,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527048,"kb_used_data":62264,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844792,"statfs":{"total":96636764160,"available":96097067008,"internally_reserved":0,"allocated":63758336,"data_stored":63439558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953612,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:35.973 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:36.309 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:36.309 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:36.321 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-10T16:03:34.655138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26249,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487864,"kb_used_data":187912,"kb_used_omap":12,"kb_used_meta":1299891,"kb_avail":281627656,"statfs":{"total":289910292480,"available":288386719744,"internally_reserved":0,"allocated":192421888,"data_stored":191422198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1331088389},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.522668"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28824,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.138148+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:24.138148+0000","last_peered":"2024-09-10T16:03:24.138148+0000","last_clean":"2024-09-10T16:03:24.138148+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:24.138148+0000","last_undegraded":"2024-09-10T16:03:24.138148+0000","last_fullsized":"2024-09-10T16:03:24.138148+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":23404,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:31.201841+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:31.201841+0000","last_peered":"2024-09-10T16:03:31.201841+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:31.201841+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1313,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2490}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20784,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146854+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146854+0000","last_peered":"2024-09-10T16:03:24.146854+0000","last_clean":"2024-09-10T16:03:24.146854+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146854+0000","last_undegraded":"2024-09-10T16:03:24.146854+0000","last_fullsized":"2024-09-10T16:03:24.146854+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135509+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:24.135509+0000","last_peered":"2024-09-10T16:03:24.135509+0000","last_clean":"2024-09-10T16:03:24.135509+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:24.135509+0000","last_undegraded":"2024-09-10T16:03:24.135509+0000","last_fullsized":"2024-09-10T16:03:24.135509+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28778,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135429+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:24.135429+0000","last_peered":"2024-09-10T16:03:24.135429+0000","last_clean":"2024-09-10T16:03:24.135429+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:24.135429+0000","last_undegraded":"2024-09-10T16:03:24.135429+0000","last_fullsized":"2024-09-10T16:03:24.135429+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:24.135450+0000","last_peered":"2024-09-10T16:03:24.135450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:24.135450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21068,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135417+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:24.135417+0000","last_peered":"2024-09-10T16:03:24.135417+0000","last_clean":"2024-09-10T16:03:24.135417+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:24.135417+0000","last_undegraded":"2024-09-10T16:03:24.135417+0000","last_fullsized":"2024-09-10T16:03:24.135417+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135158+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:24.135158+0000","last_peered":"2024-09-10T16:03:24.135158+0000","last_clean":"2024-09-10T16:03:24.135158+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:24.135158+0000","last_undegraded":"2024-09-10T16:03:24.135158+0000","last_fullsized":"2024-09-10T16:03:24.135158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28869,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135532+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:24.135532+0000","last_peered":"2024-09-10T16:03:24.135532+0000","last_clean":"2024-09-10T16:03:24.135532+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:24.135532+0000","last_undegraded":"2024-09-10T16:03:24.135532+0000","last_fullsized":"2024-09-10T16:03:24.135532+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21416,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135875+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:24.135875+0000","last_peered":"2024-09-10T16:03:24.135875+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:24.135875+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135720+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:24.135720+0000","last_peered":"2024-09-10T16:03:24.135720+0000","last_clean":"2024-09-10T16:03:24.135720+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:24.135720+0000","last_undegraded":"2024-09-10T16:03:24.135720+0000","last_fullsized":"2024-09-10T16:03:24.135720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":101,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135526+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:24.135526+0000","last_peered":"2024-09-10T16:03:24.135526+0000","last_clean":"2024-09-10T16:03:24.135526+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:24.135526+0000","last_undegraded":"2024-09-10T16:03:24.135526+0000","last_fullsized":"2024-09-10T16:03:24.135526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21714,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.239868+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:24.239868+0000","last_peered":"2024-09-10T16:03:24.239868+0000","last_clean":"2024-09-10T16:03:24.239868+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:24.239868+0000","last_undegraded":"2024-09-10T16:03:24.239868+0000","last_fullsized":"2024-09-10T16:03:24.239868+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135778+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:24.135778+0000","last_peered":"2024-09-10T16:03:24.135778+0000","last_clean":"2024-09-10T16:03:24.135778+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:24.135778+0000","last_undegraded":"2024-09-10T16:03:24.135778+0000","last_fullsized":"2024-09-10T16:03:24.135778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21487,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.137945+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:24.137945+0000","last_peered":"2024-09-10T16:03:24.137945+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:24.137945+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.136096+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:24.136096+0000","last_peered":"2024-09-10T16:03:24.136096+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:24.136096+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135300+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:24.135300+0000","last_peered":"2024-09-10T16:03:24.135300+0000","last_clean":"2024-09-10T16:03:24.135300+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:24.135300+0000","last_undegraded":"2024-09-10T16:03:24.135300+0000","last_fullsized":"2024-09-10T16:03:24.135300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135919+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:24.135919+0000","last_peered":"2024-09-10T16:03:24.135919+0000","last_clean":"2024-09-10T16:03:24.135919+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:24.135919+0000","last_undegraded":"2024-09-10T16:03:24.135919+0000","last_fullsized":"2024-09-10T16:03:24.135919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21878,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135573+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:24.135573+0000","last_peered":"2024-09-10T16:03:24.135573+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:24.135573+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26247,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626560,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479184,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892656,"statfs":{"total":96636764160,"available":96146079744,"internally_reserved":0,"allocated":524288,"data_stored":162154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490139310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149841,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527048,"kb_used_data":62264,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844792,"statfs":{"total":96636764160,"available":96097067008,"internally_reserved":0,"allocated":63758336,"data_stored":63439558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953612,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:36.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:03:36.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 23404, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:31.201841+0000', 'last_change': '2024-09-10T16:03:22.157387+0000', 'last_active': '2024-09-10T16:03:31.201841+0000', 'last_peered': '2024-09-10T16:03:31.201841+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:03:31.201841+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2490, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1313, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2490}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:36.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:36.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21070, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.135450+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:24.135450+0000', 'last_peered': '2024-09-10T16:03:24.135450+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:24.135450+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:36.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:36.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21416, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.135875+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:24.135875+0000', 'last_peered': '2024-09-10T16:03:24.135875+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:24.135875+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:36.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:36.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21487, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.137945+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:24.137945+0000', 'last_peered': '2024-09-10T16:03:24.137945+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:24.137945+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:36.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:36.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21611, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.136096+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:03:24.136096+0000', 'last_peered': '2024-09-10T16:03:24.136096+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:03:24.136096+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:36.324 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:36.324 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21878, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.135573+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:24.135573+0000', 'last_peered': '2024-09-10T16:03:24.135573+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:24.135573+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:36.324 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:36.701 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:36.702 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20401/6 objects degraded (340016.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":742,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1523572736,"bytes_avail":288386719744,"bytes_total":289910292480,"degraded_objects":20401,"degraded_total":6,"degraded_ratio":3400.1666666666661,"recovering_objects_per_sec":120,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1268,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:36.714 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:36.715 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:37.823 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:37.823 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:37.836 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-10T16:03:36.655460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19673,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26977,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487864,"kb_used_data":187912,"kb_used_omap":12,"kb_used_meta":1299891,"kb_avail":281627656,"statfs":{"total":289910292480,"available":288386719744,"internally_reserved":0,"allocated":192421888,"data_stored":191422198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1331088389},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1996,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001969"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28824,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.138148+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:24.138148+0000","last_peered":"2024-09-10T16:03:24.138148+0000","last_clean":"2024-09-10T16:03:24.138148+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:24.138148+0000","last_undegraded":"2024-09-10T16:03:24.138148+0000","last_fullsized":"2024-09-10T16:03:24.138148+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":24860,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:36.207074+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:36.207074+0000","last_peered":"2024-09-10T16:03:36.207074+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:36.207074+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1762}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20784,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146854+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146854+0000","last_peered":"2024-09-10T16:03:24.146854+0000","last_clean":"2024-09-10T16:03:24.146854+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146854+0000","last_undegraded":"2024-09-10T16:03:24.146854+0000","last_fullsized":"2024-09-10T16:03:24.146854+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135509+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:24.135509+0000","last_peered":"2024-09-10T16:03:24.135509+0000","last_clean":"2024-09-10T16:03:24.135509+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:24.135509+0000","last_undegraded":"2024-09-10T16:03:24.135509+0000","last_fullsized":"2024-09-10T16:03:24.135509+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28778,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135429+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:24.135429+0000","last_peered":"2024-09-10T16:03:24.135429+0000","last_clean":"2024-09-10T16:03:24.135429+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:24.135429+0000","last_undegraded":"2024-09-10T16:03:24.135429+0000","last_fullsized":"2024-09-10T16:03:24.135429+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:24.135450+0000","last_peered":"2024-09-10T16:03:24.135450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:24.135450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21068,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135417+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:24.135417+0000","last_peered":"2024-09-10T16:03:24.135417+0000","last_clean":"2024-09-10T16:03:24.135417+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:24.135417+0000","last_undegraded":"2024-09-10T16:03:24.135417+0000","last_fullsized":"2024-09-10T16:03:24.135417+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135158+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:24.135158+0000","last_peered":"2024-09-10T16:03:24.135158+0000","last_clean":"2024-09-10T16:03:24.135158+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:24.135158+0000","last_undegraded":"2024-09-10T16:03:24.135158+0000","last_fullsized":"2024-09-10T16:03:24.135158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28869,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135532+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:24.135532+0000","last_peered":"2024-09-10T16:03:24.135532+0000","last_clean":"2024-09-10T16:03:24.135532+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:24.135532+0000","last_undegraded":"2024-09-10T16:03:24.135532+0000","last_fullsized":"2024-09-10T16:03:24.135532+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21416,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135875+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:24.135875+0000","last_peered":"2024-09-10T16:03:24.135875+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:24.135875+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135720+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:24.135720+0000","last_peered":"2024-09-10T16:03:24.135720+0000","last_clean":"2024-09-10T16:03:24.135720+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:24.135720+0000","last_undegraded":"2024-09-10T16:03:24.135720+0000","last_fullsized":"2024-09-10T16:03:24.135720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":101,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135526+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:24.135526+0000","last_peered":"2024-09-10T16:03:24.135526+0000","last_clean":"2024-09-10T16:03:24.135526+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:24.135526+0000","last_undegraded":"2024-09-10T16:03:24.135526+0000","last_fullsized":"2024-09-10T16:03:24.135526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21714,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.239868+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:24.239868+0000","last_peered":"2024-09-10T16:03:24.239868+0000","last_clean":"2024-09-10T16:03:24.239868+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:24.239868+0000","last_undegraded":"2024-09-10T16:03:24.239868+0000","last_fullsized":"2024-09-10T16:03:24.239868+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135778+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:24.135778+0000","last_peered":"2024-09-10T16:03:24.135778+0000","last_clean":"2024-09-10T16:03:24.135778+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:24.135778+0000","last_undegraded":"2024-09-10T16:03:24.135778+0000","last_fullsized":"2024-09-10T16:03:24.135778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21487,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.137945+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:24.137945+0000","last_peered":"2024-09-10T16:03:24.137945+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:24.137945+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.136096+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:24.136096+0000","last_peered":"2024-09-10T16:03:24.136096+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:24.136096+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135300+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:24.135300+0000","last_peered":"2024-09-10T16:03:24.135300+0000","last_clean":"2024-09-10T16:03:24.135300+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:24.135300+0000","last_undegraded":"2024-09-10T16:03:24.135300+0000","last_fullsized":"2024-09-10T16:03:24.135300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135919+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:24.135919+0000","last_peered":"2024-09-10T16:03:24.135919+0000","last_clean":"2024-09-10T16:03:24.135919+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:24.135919+0000","last_undegraded":"2024-09-10T16:03:24.135919+0000","last_fullsized":"2024-09-10T16:03:24.135919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21878,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135573+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:24.135573+0000","last_peered":"2024-09-10T16:03:24.135573+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:24.135573+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19673,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26975,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626560,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479184,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892656,"statfs":{"total":96636764160,"available":96146079744,"internally_reserved":0,"allocated":524288,"data_stored":162154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490139310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149841,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527048,"kb_used_data":62264,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844792,"statfs":{"total":96636764160,"available":96097067008,"internally_reserved":0,"allocated":63758336,"data_stored":63439558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953613,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:40.838 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:41.138 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:41.139 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:41.151 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-09-10T16:03:40.656139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19673,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26977,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492024,"kb_used_data":187912,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623496,"statfs":{"total":289910292480,"available":288382459904,"internally_reserved":0,"allocated":192421888,"data_stored":191422198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002008"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28824,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.138148+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:24.138148+0000","last_peered":"2024-09-10T16:03:24.138148+0000","last_clean":"2024-09-10T16:03:24.138148+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:24.138148+0000","last_undegraded":"2024-09-10T16:03:24.138148+0000","last_fullsized":"2024-09-10T16:03:24.138148+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":24860,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:36.207074+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:36.207074+0000","last_peered":"2024-09-10T16:03:36.207074+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:36.207074+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1762}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20784,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146854+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146854+0000","last_peered":"2024-09-10T16:03:24.146854+0000","last_clean":"2024-09-10T16:03:24.146854+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146854+0000","last_undegraded":"2024-09-10T16:03:24.146854+0000","last_fullsized":"2024-09-10T16:03:24.146854+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135509+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:24.135509+0000","last_peered":"2024-09-10T16:03:24.135509+0000","last_clean":"2024-09-10T16:03:24.135509+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:24.135509+0000","last_undegraded":"2024-09-10T16:03:24.135509+0000","last_fullsized":"2024-09-10T16:03:24.135509+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28778,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135429+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:24.135429+0000","last_peered":"2024-09-10T16:03:24.135429+0000","last_clean":"2024-09-10T16:03:24.135429+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:24.135429+0000","last_undegraded":"2024-09-10T16:03:24.135429+0000","last_fullsized":"2024-09-10T16:03:24.135429+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:24.135450+0000","last_peered":"2024-09-10T16:03:24.135450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:24.135450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21068,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135417+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:24.135417+0000","last_peered":"2024-09-10T16:03:24.135417+0000","last_clean":"2024-09-10T16:03:24.135417+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:24.135417+0000","last_undegraded":"2024-09-10T16:03:24.135417+0000","last_fullsized":"2024-09-10T16:03:24.135417+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135158+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:24.135158+0000","last_peered":"2024-09-10T16:03:24.135158+0000","last_clean":"2024-09-10T16:03:24.135158+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:24.135158+0000","last_undegraded":"2024-09-10T16:03:24.135158+0000","last_fullsized":"2024-09-10T16:03:24.135158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28869,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135532+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:24.135532+0000","last_peered":"2024-09-10T16:03:24.135532+0000","last_clean":"2024-09-10T16:03:24.135532+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:24.135532+0000","last_undegraded":"2024-09-10T16:03:24.135532+0000","last_fullsized":"2024-09-10T16:03:24.135532+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21416,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135875+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:24.135875+0000","last_peered":"2024-09-10T16:03:24.135875+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:24.135875+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135720+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:24.135720+0000","last_peered":"2024-09-10T16:03:24.135720+0000","last_clean":"2024-09-10T16:03:24.135720+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:24.135720+0000","last_undegraded":"2024-09-10T16:03:24.135720+0000","last_fullsized":"2024-09-10T16:03:24.135720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":101,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135526+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:24.135526+0000","last_peered":"2024-09-10T16:03:24.135526+0000","last_clean":"2024-09-10T16:03:24.135526+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:24.135526+0000","last_undegraded":"2024-09-10T16:03:24.135526+0000","last_fullsized":"2024-09-10T16:03:24.135526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21714,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.239868+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:24.239868+0000","last_peered":"2024-09-10T16:03:24.239868+0000","last_clean":"2024-09-10T16:03:24.239868+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:24.239868+0000","last_undegraded":"2024-09-10T16:03:24.239868+0000","last_fullsized":"2024-09-10T16:03:24.239868+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135778+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:24.135778+0000","last_peered":"2024-09-10T16:03:24.135778+0000","last_clean":"2024-09-10T16:03:24.135778+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:24.135778+0000","last_undegraded":"2024-09-10T16:03:24.135778+0000","last_fullsized":"2024-09-10T16:03:24.135778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21487,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.137945+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:24.137945+0000","last_peered":"2024-09-10T16:03:24.137945+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:24.137945+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.136096+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:24.136096+0000","last_peered":"2024-09-10T16:03:24.136096+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:24.136096+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135300+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:24.135300+0000","last_peered":"2024-09-10T16:03:24.135300+0000","last_clean":"2024-09-10T16:03:24.135300+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:24.135300+0000","last_undegraded":"2024-09-10T16:03:24.135300+0000","last_fullsized":"2024-09-10T16:03:24.135300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135919+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:24.135919+0000","last_peered":"2024-09-10T16:03:24.135919+0000","last_clean":"2024-09-10T16:03:24.135919+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:24.135919+0000","last_undegraded":"2024-09-10T16:03:24.135919+0000","last_fullsized":"2024-09-10T16:03:24.135919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21878,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135573+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:24.135573+0000","last_peered":"2024-09-10T16:03:24.135573+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:24.135573+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19673,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26975,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483344,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888496,"statfs":{"total":96636764160,"available":96141819904,"internally_reserved":0,"allocated":524288,"data_stored":162154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149842,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527048,"kb_used_data":62264,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844792,"statfs":{"total":96636764160,"available":96097067008,"internally_reserved":0,"allocated":63758336,"data_stored":63439558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953613,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:41.152 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:41.456 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:41.457 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:41.469 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-09-10T16:03:40.656139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19673,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26977,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492024,"kb_used_data":187912,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623496,"statfs":{"total":289910292480,"available":288382459904,"internally_reserved":0,"allocated":192421888,"data_stored":191422198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002008"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28824,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.138148+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:24.138148+0000","last_peered":"2024-09-10T16:03:24.138148+0000","last_clean":"2024-09-10T16:03:24.138148+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:24.138148+0000","last_undegraded":"2024-09-10T16:03:24.138148+0000","last_fullsized":"2024-09-10T16:03:24.138148+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":24860,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:36.207074+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:36.207074+0000","last_peered":"2024-09-10T16:03:36.207074+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:36.207074+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1762}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20784,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146854+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146854+0000","last_peered":"2024-09-10T16:03:24.146854+0000","last_clean":"2024-09-10T16:03:24.146854+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146854+0000","last_undegraded":"2024-09-10T16:03:24.146854+0000","last_fullsized":"2024-09-10T16:03:24.146854+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135509+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:24.135509+0000","last_peered":"2024-09-10T16:03:24.135509+0000","last_clean":"2024-09-10T16:03:24.135509+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:24.135509+0000","last_undegraded":"2024-09-10T16:03:24.135509+0000","last_fullsized":"2024-09-10T16:03:24.135509+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28778,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135429+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:24.135429+0000","last_peered":"2024-09-10T16:03:24.135429+0000","last_clean":"2024-09-10T16:03:24.135429+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:24.135429+0000","last_undegraded":"2024-09-10T16:03:24.135429+0000","last_fullsized":"2024-09-10T16:03:24.135429+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:24.135450+0000","last_peered":"2024-09-10T16:03:24.135450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:24.135450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21068,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135417+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:24.135417+0000","last_peered":"2024-09-10T16:03:24.135417+0000","last_clean":"2024-09-10T16:03:24.135417+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:24.135417+0000","last_undegraded":"2024-09-10T16:03:24.135417+0000","last_fullsized":"2024-09-10T16:03:24.135417+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135158+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:24.135158+0000","last_peered":"2024-09-10T16:03:24.135158+0000","last_clean":"2024-09-10T16:03:24.135158+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:24.135158+0000","last_undegraded":"2024-09-10T16:03:24.135158+0000","last_fullsized":"2024-09-10T16:03:24.135158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28869,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135532+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:24.135532+0000","last_peered":"2024-09-10T16:03:24.135532+0000","last_clean":"2024-09-10T16:03:24.135532+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:24.135532+0000","last_undegraded":"2024-09-10T16:03:24.135532+0000","last_fullsized":"2024-09-10T16:03:24.135532+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21416,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135875+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:24.135875+0000","last_peered":"2024-09-10T16:03:24.135875+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:24.135875+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135720+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:24.135720+0000","last_peered":"2024-09-10T16:03:24.135720+0000","last_clean":"2024-09-10T16:03:24.135720+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:24.135720+0000","last_undegraded":"2024-09-10T16:03:24.135720+0000","last_fullsized":"2024-09-10T16:03:24.135720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":101,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135526+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:24.135526+0000","last_peered":"2024-09-10T16:03:24.135526+0000","last_clean":"2024-09-10T16:03:24.135526+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:24.135526+0000","last_undegraded":"2024-09-10T16:03:24.135526+0000","last_fullsized":"2024-09-10T16:03:24.135526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21714,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.239868+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:24.239868+0000","last_peered":"2024-09-10T16:03:24.239868+0000","last_clean":"2024-09-10T16:03:24.239868+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:24.239868+0000","last_undegraded":"2024-09-10T16:03:24.239868+0000","last_fullsized":"2024-09-10T16:03:24.239868+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135778+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:24.135778+0000","last_peered":"2024-09-10T16:03:24.135778+0000","last_clean":"2024-09-10T16:03:24.135778+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:24.135778+0000","last_undegraded":"2024-09-10T16:03:24.135778+0000","last_fullsized":"2024-09-10T16:03:24.135778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21487,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.137945+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:24.137945+0000","last_peered":"2024-09-10T16:03:24.137945+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:24.137945+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.136096+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:24.136096+0000","last_peered":"2024-09-10T16:03:24.136096+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:24.136096+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135300+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:24.135300+0000","last_peered":"2024-09-10T16:03:24.135300+0000","last_clean":"2024-09-10T16:03:24.135300+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:24.135300+0000","last_undegraded":"2024-09-10T16:03:24.135300+0000","last_fullsized":"2024-09-10T16:03:24.135300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135919+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:24.135919+0000","last_peered":"2024-09-10T16:03:24.135919+0000","last_clean":"2024-09-10T16:03:24.135919+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:24.135919+0000","last_undegraded":"2024-09-10T16:03:24.135919+0000","last_fullsized":"2024-09-10T16:03:24.135919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21878,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135573+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:24.135573+0000","last_peered":"2024-09-10T16:03:24.135573+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:24.135573+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19673,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26975,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483344,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888496,"statfs":{"total":96636764160,"available":96141819904,"internally_reserved":0,"allocated":524288,"data_stored":162154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149842,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527048,"kb_used_data":62264,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844792,"statfs":{"total":96636764160,"available":96097067008,"internally_reserved":0,"allocated":63758336,"data_stored":63439558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953613,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:41.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:03:41.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 24860, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:36.207074+0000', 'last_change': '2024-09-10T16:03:22.157387+0000', 'last_active': '2024-09-10T16:03:36.207074+0000', 'last_peered': '2024-09-10T16:03:36.207074+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:03:36.207074+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1762, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2041, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1762}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:41.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:41.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21070, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.135450+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:24.135450+0000', 'last_peered': '2024-09-10T16:03:24.135450+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:24.135450+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:41.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:41.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21416, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.135875+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:24.135875+0000', 'last_peered': '2024-09-10T16:03:24.135875+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:24.135875+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:41.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:41.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21487, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.137945+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:24.137945+0000', 'last_peered': '2024-09-10T16:03:24.137945+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:24.137945+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:41.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:41.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21611, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.136096+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:03:24.136096+0000', 'last_peered': '2024-09-10T16:03:24.136096+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:03:24.136096+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:41.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:41.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21878, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.135573+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:24.135573+0000', 'last_peered': '2024-09-10T16:03:24.135573+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:24.135573+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:41.472 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:41.857 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:41.857 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19673/6 objects degraded (327883.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":747,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1527832576,"bytes_avail":288382459904,"bytes_total":289910292480,"degraded_objects":19673,"degraded_total":6,"degraded_ratio":3278.8333333333339,"recovering_objects_per_sec":113,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1364,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:41.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:41.871 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:42.167 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:42.168 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:42.179 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-09-10T16:03:40.656139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19673,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26977,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492024,"kb_used_data":187912,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623496,"statfs":{"total":289910292480,"available":288382459904,"internally_reserved":0,"allocated":192421888,"data_stored":191422198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002008"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28824,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.138148+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:24.138148+0000","last_peered":"2024-09-10T16:03:24.138148+0000","last_clean":"2024-09-10T16:03:24.138148+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:24.138148+0000","last_undegraded":"2024-09-10T16:03:24.138148+0000","last_fullsized":"2024-09-10T16:03:24.138148+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":24860,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:36.207074+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:36.207074+0000","last_peered":"2024-09-10T16:03:36.207074+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:36.207074+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1762}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20784,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146854+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146854+0000","last_peered":"2024-09-10T16:03:24.146854+0000","last_clean":"2024-09-10T16:03:24.146854+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146854+0000","last_undegraded":"2024-09-10T16:03:24.146854+0000","last_fullsized":"2024-09-10T16:03:24.146854+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135509+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:24.135509+0000","last_peered":"2024-09-10T16:03:24.135509+0000","last_clean":"2024-09-10T16:03:24.135509+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:24.135509+0000","last_undegraded":"2024-09-10T16:03:24.135509+0000","last_fullsized":"2024-09-10T16:03:24.135509+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28778,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135429+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:24.135429+0000","last_peered":"2024-09-10T16:03:24.135429+0000","last_clean":"2024-09-10T16:03:24.135429+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:24.135429+0000","last_undegraded":"2024-09-10T16:03:24.135429+0000","last_fullsized":"2024-09-10T16:03:24.135429+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:24.135450+0000","last_peered":"2024-09-10T16:03:24.135450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:24.135450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21068,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135417+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:24.135417+0000","last_peered":"2024-09-10T16:03:24.135417+0000","last_clean":"2024-09-10T16:03:24.135417+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:24.135417+0000","last_undegraded":"2024-09-10T16:03:24.135417+0000","last_fullsized":"2024-09-10T16:03:24.135417+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135158+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:24.135158+0000","last_peered":"2024-09-10T16:03:24.135158+0000","last_clean":"2024-09-10T16:03:24.135158+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:24.135158+0000","last_undegraded":"2024-09-10T16:03:24.135158+0000","last_fullsized":"2024-09-10T16:03:24.135158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28869,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135532+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:24.135532+0000","last_peered":"2024-09-10T16:03:24.135532+0000","last_clean":"2024-09-10T16:03:24.135532+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:24.135532+0000","last_undegraded":"2024-09-10T16:03:24.135532+0000","last_fullsized":"2024-09-10T16:03:24.135532+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21416,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135875+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:24.135875+0000","last_peered":"2024-09-10T16:03:24.135875+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:24.135875+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135720+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:24.135720+0000","last_peered":"2024-09-10T16:03:24.135720+0000","last_clean":"2024-09-10T16:03:24.135720+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:24.135720+0000","last_undegraded":"2024-09-10T16:03:24.135720+0000","last_fullsized":"2024-09-10T16:03:24.135720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":101,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135526+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:24.135526+0000","last_peered":"2024-09-10T16:03:24.135526+0000","last_clean":"2024-09-10T16:03:24.135526+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:24.135526+0000","last_undegraded":"2024-09-10T16:03:24.135526+0000","last_fullsized":"2024-09-10T16:03:24.135526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21714,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.239868+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:24.239868+0000","last_peered":"2024-09-10T16:03:24.239868+0000","last_clean":"2024-09-10T16:03:24.239868+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:24.239868+0000","last_undegraded":"2024-09-10T16:03:24.239868+0000","last_fullsized":"2024-09-10T16:03:24.239868+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135778+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:24.135778+0000","last_peered":"2024-09-10T16:03:24.135778+0000","last_clean":"2024-09-10T16:03:24.135778+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:24.135778+0000","last_undegraded":"2024-09-10T16:03:24.135778+0000","last_fullsized":"2024-09-10T16:03:24.135778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21487,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.137945+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:24.137945+0000","last_peered":"2024-09-10T16:03:24.137945+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:24.137945+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.136096+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:24.136096+0000","last_peered":"2024-09-10T16:03:24.136096+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:24.136096+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135300+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:24.135300+0000","last_peered":"2024-09-10T16:03:24.135300+0000","last_clean":"2024-09-10T16:03:24.135300+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:24.135300+0000","last_undegraded":"2024-09-10T16:03:24.135300+0000","last_fullsized":"2024-09-10T16:03:24.135300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135919+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:24.135919+0000","last_peered":"2024-09-10T16:03:24.135919+0000","last_clean":"2024-09-10T16:03:24.135919+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:24.135919+0000","last_undegraded":"2024-09-10T16:03:24.135919+0000","last_fullsized":"2024-09-10T16:03:24.135919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21878,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135573+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:24.135573+0000","last_peered":"2024-09-10T16:03:24.135573+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:24.135573+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19673,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26975,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483344,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888496,"statfs":{"total":96636764160,"available":96141819904,"internally_reserved":0,"allocated":524288,"data_stored":162154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149842,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527048,"kb_used_data":62264,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844792,"statfs":{"total":96636764160,"available":96097067008,"internally_reserved":0,"allocated":63758336,"data_stored":63439558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953613,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:45.182 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:45.528 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:45.529 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:45.544 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-10T16:03:45.305834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27877,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500344,"kb_used_data":187912,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281615176,"statfs":{"total":289910292480,"available":288373940224,"internally_reserved":0,"allocated":192421888,"data_stored":191422198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1343867909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1628,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1628,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.650695"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28824,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.138148+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:24.138148+0000","last_peered":"2024-09-10T16:03:24.138148+0000","last_clean":"2024-09-10T16:03:24.138148+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:24.138148+0000","last_undegraded":"2024-09-10T16:03:24.138148+0000","last_fullsized":"2024-09-10T16:03:24.138148+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":26660,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:41.207157+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:41.207157+0000","last_peered":"2024-09-10T16:03:41.207157+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:41.207157+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2941,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-862}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20785,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146898+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146898+0000","last_peered":"2024-09-10T16:03:24.146898+0000","last_clean":"2024-09-10T16:03:24.146898+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146898+0000","last_undegraded":"2024-09-10T16:03:24.146898+0000","last_fullsized":"2024-09-10T16:03:24.146898+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135509+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:24.135509+0000","last_peered":"2024-09-10T16:03:24.135509+0000","last_clean":"2024-09-10T16:03:24.135509+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:24.135509+0000","last_undegraded":"2024-09-10T16:03:24.135509+0000","last_fullsized":"2024-09-10T16:03:24.135509+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28778,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135429+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:24.135429+0000","last_peered":"2024-09-10T16:03:24.135429+0000","last_clean":"2024-09-10T16:03:24.135429+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:24.135429+0000","last_undegraded":"2024-09-10T16:03:24.135429+0000","last_fullsized":"2024-09-10T16:03:24.135429+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:24.135450+0000","last_peered":"2024-09-10T16:03:24.135450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:24.135450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21068,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135417+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:24.135417+0000","last_peered":"2024-09-10T16:03:24.135417+0000","last_clean":"2024-09-10T16:03:24.135417+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:24.135417+0000","last_undegraded":"2024-09-10T16:03:24.135417+0000","last_fullsized":"2024-09-10T16:03:24.135417+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135158+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:24.135158+0000","last_peered":"2024-09-10T16:03:24.135158+0000","last_clean":"2024-09-10T16:03:24.135158+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:24.135158+0000","last_undegraded":"2024-09-10T16:03:24.135158+0000","last_fullsized":"2024-09-10T16:03:24.135158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28869,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135532+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:24.135532+0000","last_peered":"2024-09-10T16:03:24.135532+0000","last_clean":"2024-09-10T16:03:24.135532+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:24.135532+0000","last_undegraded":"2024-09-10T16:03:24.135532+0000","last_fullsized":"2024-09-10T16:03:24.135532+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21416,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135875+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:24.135875+0000","last_peered":"2024-09-10T16:03:24.135875+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:24.135875+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135720+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:24.135720+0000","last_peered":"2024-09-10T16:03:24.135720+0000","last_clean":"2024-09-10T16:03:24.135720+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:24.135720+0000","last_undegraded":"2024-09-10T16:03:24.135720+0000","last_fullsized":"2024-09-10T16:03:24.135720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":101,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135526+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:24.135526+0000","last_peered":"2024-09-10T16:03:24.135526+0000","last_clean":"2024-09-10T16:03:24.135526+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:24.135526+0000","last_undegraded":"2024-09-10T16:03:24.135526+0000","last_fullsized":"2024-09-10T16:03:24.135526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21714,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.239868+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:24.239868+0000","last_peered":"2024-09-10T16:03:24.239868+0000","last_clean":"2024-09-10T16:03:24.239868+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:24.239868+0000","last_undegraded":"2024-09-10T16:03:24.239868+0000","last_fullsized":"2024-09-10T16:03:24.239868+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135778+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:24.135778+0000","last_peered":"2024-09-10T16:03:24.135778+0000","last_clean":"2024-09-10T16:03:24.135778+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:24.135778+0000","last_undegraded":"2024-09-10T16:03:24.135778+0000","last_fullsized":"2024-09-10T16:03:24.135778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21487,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.137945+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:24.137945+0000","last_peered":"2024-09-10T16:03:24.137945+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:24.137945+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.136096+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:24.136096+0000","last_peered":"2024-09-10T16:03:24.136096+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:24.136096+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135300+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:24.135300+0000","last_peered":"2024-09-10T16:03:24.135300+0000","last_clean":"2024-09-10T16:03:24.135300+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:24.135300+0000","last_undegraded":"2024-09-10T16:03:24.135300+0000","last_fullsized":"2024-09-10T16:03:24.135300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135919+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:24.135919+0000","last_peered":"2024-09-10T16:03:24.135919+0000","last_clean":"2024-09-10T16:03:24.135919+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:24.135919+0000","last_undegraded":"2024-09-10T16:03:24.135919+0000","last_fullsized":"2024-09-10T16:03:24.135919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21878,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135573+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:24.135573+0000","last_peered":"2024-09-10T16:03:24.135573+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:24.135573+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27875,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491664,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880176,"statfs":{"total":96636764160,"available":96133300224,"internally_reserved":0,"allocated":524288,"data_stored":162154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502918830},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149843,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527048,"kb_used_data":62264,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844792,"statfs":{"total":96636764160,"available":96097067008,"internally_reserved":0,"allocated":63758336,"data_stored":63439558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953614,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:45.545 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:45.910 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:45.911 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:45.924 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-10T16:03:45.305834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27877,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500344,"kb_used_data":187912,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281615176,"statfs":{"total":289910292480,"available":288373940224,"internally_reserved":0,"allocated":192421888,"data_stored":191422198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1343867909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1628,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1628,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.650695"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28824,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.138148+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:24.138148+0000","last_peered":"2024-09-10T16:03:24.138148+0000","last_clean":"2024-09-10T16:03:24.138148+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:24.138148+0000","last_undegraded":"2024-09-10T16:03:24.138148+0000","last_fullsized":"2024-09-10T16:03:24.138148+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21204,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147129+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:24.147129+0000","last_peered":"2024-09-10T16:03:24.147129+0000","last_clean":"2024-09-10T16:03:24.147129+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:24.147129+0000","last_undegraded":"2024-09-10T16:03:24.147129+0000","last_fullsized":"2024-09-10T16:03:24.147129+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":26660,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:41.207157+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:41.207157+0000","last_peered":"2024-09-10T16:03:41.207157+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:41.207157+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2941,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-862}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20785,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146898+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:24.146898+0000","last_peered":"2024-09-10T16:03:24.146898+0000","last_clean":"2024-09-10T16:03:24.146898+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:24.146898+0000","last_undegraded":"2024-09-10T16:03:24.146898+0000","last_fullsized":"2024-09-10T16:03:24.146898+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135509+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:24.135509+0000","last_peered":"2024-09-10T16:03:24.135509+0000","last_clean":"2024-09-10T16:03:24.135509+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:24.135509+0000","last_undegraded":"2024-09-10T16:03:24.135509+0000","last_fullsized":"2024-09-10T16:03:24.135509+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28778,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135429+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:24.135429+0000","last_peered":"2024-09-10T16:03:24.135429+0000","last_clean":"2024-09-10T16:03:24.135429+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:24.135429+0000","last_undegraded":"2024-09-10T16:03:24.135429+0000","last_fullsized":"2024-09-10T16:03:24.135429+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:24.135450+0000","last_peered":"2024-09-10T16:03:24.135450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:24.135450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21068,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135417+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:24.135417+0000","last_peered":"2024-09-10T16:03:24.135417+0000","last_clean":"2024-09-10T16:03:24.135417+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:24.135417+0000","last_undegraded":"2024-09-10T16:03:24.135417+0000","last_fullsized":"2024-09-10T16:03:24.135417+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135158+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:24.135158+0000","last_peered":"2024-09-10T16:03:24.135158+0000","last_clean":"2024-09-10T16:03:24.135158+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:24.135158+0000","last_undegraded":"2024-09-10T16:03:24.135158+0000","last_fullsized":"2024-09-10T16:03:24.135158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28869,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135532+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:24.135532+0000","last_peered":"2024-09-10T16:03:24.135532+0000","last_clean":"2024-09-10T16:03:24.135532+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:24.135532+0000","last_undegraded":"2024-09-10T16:03:24.135532+0000","last_fullsized":"2024-09-10T16:03:24.135532+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146749+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:24.146749+0000","last_peered":"2024-09-10T16:03:24.146749+0000","last_clean":"2024-09-10T16:03:24.146749+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:24.146749+0000","last_undegraded":"2024-09-10T16:03:24.146749+0000","last_fullsized":"2024-09-10T16:03:24.146749+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21416,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135875+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:24.135875+0000","last_peered":"2024-09-10T16:03:24.135875+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:24.135875+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135720+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:24.135720+0000","last_peered":"2024-09-10T16:03:24.135720+0000","last_clean":"2024-09-10T16:03:24.135720+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:24.135720+0000","last_undegraded":"2024-09-10T16:03:24.135720+0000","last_fullsized":"2024-09-10T16:03:24.135720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:24.146834+0000","last_peered":"2024-09-10T16:03:24.146834+0000","last_clean":"2024-09-10T16:03:24.146834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:24.146834+0000","last_undegraded":"2024-09-10T16:03:24.146834+0000","last_fullsized":"2024-09-10T16:03:24.146834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":101,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135526+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:24.135526+0000","last_peered":"2024-09-10T16:03:24.135526+0000","last_clean":"2024-09-10T16:03:24.135526+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:24.135526+0000","last_undegraded":"2024-09-10T16:03:24.135526+0000","last_fullsized":"2024-09-10T16:03:24.135526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28276,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.146672+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:24.146672+0000","last_peered":"2024-09-10T16:03:24.146672+0000","last_clean":"2024-09-10T16:03:24.146672+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:24.146672+0000","last_undegraded":"2024-09-10T16:03:24.146672+0000","last_fullsized":"2024-09-10T16:03:24.146672+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21714,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.239868+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:24.239868+0000","last_peered":"2024-09-10T16:03:24.239868+0000","last_clean":"2024-09-10T16:03:24.239868+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:24.239868+0000","last_undegraded":"2024-09-10T16:03:24.239868+0000","last_fullsized":"2024-09-10T16:03:24.239868+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135778+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:24.135778+0000","last_peered":"2024-09-10T16:03:24.135778+0000","last_clean":"2024-09-10T16:03:24.135778+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:24.135778+0000","last_undegraded":"2024-09-10T16:03:24.135778+0000","last_fullsized":"2024-09-10T16:03:24.135778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21487,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.137945+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:24.137945+0000","last_peered":"2024-09-10T16:03:24.137945+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:24.137945+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":117,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.147041+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:24.147041+0000","last_peered":"2024-09-10T16:03:24.147041+0000","last_clean":"2024-09-10T16:03:24.147041+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:24.147041+0000","last_undegraded":"2024-09-10T16:03:24.147041+0000","last_fullsized":"2024-09-10T16:03:24.147041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.136096+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:24.136096+0000","last_peered":"2024-09-10T16:03:24.136096+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:24.136096+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135300+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:24.135300+0000","last_peered":"2024-09-10T16:03:24.135300+0000","last_clean":"2024-09-10T16:03:24.135300+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:24.135300+0000","last_undegraded":"2024-09-10T16:03:24.135300+0000","last_fullsized":"2024-09-10T16:03:24.135300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135919+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:24.135919+0000","last_peered":"2024-09-10T16:03:24.135919+0000","last_clean":"2024-09-10T16:03:24.135919+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:24.135919+0000","last_undegraded":"2024-09-10T16:03:24.135919+0000","last_fullsized":"2024-09-10T16:03:24.135919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21878,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135573+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:24.135573+0000","last_peered":"2024-09-10T16:03:24.135573+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:24.135573+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27875,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":191008768,"data_stored":190954573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491664,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880176,"statfs":{"total":96636764160,"available":96133300224,"internally_reserved":0,"allocated":524288,"data_stored":162154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502918830},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149843,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527048,"kb_used_data":62264,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844792,"statfs":{"total":96636764160,"available":96097067008,"internally_reserved":0,"allocated":63758336,"data_stored":63439558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953614,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481632,"kb_used_data":125136,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890208,"statfs":{"total":96636764160,"available":96143572992,"internally_reserved":0,"allocated":128139264,"data_stored":127820486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127668224,"data_stored":127664611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:45.926 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-10T16:03:45.926 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "50'20630", 'reported_seq': 26660, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:41.207157+0000', 'last_change': '2024-09-10T16:03:22.157387+0000', 'last_active': '2024-09-10T16:03:41.207157+0000', 'last_peered': '2024-09-10T16:03:41.207157+0000', 'last_clean': '2024-09-10T15:53:17.329672+0000', 'last_became_active': '2024-09-10T15:53:19.963691+0000', 'last_became_peered': '2024-09-10T15:53:19.963691+0000', 'last_unstale': '2024-09-10T16:03:41.207157+0000', 'last_undegraded': '2024-09-10T15:53:19.509162+0000', 'last_fullsized': '2024-09-10T15:53:19.470428+0000', 'mapping_epoch': 43, 'log_start': "46'10600", 'ondisk_log_start': "46'10600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1763", 'last_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:52:50.491042+0000', 'objects_scrubbed': 1728, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 573, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 862, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18898, 'num_write_kb': 34337, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2941, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -862}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T16:03:45.926 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:45.926 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21070, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.135450+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:24.135450+0000', 'last_peered': '2024-09-10T16:03:24.135450+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:24.135450+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:45.926 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:45.926 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21416, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.135875+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:24.135875+0000', 'last_peered': '2024-09-10T16:03:24.135875+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:24.135875+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:45.926 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:45.926 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21487, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.137945+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:24.137945+0000', 'last_peered': '2024-09-10T16:03:24.137945+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:24.137945+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:45.927 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:45.927 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 21611, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.136096+0000', 'last_change': '2024-09-10T15:53:19.946104+0000', 'last_active': '2024-09-10T16:03:24.136096+0000', 'last_peered': '2024-09-10T16:03:24.136096+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:03:24.136096+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3538, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:45.927 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:45.927 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21878, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:24.135573+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:24.135573+0000', 'last_peered': '2024-09-10T16:03:24.135573+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:24.135573+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:45.927 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:46.375 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:46.375 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18773/6 objects degraded (312883.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":752,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1536352256,"bytes_avail":288373940224,"bytes_total":289910292480,"degraded_objects":18773,"degraded_total":6,"degraded_ratio":3128.8333333333339,"recovering_objects_per_sec":135,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1628,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:46.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:46.389 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:46.685 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:46.685 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:46.698 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-10T16:03:46.657146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27877,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500000,"kb_used_data":187568,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281615520,"statfs":{"total":289910292480,"available":288374292480,"internally_reserved":0,"allocated":192069632,"data_stored":191068682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1343867909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-900,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":900,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001363"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28824,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.138148+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:24.138148+0000","last_peered":"2024-09-10T16:03:24.138148+0000","last_clean":"2024-09-10T16:03:24.138148+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:24.138148+0000","last_undegraded":"2024-09-10T16:03:24.138148+0000","last_fullsized":"2024-09-10T16:03:24.138148+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21206,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.311689+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:45.311689+0000","last_peered":"2024-09-10T16:03:45.311689+0000","last_clean":"2024-09-10T16:03:45.311689+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:45.311689+0000","last_undegraded":"2024-09-10T16:03:45.311689+0000","last_fullsized":"2024-09-10T16:03:45.311689+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":26660,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:41.207157+0000","last_change":"2024-09-10T16:03:22.157387+0000","last_active":"2024-09-10T16:03:41.207157+0000","last_peered":"2024-09-10T16:03:41.207157+0000","last_clean":"2024-09-10T15:53:17.329672+0000","last_became_active":"2024-09-10T15:53:19.963691+0000","last_became_peered":"2024-09-10T15:53:19.963691+0000","last_unstale":"2024-09-10T16:03:41.207157+0000","last_undegraded":"2024-09-10T15:53:19.509162+0000","last_fullsized":"2024-09-10T15:53:19.470428+0000","mapping_epoch":43,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2941,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-862}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20787,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.309531+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:45.309531+0000","last_peered":"2024-09-10T16:03:45.309531+0000","last_clean":"2024-09-10T16:03:45.309531+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:45.309531+0000","last_undegraded":"2024-09-10T16:03:45.309531+0000","last_fullsized":"2024-09-10T16:03:45.309531+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135509+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:24.135509+0000","last_peered":"2024-09-10T16:03:24.135509+0000","last_clean":"2024-09-10T16:03:24.135509+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:24.135509+0000","last_undegraded":"2024-09-10T16:03:24.135509+0000","last_fullsized":"2024-09-10T16:03:24.135509+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28778,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135429+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:24.135429+0000","last_peered":"2024-09-10T16:03:24.135429+0000","last_clean":"2024-09-10T16:03:24.135429+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:24.135429+0000","last_undegraded":"2024-09-10T16:03:24.135429+0000","last_fullsized":"2024-09-10T16:03:24.135429+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135450+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:24.135450+0000","last_peered":"2024-09-10T16:03:24.135450+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:24.135450+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21068,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135417+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:24.135417+0000","last_peered":"2024-09-10T16:03:24.135417+0000","last_clean":"2024-09-10T16:03:24.135417+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:24.135417+0000","last_undegraded":"2024-09-10T16:03:24.135417+0000","last_fullsized":"2024-09-10T16:03:24.135417+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135158+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:24.135158+0000","last_peered":"2024-09-10T16:03:24.135158+0000","last_clean":"2024-09-10T16:03:24.135158+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:24.135158+0000","last_undegraded":"2024-09-10T16:03:24.135158+0000","last_fullsized":"2024-09-10T16:03:24.135158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28869,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135532+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:24.135532+0000","last_peered":"2024-09-10T16:03:24.135532+0000","last_clean":"2024-09-10T16:03:24.135532+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:24.135532+0000","last_undegraded":"2024-09-10T16:03:24.135532+0000","last_fullsized":"2024-09-10T16:03:24.135532+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.309165+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:45.309165+0000","last_peered":"2024-09-10T16:03:45.309165+0000","last_clean":"2024-09-10T16:03:45.309165+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:45.309165+0000","last_undegraded":"2024-09-10T16:03:45.309165+0000","last_fullsized":"2024-09-10T16:03:45.309165+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21416,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135875+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:24.135875+0000","last_peered":"2024-09-10T16:03:24.135875+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:24.135875+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135720+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:24.135720+0000","last_peered":"2024-09-10T16:03:24.135720+0000","last_clean":"2024-09-10T16:03:24.135720+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:24.135720+0000","last_undegraded":"2024-09-10T16:03:24.135720+0000","last_fullsized":"2024-09-10T16:03:24.135720+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28610,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.309254+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:45.309254+0000","last_peered":"2024-09-10T16:03:45.309254+0000","last_clean":"2024-09-10T16:03:45.309254+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:45.309254+0000","last_undegraded":"2024-09-10T16:03:45.309254+0000","last_fullsized":"2024-09-10T16:03:45.309254+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":101,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135526+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:24.135526+0000","last_peered":"2024-09-10T16:03:24.135526+0000","last_clean":"2024-09-10T16:03:24.135526+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:24.135526+0000","last_undegraded":"2024-09-10T16:03:24.135526+0000","last_fullsized":"2024-09-10T16:03:24.135526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28278,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.308235+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:45.308235+0000","last_peered":"2024-09-10T16:03:45.308235+0000","last_clean":"2024-09-10T16:03:45.308235+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:45.308235+0000","last_undegraded":"2024-09-10T16:03:45.308235+0000","last_fullsized":"2024-09-10T16:03:45.308235+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21714,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.239868+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:24.239868+0000","last_peered":"2024-09-10T16:03:24.239868+0000","last_clean":"2024-09-10T16:03:24.239868+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:24.239868+0000","last_undegraded":"2024-09-10T16:03:24.239868+0000","last_fullsized":"2024-09-10T16:03:24.239868+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135778+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:24.135778+0000","last_peered":"2024-09-10T16:03:24.135778+0000","last_clean":"2024-09-10T16:03:24.135778+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:24.135778+0000","last_undegraded":"2024-09-10T16:03:24.135778+0000","last_fullsized":"2024-09-10T16:03:24.135778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21487,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.137945+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:24.137945+0000","last_peered":"2024-09-10T16:03:24.137945+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:24.137945+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.311628+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:45.311628+0000","last_peered":"2024-09-10T16:03:45.311628+0000","last_clean":"2024-09-10T16:03:45.311628+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:45.311628+0000","last_undegraded":"2024-09-10T16:03:45.311628+0000","last_fullsized":"2024-09-10T16:03:45.311628+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.136096+0000","last_change":"2024-09-10T15:53:19.946104+0000","last_active":"2024-09-10T16:03:24.136096+0000","last_peered":"2024-09-10T16:03:24.136096+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:24.136096+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3538,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3536}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135300+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:24.135300+0000","last_peered":"2024-09-10T16:03:24.135300+0000","last_clean":"2024-09-10T16:03:24.135300+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:24.135300+0000","last_undegraded":"2024-09-10T16:03:24.135300+0000","last_fullsized":"2024-09-10T16:03:24.135300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T16:03:24.135919+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:24.135919+0000","last_peered":"2024-09-10T16:03:24.135919+0000","last_clean":"2024-09-10T16:03:24.135919+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:24.135919+0000","last_undegraded":"2024-09-10T16:03:24.135919+0000","last_fullsized":"2024-09-10T16:03:24.135919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21878,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:24.135573+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:24.135573+0000","last_peered":"2024-09-10T16:03:24.135573+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:24.135573+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27875,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190283776,"data_stored":190229581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491664,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880176,"statfs":{"total":96636764160,"available":96133300224,"internally_reserved":0,"allocated":524288,"data_stored":162154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502918830},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149843,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527048,"kb_used_data":62264,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844792,"statfs":{"total":96636764160,"available":96097067008,"internally_reserved":0,"allocated":63758336,"data_stored":63439558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953615,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481288,"kb_used_data":124792,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890552,"statfs":{"total":96636764160,"available":96143925248,"internally_reserved":0,"allocated":127787008,"data_stored":127466970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126943232,"data_stored":126939619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:49.701 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:50.010 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:50.010 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:50.022 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-09-10T16:03:48.657443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29035,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491696,"kb_used_data":187584,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623824,"statfs":{"total":289910292480,"available":288382795776,"internally_reserved":0,"allocated":192086016,"data_stored":191076260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2058,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001303"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21206,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.311689+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:45.311689+0000","last_peered":"2024-09-10T16:03:45.311689+0000","last_clean":"2024-09-10T16:03:45.311689+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:45.311689+0000","last_undegraded":"2024-09-10T16:03:45.311689+0000","last_fullsized":"2024-09-10T16:03:45.311689+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20787,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.309531+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:45.309531+0000","last_peered":"2024-09-10T16:03:45.309531+0000","last_clean":"2024-09-10T16:03:45.309531+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:45.309531+0000","last_undegraded":"2024-09-10T16:03:45.309531+0000","last_fullsized":"2024-09-10T16:03:45.309531+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21072,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309134+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309134+0000","last_peered":"2024-09-10T16:03:46.309134+0000","last_clean":"2024-09-10T16:03:46.309134+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309134+0000","last_undegraded":"2024-09-10T16:03:46.309134+0000","last_fullsized":"2024-09-10T16:03:46.309134+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.309165+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:45.309165+0000","last_peered":"2024-09-10T16:03:45.309165+0000","last_clean":"2024-09-10T16:03:45.309165+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:45.309165+0000","last_undegraded":"2024-09-10T16:03:45.309165+0000","last_fullsized":"2024-09-10T16:03:45.309165+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28610,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.309254+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:45.309254+0000","last_peered":"2024-09-10T16:03:45.309254+0000","last_clean":"2024-09-10T16:03:45.309254+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:45.309254+0000","last_undegraded":"2024-09-10T16:03:45.309254+0000","last_fullsized":"2024-09-10T16:03:45.309254+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28278,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.308235+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:45.308235+0000","last_peered":"2024-09-10T16:03:45.308235+0000","last_clean":"2024-09-10T16:03:45.308235+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:45.308235+0000","last_undegraded":"2024-09-10T16:03:45.308235+0000","last_fullsized":"2024-09-10T16:03:45.308235+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.311628+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:45.311628+0000","last_peered":"2024-09-10T16:03:45.311628+0000","last_clean":"2024-09-10T16:03:45.311628+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:45.311628+0000","last_undegraded":"2024-09-10T16:03:45.311628+0000","last_fullsized":"2024-09-10T16:03:45.311628+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":22208,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.738429+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:03:46.738429+0000","last_peered":"2024-09-10T16:03:46.738429+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:46.738429+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3242}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309439+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309439+0000","last_peered":"2024-09-10T16:03:46.309439+0000","last_clean":"2024-09-10T16:03:46.309439+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309439+0000","last_undegraded":"2024-09-10T16:03:46.309439+0000","last_fullsized":"2024-09-10T16:03:46.309439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29033,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190275584,"data_stored":190228615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626563,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888488,"statfs":{"total":96636764160,"available":96141811712,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149844,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527056,"kb_used_data":62272,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844784,"statfs":{"total":96636764160,"available":96097058816,"internally_reserved":0,"allocated":63766528,"data_stored":63442413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953615,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481288,"kb_used_data":124792,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890552,"statfs":{"total":96636764160,"available":96143925248,"internally_reserved":0,"allocated":127787008,"data_stored":127466970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126943232,"data_stored":126939619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:50.023 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:50.317 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:50.317 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:50.330 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-09-10T16:03:48.657443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29035,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491696,"kb_used_data":187584,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623824,"statfs":{"total":289910292480,"available":288382795776,"internally_reserved":0,"allocated":192086016,"data_stored":191076260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2058,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001303"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21206,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.311689+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:45.311689+0000","last_peered":"2024-09-10T16:03:45.311689+0000","last_clean":"2024-09-10T16:03:45.311689+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:45.311689+0000","last_undegraded":"2024-09-10T16:03:45.311689+0000","last_fullsized":"2024-09-10T16:03:45.311689+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20787,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.309531+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:45.309531+0000","last_peered":"2024-09-10T16:03:45.309531+0000","last_clean":"2024-09-10T16:03:45.309531+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:45.309531+0000","last_undegraded":"2024-09-10T16:03:45.309531+0000","last_fullsized":"2024-09-10T16:03:45.309531+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21072,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309134+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309134+0000","last_peered":"2024-09-10T16:03:46.309134+0000","last_clean":"2024-09-10T16:03:46.309134+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309134+0000","last_undegraded":"2024-09-10T16:03:46.309134+0000","last_fullsized":"2024-09-10T16:03:46.309134+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.309165+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:45.309165+0000","last_peered":"2024-09-10T16:03:45.309165+0000","last_clean":"2024-09-10T16:03:45.309165+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:45.309165+0000","last_undegraded":"2024-09-10T16:03:45.309165+0000","last_fullsized":"2024-09-10T16:03:45.309165+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28610,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.309254+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:45.309254+0000","last_peered":"2024-09-10T16:03:45.309254+0000","last_clean":"2024-09-10T16:03:45.309254+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:45.309254+0000","last_undegraded":"2024-09-10T16:03:45.309254+0000","last_fullsized":"2024-09-10T16:03:45.309254+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28278,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.308235+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:45.308235+0000","last_peered":"2024-09-10T16:03:45.308235+0000","last_clean":"2024-09-10T16:03:45.308235+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:45.308235+0000","last_undegraded":"2024-09-10T16:03:45.308235+0000","last_fullsized":"2024-09-10T16:03:45.308235+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.311628+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:45.311628+0000","last_peered":"2024-09-10T16:03:45.311628+0000","last_clean":"2024-09-10T16:03:45.311628+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:45.311628+0000","last_undegraded":"2024-09-10T16:03:45.311628+0000","last_fullsized":"2024-09-10T16:03:45.311628+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":22208,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.738429+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:03:46.738429+0000","last_peered":"2024-09-10T16:03:46.738429+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:46.738429+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3242}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309439+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309439+0000","last_peered":"2024-09-10T16:03:46.309439+0000","last_clean":"2024-09-10T16:03:46.309439+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309439+0000","last_undegraded":"2024-09-10T16:03:46.309439+0000","last_fullsized":"2024-09-10T16:03:46.309439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29033,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190275584,"data_stored":190228615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626563,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888488,"statfs":{"total":96636764160,"available":96141811712,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149844,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527056,"kb_used_data":62272,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844784,"statfs":{"total":96636764160,"available":96097058816,"internally_reserved":0,"allocated":63766528,"data_stored":63442413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953615,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481288,"kb_used_data":124792,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890552,"statfs":{"total":96636764160,"available":96143925248,"internally_reserved":0,"allocated":127787008,"data_stored":127466970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126943232,"data_stored":126939619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:50.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:50.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21074, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.309378+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:46.309378+0000', 'last_peered': '2024-09-10T16:03:46.309378+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:46.309378+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:50.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:50.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21420, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.309475+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:46.309475+0000', 'last_peered': '2024-09-10T16:03:46.309475+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:46.309475+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:50.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:50.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21491, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.310055+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:46.310055+0000', 'last_peered': '2024-09-10T16:03:46.310055+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:46.310055+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:50.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:50.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 22208, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.738429+0000', 'last_change': '2024-09-10T16:03:44.544788+0000', 'last_active': '2024-09-10T16:03:46.738429+0000', 'last_peered': '2024-09-10T16:03:46.738429+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:03:46.738429+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3242, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 638, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3242}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:50.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:50.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21882, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.311632+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:46.311632+0000', 'last_peered': '2024-09-10T16:03:46.311632+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:46.311632+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:50.332 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:50.714 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:50.714 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17615/6 objects degraded (293583.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":756,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1527496704,"bytes_avail":288382795776,"bytes_total":289910292480,"degraded_objects":17615,"degraded_total":6,"degraded_ratio":2935.8333333333339,"recovering_objects_per_sec":257,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2058,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:50.727 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:50.727 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:51.052 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:51.052 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:51.065 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-10T16:03:50.657812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29035,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491696,"kb_used_data":187584,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281623824,"statfs":{"total":289910292480,"available":288382795776,"internally_reserved":0,"allocated":192086016,"data_stored":191076260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1158,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001366"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21206,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.311689+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:45.311689+0000","last_peered":"2024-09-10T16:03:45.311689+0000","last_clean":"2024-09-10T16:03:45.311689+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:45.311689+0000","last_undegraded":"2024-09-10T16:03:45.311689+0000","last_fullsized":"2024-09-10T16:03:45.311689+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20787,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.309531+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:45.309531+0000","last_peered":"2024-09-10T16:03:45.309531+0000","last_clean":"2024-09-10T16:03:45.309531+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:45.309531+0000","last_undegraded":"2024-09-10T16:03:45.309531+0000","last_fullsized":"2024-09-10T16:03:45.309531+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21072,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309134+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309134+0000","last_peered":"2024-09-10T16:03:46.309134+0000","last_clean":"2024-09-10T16:03:46.309134+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309134+0000","last_undegraded":"2024-09-10T16:03:46.309134+0000","last_fullsized":"2024-09-10T16:03:46.309134+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.309165+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:45.309165+0000","last_peered":"2024-09-10T16:03:45.309165+0000","last_clean":"2024-09-10T16:03:45.309165+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:45.309165+0000","last_undegraded":"2024-09-10T16:03:45.309165+0000","last_fullsized":"2024-09-10T16:03:45.309165+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28610,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.309254+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:45.309254+0000","last_peered":"2024-09-10T16:03:45.309254+0000","last_clean":"2024-09-10T16:03:45.309254+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:45.309254+0000","last_undegraded":"2024-09-10T16:03:45.309254+0000","last_fullsized":"2024-09-10T16:03:45.309254+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28278,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.308235+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:45.308235+0000","last_peered":"2024-09-10T16:03:45.308235+0000","last_clean":"2024-09-10T16:03:45.308235+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:45.308235+0000","last_undegraded":"2024-09-10T16:03:45.308235+0000","last_fullsized":"2024-09-10T16:03:45.308235+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T16:03:45.311628+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:45.311628+0000","last_peered":"2024-09-10T16:03:45.311628+0000","last_clean":"2024-09-10T16:03:45.311628+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:45.311628+0000","last_undegraded":"2024-09-10T16:03:45.311628+0000","last_fullsized":"2024-09-10T16:03:45.311628+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":22208,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.738429+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:03:46.738429+0000","last_peered":"2024-09-10T16:03:46.738429+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:46.738429+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3242}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309439+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309439+0000","last_peered":"2024-09-10T16:03:46.309439+0000","last_clean":"2024-09-10T16:03:46.309439+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309439+0000","last_undegraded":"2024-09-10T16:03:46.309439+0000","last_fullsized":"2024-09-10T16:03:46.309439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29033,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190275584,"data_stored":190228615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626563,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888488,"statfs":{"total":96636764160,"available":96141811712,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149844,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527056,"kb_used_data":62272,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844784,"statfs":{"total":96636764160,"available":96097058816,"internally_reserved":0,"allocated":63766528,"data_stored":63442413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953615,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481288,"kb_used_data":124792,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93890552,"statfs":{"total":96636764160,"available":96143925248,"internally_reserved":0,"allocated":127787008,"data_stored":127466970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126943232,"data_stored":126939619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:54.067 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:54.374 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:54.374 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:54.388 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-10T16:03:52.658161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29639,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494072,"kb_used_data":185800,"kb_used_omap":12,"kb_used_meta":1308211,"kb_avail":281621448,"statfs":{"total":289910292480,"available":288380362752,"internally_reserved":0,"allocated":190259200,"data_stored":189238728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1339608069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001352"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21208,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314272+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314272+0000","last_peered":"2024-09-10T16:03:46.314272+0000","last_clean":"2024-09-10T16:03:46.314272+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314272+0000","last_undegraded":"2024-09-10T16:03:46.314272+0000","last_fullsized":"2024-09-10T16:03:46.314272+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21072,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309134+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309134+0000","last_peered":"2024-09-10T16:03:46.309134+0000","last_clean":"2024-09-10T16:03:46.309134+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309134+0000","last_undegraded":"2024-09-10T16:03:46.309134+0000","last_fullsized":"2024-09-10T16:03:46.309134+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":23416,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:51.736371+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:03:51.736371+0000","last_peered":"2024-09-10T16:03:51.736371+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:51.736371+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1242,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309439+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309439+0000","last_peered":"2024-09-10T16:03:46.309439+0000","last_clean":"2024-09-10T16:03:46.309439+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309439+0000","last_undegraded":"2024-09-10T16:03:46.309439+0000","last_fullsized":"2024-09-10T16:03:46.309439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29637,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188432384,"data_stored":188385415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626564,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487512,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884328,"statfs":{"total":96636764160,"available":96137551872,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498658990},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149845,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953616,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479496,"kb_used_data":123000,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93892344,"statfs":{"total":96636764160,"available":96145760256,"internally_reserved":0,"allocated":125952000,"data_stored":125626604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":125100032,"data_stored":125096419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:54.390 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:54.702 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:54.702 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:54.714 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-10T16:03:54.658525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29639,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494072,"kb_used_data":185800,"kb_used_omap":12,"kb_used_meta":1308211,"kb_avail":281621448,"statfs":{"total":289910292480,"available":288380362752,"internally_reserved":0,"allocated":190259200,"data_stored":189238728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1339608069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.352690"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21208,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314272+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314272+0000","last_peered":"2024-09-10T16:03:46.314272+0000","last_clean":"2024-09-10T16:03:46.314272+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314272+0000","last_undegraded":"2024-09-10T16:03:46.314272+0000","last_fullsized":"2024-09-10T16:03:46.314272+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21072,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309134+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309134+0000","last_peered":"2024-09-10T16:03:46.309134+0000","last_clean":"2024-09-10T16:03:46.309134+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309134+0000","last_undegraded":"2024-09-10T16:03:46.309134+0000","last_fullsized":"2024-09-10T16:03:46.309134+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":23416,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:51.736371+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:03:51.736371+0000","last_peered":"2024-09-10T16:03:51.736371+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:51.736371+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1242,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309439+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309439+0000","last_peered":"2024-09-10T16:03:46.309439+0000","last_clean":"2024-09-10T16:03:46.309439+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309439+0000","last_undegraded":"2024-09-10T16:03:46.309439+0000","last_fullsized":"2024-09-10T16:03:46.309439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29637,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188432384,"data_stored":188385415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626564,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487512,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884328,"statfs":{"total":96636764160,"available":96137551872,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498658990},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149845,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953616,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479496,"kb_used_data":123000,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93892344,"statfs":{"total":96636764160,"available":96145760256,"internally_reserved":0,"allocated":125952000,"data_stored":125626604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":125100032,"data_stored":125096419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:54.715 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:54.715 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21074, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.309378+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:46.309378+0000', 'last_peered': '2024-09-10T16:03:46.309378+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:46.309378+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.715 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:54.716 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21420, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.309475+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:46.309475+0000', 'last_peered': '2024-09-10T16:03:46.309475+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:46.309475+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.716 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:54.716 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21491, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.310055+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:46.310055+0000', 'last_peered': '2024-09-10T16:03:46.310055+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:46.310055+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.716 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:54.716 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 23416, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:51.736371+0000', 'last_change': '2024-09-10T16:03:44.544788+0000', 'last_active': '2024-09-10T16:03:51.736371+0000', 'last_peered': '2024-09-10T16:03:51.736371+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:03:51.736371+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2638, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1242, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2638}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.716 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:54.716 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21882, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.311632+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:46.311632+0000', 'last_peered': '2024-09-10T16:03:46.311632+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:46.311632+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:54.716 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:55.102 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:55.103 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17011/6 objects degraded (283516.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":761,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1529929728,"bytes_avail":288380362752,"bytes_total":289910292480,"degraded_objects":17011,"degraded_total":6,"degraded_ratio":2835.1666666666661,"recovering_objects_per_sec":220,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:55.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:55.115 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:55.449 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:55.449 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:55.463 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-10T16:03:54.658525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29639,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494072,"kb_used_data":185800,"kb_used_omap":12,"kb_used_meta":1308211,"kb_avail":281621448,"statfs":{"total":289910292480,"available":288380362752,"internally_reserved":0,"allocated":190259200,"data_stored":189238728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1339608069},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.352690"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21208,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314272+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314272+0000","last_peered":"2024-09-10T16:03:46.314272+0000","last_clean":"2024-09-10T16:03:46.314272+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314272+0000","last_undegraded":"2024-09-10T16:03:46.314272+0000","last_fullsized":"2024-09-10T16:03:46.314272+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21072,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309134+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309134+0000","last_peered":"2024-09-10T16:03:46.309134+0000","last_clean":"2024-09-10T16:03:46.309134+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309134+0000","last_undegraded":"2024-09-10T16:03:46.309134+0000","last_fullsized":"2024-09-10T16:03:46.309134+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":23416,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:51.736371+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:03:51.736371+0000","last_peered":"2024-09-10T16:03:51.736371+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:51.736371+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1242,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":95,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309439+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309439+0000","last_peered":"2024-09-10T16:03:46.309439+0000","last_clean":"2024-09-10T16:03:46.309439+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309439+0000","last_undegraded":"2024-09-10T16:03:46.309439+0000","last_fullsized":"2024-09-10T16:03:46.309439+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29637,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188432384,"data_stored":188385415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626564,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487512,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884328,"statfs":{"total":96636764160,"available":96137551872,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498658990},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149845,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953616,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479496,"kb_used_data":123000,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93892344,"statfs":{"total":96636764160,"available":96145760256,"internally_reserved":0,"allocated":125952000,"data_stored":125626604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":125100032,"data_stored":125096419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:58.465 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:58.804 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:58.804 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:58.816 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-09-10T16:03:58.659138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30328,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483712,"kb_used_data":183760,"kb_used_omap":12,"kb_used_meta":1299891,"kb_avail":281631808,"statfs":{"total":289910292480,"available":288390971392,"internally_reserved":0,"allocated":188170240,"data_stored":187149768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1331088389},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2451,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2451,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001992"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21208,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314272+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314272+0000","last_peered":"2024-09-10T16:03:46.314272+0000","last_clean":"2024-09-10T16:03:46.314272+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314272+0000","last_undegraded":"2024-09-10T16:03:46.314272+0000","last_fullsized":"2024-09-10T16:03:46.314272+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21072,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309134+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309134+0000","last_peered":"2024-09-10T16:03:46.309134+0000","last_clean":"2024-09-10T16:03:46.309134+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309134+0000","last_undegraded":"2024-09-10T16:03:46.309134+0000","last_fullsized":"2024-09-10T16:03:46.309134+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":24794,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:56.741118+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:03:56.741118+0000","last_peered":"2024-09-10T16:03:56.741118+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:56.741118+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1931,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309469+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309469+0000","last_peered":"2024-09-10T16:03:46.309469+0000","last_clean":"2024-09-10T16:03:46.309469+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309469+0000","last_undegraded":"2024-09-10T16:03:46.309469+0000","last_fullsized":"2024-09-10T16:03:46.309469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30326,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":186589184,"data_stored":186542215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626565,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479192,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892648,"statfs":{"total":96636764160,"available":96146071552,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490139310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149846,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953617,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477456,"kb_used_data":120960,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93894384,"statfs":{"total":96636764160,"available":96147849216,"internally_reserved":0,"allocated":123863040,"data_stored":123537644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":123256832,"data_stored":123253219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:58.817 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:59.119 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:59.119 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:59.131 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-09-10T16:03:58.659138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30328,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483712,"kb_used_data":183760,"kb_used_omap":12,"kb_used_meta":1299891,"kb_avail":281631808,"statfs":{"total":289910292480,"available":288390971392,"internally_reserved":0,"allocated":188170240,"data_stored":187149768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1331088389},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2451,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2451,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001992"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21208,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314272+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314272+0000","last_peered":"2024-09-10T16:03:46.314272+0000","last_clean":"2024-09-10T16:03:46.314272+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314272+0000","last_undegraded":"2024-09-10T16:03:46.314272+0000","last_fullsized":"2024-09-10T16:03:46.314272+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21072,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309134+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309134+0000","last_peered":"2024-09-10T16:03:46.309134+0000","last_clean":"2024-09-10T16:03:46.309134+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309134+0000","last_undegraded":"2024-09-10T16:03:46.309134+0000","last_fullsized":"2024-09-10T16:03:46.309134+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":24794,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:56.741118+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:03:56.741118+0000","last_peered":"2024-09-10T16:03:56.741118+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:56.741118+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1931,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309469+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309469+0000","last_peered":"2024-09-10T16:03:46.309469+0000","last_clean":"2024-09-10T16:03:46.309469+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309469+0000","last_undegraded":"2024-09-10T16:03:46.309469+0000","last_fullsized":"2024-09-10T16:03:46.309469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30326,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":186589184,"data_stored":186542215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626565,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479192,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892648,"statfs":{"total":96636764160,"available":96146071552,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490139310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149846,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953617,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477456,"kb_used_data":120960,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93894384,"statfs":{"total":96636764160,"available":96147849216,"internally_reserved":0,"allocated":123863040,"data_stored":123537644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":123256832,"data_stored":123253219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:03:59.132 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:03:59.132 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21074, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.309378+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:46.309378+0000', 'last_peered': '2024-09-10T16:03:46.309378+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:46.309378+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:59.132 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:03:59.132 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21420, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.309475+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:46.309475+0000', 'last_peered': '2024-09-10T16:03:46.309475+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:46.309475+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:59.133 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:03:59.133 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21491, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.310055+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:46.310055+0000', 'last_peered': '2024-09-10T16:03:46.310055+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:46.310055+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:59.133 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:03:59.133 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 24794, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:56.741118+0000', 'last_change': '2024-09-10T16:03:44.544788+0000', 'last_active': '2024-09-10T16:03:56.741118+0000', 'last_peered': '2024-09-10T16:03:56.741118+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:03:56.741118+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1949, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1931, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1949}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:59.133 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:03:59.133 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21882, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.311632+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:46.311632+0000', 'last_peered': '2024-09-10T16:03:46.311632+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:46.311632+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:03:59.133 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:03:59.526 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:59.527 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16322/6 objects degraded (272033.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":765,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1519321088,"bytes_avail":288390971392,"bytes_total":289910292480,"degraded_objects":16322,"degraded_total":6,"degraded_ratio":2720.3333333333339,"recovering_objects_per_sec":204,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2451,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:03:59.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:03:59.540 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:03:59.842 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:03:59.842 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:03:59.854 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-09-10T16:03:58.659138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30328,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483712,"kb_used_data":183760,"kb_used_omap":12,"kb_used_meta":1299891,"kb_avail":281631808,"statfs":{"total":289910292480,"available":288390971392,"internally_reserved":0,"allocated":188170240,"data_stored":187149768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1331088389},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2451,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2451,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001992"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21208,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314272+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314272+0000","last_peered":"2024-09-10T16:03:46.314272+0000","last_clean":"2024-09-10T16:03:46.314272+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314272+0000","last_undegraded":"2024-09-10T16:03:46.314272+0000","last_fullsized":"2024-09-10T16:03:46.314272+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21072,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309134+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309134+0000","last_peered":"2024-09-10T16:03:46.309134+0000","last_clean":"2024-09-10T16:03:46.309134+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309134+0000","last_undegraded":"2024-09-10T16:03:46.309134+0000","last_fullsized":"2024-09-10T16:03:46.309134+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":24794,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:56.741118+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:03:56.741118+0000","last_peered":"2024-09-10T16:03:56.741118+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:03:56.741118+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1931,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309469+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309469+0000","last_peered":"2024-09-10T16:03:46.309469+0000","last_clean":"2024-09-10T16:03:46.309469+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309469+0000","last_undegraded":"2024-09-10T16:03:46.309469+0000","last_fullsized":"2024-09-10T16:03:46.309469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30326,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":186589184,"data_stored":186542215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626565,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479192,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892648,"statfs":{"total":96636764160,"available":96146071552,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490139310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149846,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953617,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477456,"kb_used_data":120960,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93894384,"statfs":{"total":96636764160,"available":96147849216,"internally_reserved":0,"allocated":123863040,"data_stored":123537644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":123256832,"data_stored":123253219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:02.857 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:03.154 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:03.155 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:03.166 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-09-10T16:04:02.659871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31109,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485952,"kb_used_data":181840,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281629568,"statfs":{"total":289910292480,"available":288388677632,"internally_reserved":0,"allocated":186204160,"data_stored":185183688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2074,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002059"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21208,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314272+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314272+0000","last_peered":"2024-09-10T16:03:46.314272+0000","last_clean":"2024-09-10T16:03:46.314272+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314272+0000","last_undegraded":"2024-09-10T16:03:46.314272+0000","last_fullsized":"2024-09-10T16:03:46.314272+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21073,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309184+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309184+0000","last_peered":"2024-09-10T16:03:46.309184+0000","last_clean":"2024-09-10T16:03:46.309184+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309184+0000","last_undegraded":"2024-09-10T16:03:46.309184+0000","last_fullsized":"2024-09-10T16:03:46.309184+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":26356,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:01.743402+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:04:01.743402+0000","last_peered":"2024-09-10T16:04:01.743402+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:04:01.743402+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2712,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309469+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309469+0000","last_peered":"2024-09-10T16:03:46.309469+0000","last_clean":"2024-09-10T16:03:46.309469+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309469+0000","last_undegraded":"2024-09-10T16:03:46.309469+0000","last_fullsized":"2024-09-10T16:03:46.309469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31107,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184745984,"data_stored":184699015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888488,"statfs":{"total":96636764160,"available":96141811712,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953618,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475536,"kb_used_data":119040,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93896304,"statfs":{"total":96636764160,"available":96149815296,"internally_reserved":0,"allocated":121896960,"data_stored":121571564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":121413632,"data_stored":121410019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:03.168 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:03.480 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:03.481 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:03.492 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-09-10T16:04:02.659871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31109,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485952,"kb_used_data":181840,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281629568,"statfs":{"total":289910292480,"available":288388677632,"internally_reserved":0,"allocated":186204160,"data_stored":185183688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2074,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002059"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21208,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314272+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314272+0000","last_peered":"2024-09-10T16:03:46.314272+0000","last_clean":"2024-09-10T16:03:46.314272+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314272+0000","last_undegraded":"2024-09-10T16:03:46.314272+0000","last_fullsized":"2024-09-10T16:03:46.314272+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21073,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309184+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309184+0000","last_peered":"2024-09-10T16:03:46.309184+0000","last_clean":"2024-09-10T16:03:46.309184+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309184+0000","last_undegraded":"2024-09-10T16:03:46.309184+0000","last_fullsized":"2024-09-10T16:03:46.309184+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":26356,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:01.743402+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:04:01.743402+0000","last_peered":"2024-09-10T16:04:01.743402+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:04:01.743402+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2712,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309469+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309469+0000","last_peered":"2024-09-10T16:03:46.309469+0000","last_clean":"2024-09-10T16:03:46.309469+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309469+0000","last_undegraded":"2024-09-10T16:03:46.309469+0000","last_fullsized":"2024-09-10T16:03:46.309469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31107,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184745984,"data_stored":184699015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888488,"statfs":{"total":96636764160,"available":96141811712,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953618,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475536,"kb_used_data":119040,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93896304,"statfs":{"total":96636764160,"available":96149815296,"internally_reserved":0,"allocated":121896960,"data_stored":121571564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":121413632,"data_stored":121410019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:03.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:03.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21074, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.309378+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:46.309378+0000', 'last_peered': '2024-09-10T16:03:46.309378+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:46.309378+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:03.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:03.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21420, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.309475+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:46.309475+0000', 'last_peered': '2024-09-10T16:03:46.309475+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:46.309475+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:03.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:03.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21491, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.310055+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:46.310055+0000', 'last_peered': '2024-09-10T16:03:46.310055+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:46.310055+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:03.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:03.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 26356, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:01.743402+0000', 'last_change': '2024-09-10T16:03:44.544788+0000', 'last_active': '2024-09-10T16:04:01.743402+0000', 'last_peered': '2024-09-10T16:04:01.743402+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:04:01.743402+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1168, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2712, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1168}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:03.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:03.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21882, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.311632+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:46.311632+0000', 'last_peered': '2024-09-10T16:03:46.311632+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:46.311632+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:03.495 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:03.902 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:03.903 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15541/6 objects degraded (259016.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":769,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1521614848,"bytes_avail":288388677632,"bytes_total":289910292480,"degraded_objects":15541,"degraded_total":6,"degraded_ratio":2590.1666666666665,"recovering_objects_per_sec":172,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2074,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:03.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:03.916 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:04.213 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:04.214 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:04.228 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-09-10T16:04:02.659871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31109,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485952,"kb_used_data":181840,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281629568,"statfs":{"total":289910292480,"available":288388677632,"internally_reserved":0,"allocated":186204160,"data_stored":185183688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2074,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002059"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21208,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314272+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314272+0000","last_peered":"2024-09-10T16:03:46.314272+0000","last_clean":"2024-09-10T16:03:46.314272+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314272+0000","last_undegraded":"2024-09-10T16:03:46.314272+0000","last_fullsized":"2024-09-10T16:03:46.314272+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21073,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309184+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309184+0000","last_peered":"2024-09-10T16:03:46.309184+0000","last_clean":"2024-09-10T16:03:46.309184+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309184+0000","last_undegraded":"2024-09-10T16:03:46.309184+0000","last_fullsized":"2024-09-10T16:03:46.309184+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":26356,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:01.743402+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:04:01.743402+0000","last_peered":"2024-09-10T16:04:01.743402+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:04:01.743402+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2712,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309469+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309469+0000","last_peered":"2024-09-10T16:03:46.309469+0000","last_clean":"2024-09-10T16:03:46.309469+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309469+0000","last_undegraded":"2024-09-10T16:03:46.309469+0000","last_fullsized":"2024-09-10T16:03:46.309469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31107,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184745984,"data_stored":184699015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888488,"statfs":{"total":96636764160,"available":96141811712,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953618,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475536,"kb_used_data":119040,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93896304,"statfs":{"total":96636764160,"available":96149815296,"internally_reserved":0,"allocated":121896960,"data_stored":121571564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":121413632,"data_stored":121410019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:07.231 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:07.553 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:07.553 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:07.567 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-09-10T16:04:06.660515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31109,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484152,"kb_used_data":180040,"kb_used_omap":12,"kb_used_meta":1304051,"kb_avail":281631368,"statfs":{"total":289910292480,"available":288390520832,"internally_reserved":0,"allocated":184360960,"data_stored":183340488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1335348229},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1470,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001990"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21209,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314315+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314315+0000","last_peered":"2024-09-10T16:03:46.314315+0000","last_clean":"2024-09-10T16:03:46.314315+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314315+0000","last_undegraded":"2024-09-10T16:03:46.314315+0000","last_fullsized":"2024-09-10T16:03:46.314315+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21073,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309184+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309184+0000","last_peered":"2024-09-10T16:03:46.309184+0000","last_clean":"2024-09-10T16:03:46.309184+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309184+0000","last_undegraded":"2024-09-10T16:03:46.309184+0000","last_fullsized":"2024-09-10T16:03:46.309184+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":26356,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:01.743402+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:04:01.743402+0000","last_peered":"2024-09-10T16:04:01.743402+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:04:01.743402+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2712,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309469+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309469+0000","last_peered":"2024-09-10T16:03:46.309469+0000","last_clean":"2024-09-10T16:03:46.309469+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309469+0000","last_undegraded":"2024-09-10T16:03:46.309469+0000","last_fullsized":"2024-09-10T16:03:46.309469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309570+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309570+0000","last_peered":"2024-09-10T16:03:46.309570+0000","last_clean":"2024-09-10T16:03:46.309570+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309570+0000","last_undegraded":"2024-09-10T16:03:46.309570+0000","last_fullsized":"2024-09-10T16:03:46.309570+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31107,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182902784,"data_stored":182855815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888488,"statfs":{"total":96636764160,"available":96141811712,"internally_reserved":0,"allocated":532480,"data_stored":166877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494399150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953619,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473736,"kb_used_data":117240,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93898104,"statfs":{"total":96636764160,"available":96151658496,"internally_reserved":0,"allocated":120053760,"data_stored":119728364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119570432,"data_stored":119566819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:07.568 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:07.887 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:07.888 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:07.900 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-10T16:04:07.785414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32210,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514672,"kb_used_data":180032,"kb_used_omap":12,"kb_used_meta":1334579,"kb_avail":281600848,"statfs":{"total":289910292480,"available":288359268352,"internally_reserved":0,"allocated":184352768,"data_stored":183339522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1366608901},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2571,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.126599"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21209,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314315+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314315+0000","last_peered":"2024-09-10T16:03:46.314315+0000","last_clean":"2024-09-10T16:03:46.314315+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314315+0000","last_undegraded":"2024-09-10T16:03:46.314315+0000","last_fullsized":"2024-09-10T16:03:46.314315+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21073,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309184+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309184+0000","last_peered":"2024-09-10T16:03:46.309184+0000","last_clean":"2024-09-10T16:03:46.309184+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309184+0000","last_undegraded":"2024-09-10T16:03:46.309184+0000","last_fullsized":"2024-09-10T16:03:46.309184+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":28558,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:06.740961+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:04:06.740961+0000","last_peered":"2024-09-10T16:04:06.740961+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:04:06.740961+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":67,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3813,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-67}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309469+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309469+0000","last_peered":"2024-09-10T16:03:46.309469+0000","last_clean":"2024-09-10T16:03:46.309469+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309469+0000","last_undegraded":"2024-09-10T16:03:46.309469+0000","last_fullsized":"2024-09-10T16:03:46.309469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309600+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309600+0000","last_peered":"2024-09-10T16:03:46.309600+0000","last_clean":"2024-09-10T16:03:46.309600+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309600+0000","last_undegraded":"2024-09-10T16:03:46.309600+0000","last_fullsized":"2024-09-10T16:03:46.309600+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32208,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182894592,"data_stored":182854849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513872,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":513339,"kb_avail":93857968,"statfs":{"total":96636764160,"available":96110559232,"internally_reserved":0,"allocated":524288,"data_stored":165911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":525659822},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149848,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953619,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473736,"kb_used_data":117240,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93898104,"statfs":{"total":96636764160,"available":96151658496,"internally_reserved":0,"allocated":120053760,"data_stored":119728364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119570432,"data_stored":119566819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:07.901 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:07.902 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21074, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.309378+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:46.309378+0000', 'last_peered': '2024-09-10T16:03:46.309378+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:46.309378+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:07.902 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:07.902 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21420, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.309475+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:46.309475+0000', 'last_peered': '2024-09-10T16:03:46.309475+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:46.309475+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:07.902 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:07.902 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21491, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.310055+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:46.310055+0000', 'last_peered': '2024-09-10T16:03:46.310055+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:46.310055+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:07.902 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:07.902 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 28558, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:06.740961+0000', 'last_change': '2024-09-10T16:03:44.544788+0000', 'last_active': '2024-09-10T16:04:06.740961+0000', 'last_peered': '2024-09-10T16:04:06.740961+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:04:06.740961+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 67, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3813, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -67}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:07.902 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:07.902 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21882, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.311632+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:46.311632+0000', 'last_peered': '2024-09-10T16:03:46.311632+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:46.311632+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:07.903 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:08.411 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:08.412 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15541/6 objects degraded (259016.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":774,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1519771648,"bytes_avail":288390520832,"bytes_total":289910292480,"degraded_objects":15541,"degraded_total":6,"degraded_ratio":2590.1666666666665,"recovering_objects_per_sec":122,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1470,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:08.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:08.426 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:08.838 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:08.838 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:08.855 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-10T16:04:08.802486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32210,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514672,"kb_used_data":180032,"kb_used_omap":12,"kb_used_meta":1334579,"kb_avail":281600848,"statfs":{"total":289910292480,"available":288359268352,"internally_reserved":0,"allocated":184352768,"data_stored":183339522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1366608901},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1882,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1882,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.142916"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21209,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314315+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314315+0000","last_peered":"2024-09-10T16:03:46.314315+0000","last_clean":"2024-09-10T16:03:46.314315+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314315+0000","last_undegraded":"2024-09-10T16:03:46.314315+0000","last_fullsized":"2024-09-10T16:03:46.314315+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21073,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309184+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309184+0000","last_peered":"2024-09-10T16:03:46.309184+0000","last_clean":"2024-09-10T16:03:46.309184+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309184+0000","last_undegraded":"2024-09-10T16:03:46.309184+0000","last_fullsized":"2024-09-10T16:03:46.309184+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":28558,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:06.740961+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:04:06.740961+0000","last_peered":"2024-09-10T16:04:06.740961+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:04:06.740961+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":67,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3813,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-67}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309469+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309469+0000","last_peered":"2024-09-10T16:03:46.309469+0000","last_clean":"2024-09-10T16:03:46.309469+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309469+0000","last_undegraded":"2024-09-10T16:03:46.309469+0000","last_fullsized":"2024-09-10T16:03:46.309469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309600+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309600+0000","last_peered":"2024-09-10T16:03:46.309600+0000","last_clean":"2024-09-10T16:03:46.309600+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309600+0000","last_undegraded":"2024-09-10T16:03:46.309600+0000","last_fullsized":"2024-09-10T16:03:46.309600+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32208,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182894592,"data_stored":182854849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513872,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":513339,"kb_avail":93857968,"statfs":{"total":96636764160,"available":96110559232,"internally_reserved":0,"allocated":524288,"data_stored":165911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":525659822},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149848,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953619,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473736,"kb_used_data":117240,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93898104,"statfs":{"total":96636764160,"available":96151658496,"internally_reserved":0,"allocated":120053760,"data_stored":119728364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119570432,"data_stored":119566819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:11.857 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:12.191 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:12.191 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:12.205 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-09-10T16:04:10.661185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32210,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514672,"kb_used_data":180032,"kb_used_omap":12,"kb_used_meta":1334579,"kb_avail":281600848,"statfs":{"total":289910292480,"available":288359268352,"internally_reserved":0,"allocated":184352768,"data_stored":183339522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1366608901},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1101,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1101,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001314"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21209,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314315+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314315+0000","last_peered":"2024-09-10T16:03:46.314315+0000","last_clean":"2024-09-10T16:03:46.314315+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314315+0000","last_undegraded":"2024-09-10T16:03:46.314315+0000","last_fullsized":"2024-09-10T16:03:46.314315+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21073,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309184+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309184+0000","last_peered":"2024-09-10T16:03:46.309184+0000","last_clean":"2024-09-10T16:03:46.309184+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309184+0000","last_undegraded":"2024-09-10T16:03:46.309184+0000","last_fullsized":"2024-09-10T16:03:46.309184+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":28558,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:06.740961+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:04:06.740961+0000","last_peered":"2024-09-10T16:04:06.740961+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:04:06.740961+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":67,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3813,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-67}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309469+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309469+0000","last_peered":"2024-09-10T16:03:46.309469+0000","last_clean":"2024-09-10T16:03:46.309469+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309469+0000","last_undegraded":"2024-09-10T16:03:46.309469+0000","last_fullsized":"2024-09-10T16:03:46.309469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309600+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309600+0000","last_peered":"2024-09-10T16:03:46.309600+0000","last_clean":"2024-09-10T16:03:46.309600+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309600+0000","last_undegraded":"2024-09-10T16:03:46.309600+0000","last_fullsized":"2024-09-10T16:03:46.309600+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32208,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182894592,"data_stored":182854849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513872,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":513339,"kb_avail":93857968,"statfs":{"total":96636764160,"available":96110559232,"internally_reserved":0,"allocated":524288,"data_stored":165911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":525659822},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149848,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953619,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473736,"kb_used_data":117240,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93898104,"statfs":{"total":96636764160,"available":96151658496,"internally_reserved":0,"allocated":120053760,"data_stored":119728364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119570432,"data_stored":119566819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:12.207 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:12.542 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:12.542 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:12.555 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-09-10T16:04:10.661185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32210,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514672,"kb_used_data":180032,"kb_used_omap":12,"kb_used_meta":1334579,"kb_avail":281600848,"statfs":{"total":289910292480,"available":288359268352,"internally_reserved":0,"allocated":184352768,"data_stored":183339522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1366608901},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1101,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1101,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001314"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28828,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.311684+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:03:46.311684+0000","last_peered":"2024-09-10T16:03:46.311684+0000","last_clean":"2024-09-10T16:03:46.311684+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:03:46.311684+0000","last_undegraded":"2024-09-10T16:03:46.311684+0000","last_fullsized":"2024-09-10T16:03:46.311684+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21209,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314315+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:03:46.314315+0000","last_peered":"2024-09-10T16:03:46.314315+0000","last_clean":"2024-09-10T16:03:46.314315+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:03:46.314315+0000","last_undegraded":"2024-09-10T16:03:46.314315+0000","last_fullsized":"2024-09-10T16:03:46.314315+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21124,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.408136+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:03:46.408136+0000","last_peered":"2024-09-10T16:03:46.408136+0000","last_clean":"2024-09-10T16:03:46.408136+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:03:46.408136+0000","last_undegraded":"2024-09-10T16:03:46.408136+0000","last_fullsized":"2024-09-10T16:03:46.408136+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20789,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314173+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:03:46.314173+0000","last_peered":"2024-09-10T16:03:46.314173+0000","last_clean":"2024-09-10T16:03:46.314173+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:03:46.314173+0000","last_undegraded":"2024-09-10T16:03:46.314173+0000","last_fullsized":"2024-09-10T16:03:46.314173+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20485,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309851+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:03:46.309851+0000","last_peered":"2024-09-10T16:03:46.309851+0000","last_clean":"2024-09-10T16:03:46.309851+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:03:46.309851+0000","last_undegraded":"2024-09-10T16:03:46.309851+0000","last_fullsized":"2024-09-10T16:03:46.309851+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28782,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308999+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:03:46.308999+0000","last_peered":"2024-09-10T16:03:46.308999+0000","last_clean":"2024-09-10T16:03:46.308999+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:03:46.308999+0000","last_undegraded":"2024-09-10T16:03:46.308999+0000","last_fullsized":"2024-09-10T16:03:46.308999+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21074,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309378+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:03:46.309378+0000","last_peered":"2024-09-10T16:03:46.309378+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:03:46.309378+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21073,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309184+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:03:46.309184+0000","last_peered":"2024-09-10T16:03:46.309184+0000","last_clean":"2024-09-10T16:03:46.309184+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:03:46.309184+0000","last_undegraded":"2024-09-10T16:03:46.309184+0000","last_fullsized":"2024-09-10T16:03:46.309184+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.308670+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:03:46.308670+0000","last_peered":"2024-09-10T16:03:46.308670+0000","last_clean":"2024-09-10T16:03:46.308670+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:03:46.308670+0000","last_undegraded":"2024-09-10T16:03:46.308670+0000","last_fullsized":"2024-09-10T16:03:46.308670+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28874,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309255+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:03:46.309255+0000","last_peered":"2024-09-10T16:03:46.309255+0000","last_clean":"2024-09-10T16:03:46.309255+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:03:46.309255+0000","last_undegraded":"2024-09-10T16:03:46.309255+0000","last_fullsized":"2024-09-10T16:03:46.309255+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313585+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:03:46.313585+0000","last_peered":"2024-09-10T16:03:46.313585+0000","last_clean":"2024-09-10T16:03:46.313585+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:03:46.313585+0000","last_undegraded":"2024-09-10T16:03:46.313585+0000","last_fullsized":"2024-09-10T16:03:46.313585+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":21420,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.309475+0000","last_change":"2024-09-10T15:53:19.971748+0000","last_active":"2024-09-10T16:03:46.309475+0000","last_peered":"2024-09-10T16:03:46.309475+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:03:46.309475+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309316+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:03:46.309316+0000","last_peered":"2024-09-10T16:03:46.309316+0000","last_clean":"2024-09-10T16:03:46.309316+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:03:46.309316+0000","last_undegraded":"2024-09-10T16:03:46.309316+0000","last_fullsized":"2024-09-10T16:03:46.309316+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28612,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313834+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:03:46.313834+0000","last_peered":"2024-09-10T16:03:46.313834+0000","last_clean":"2024-09-10T16:03:46.313834+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:03:46.313834+0000","last_undegraded":"2024-09-10T16:03:46.313834+0000","last_fullsized":"2024-09-10T16:03:46.313834+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309180+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:03:46.309180+0000","last_peered":"2024-09-10T16:03:46.309180+0000","last_clean":"2024-09-10T16:03:46.309180+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:03:46.309180+0000","last_undegraded":"2024-09-10T16:03:46.309180+0000","last_fullsized":"2024-09-10T16:03:46.309180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28280,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.314376+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:03:46.314376+0000","last_peered":"2024-09-10T16:03:46.314376+0000","last_clean":"2024-09-10T16:03:46.314376+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:03:46.314376+0000","last_undegraded":"2024-09-10T16:03:46.314376+0000","last_fullsized":"2024-09-10T16:03:46.314376+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21718,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309639+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:03:46.309639+0000","last_peered":"2024-09-10T16:03:46.309639+0000","last_clean":"2024-09-10T16:03:46.309639+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:03:46.309639+0000","last_undegraded":"2024-09-10T16:03:46.309639+0000","last_fullsized":"2024-09-10T16:03:46.309639+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309389+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:03:46.309389+0000","last_peered":"2024-09-10T16:03:46.309389+0000","last_clean":"2024-09-10T16:03:46.309389+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:03:46.309389+0000","last_undegraded":"2024-09-10T16:03:46.309389+0000","last_fullsized":"2024-09-10T16:03:46.309389+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21491,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.310055+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:03:46.310055+0000","last_peered":"2024-09-10T16:03:46.310055+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:03:46.310055+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":121,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.313793+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:03:46.313793+0000","last_peered":"2024-09-10T16:03:46.313793+0000","last_clean":"2024-09-10T16:03:46.313793+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:03:46.313793+0000","last_undegraded":"2024-09-10T16:03:46.313793+0000","last_fullsized":"2024-09-10T16:03:46.313793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":28558,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:06.740961+0000","last_change":"2024-09-10T16:03:44.544788+0000","last_active":"2024-09-10T16:04:06.740961+0000","last_peered":"2024-09-10T16:04:06.740961+0000","last_clean":"2024-09-10T15:53:17.323548+0000","last_became_active":"2024-09-10T15:53:19.933952+0000","last_became_peered":"2024-09-10T15:53:19.933952+0000","last_unstale":"2024-09-10T16:04:06.740961+0000","last_undegraded":"2024-09-10T15:53:19.529631+0000","last_fullsized":"2024-09-10T15:53:19.468229+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":67,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3813,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-67}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309469+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:03:46.309469+0000","last_peered":"2024-09-10T16:03:46.309469+0000","last_clean":"2024-09-10T16:03:46.309469+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:03:46.309469+0000","last_undegraded":"2024-09-10T16:03:46.309469+0000","last_fullsized":"2024-09-10T16:03:46.309469+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T16:03:46.309600+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:03:46.309600+0000","last_peered":"2024-09-10T16:03:46.309600+0000","last_clean":"2024-09-10T16:03:46.309600+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:03:46.309600+0000","last_undegraded":"2024-09-10T16:03:46.309600+0000","last_fullsized":"2024-09-10T16:03:46.309600+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21882,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:03:46.311632+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:03:46.311632+0000","last_peered":"2024-09-10T16:03:46.311632+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:03:46.311632+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32208,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182894592,"data_stored":182854849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513872,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":513339,"kb_avail":93857968,"statfs":{"total":96636764160,"available":96110559232,"internally_reserved":0,"allocated":524288,"data_stored":165911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":525659822},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149848,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527064,"kb_used_data":62280,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844776,"statfs":{"total":96636764160,"available":96097050624,"internally_reserved":0,"allocated":63774720,"data_stored":63445247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953619,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473736,"kb_used_data":117240,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93898104,"statfs":{"total":96636764160,"available":96151658496,"internally_reserved":0,"allocated":120053760,"data_stored":119728364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119570432,"data_stored":119566819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:12.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:12.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21074, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.309378+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:03:46.309378+0000', 'last_peered': '2024-09-10T16:03:46.309378+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:03:46.309378+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:12.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:12.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 21420, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.309475+0000', 'last_change': '2024-09-10T15:53:19.971748+0000', 'last_active': '2024-09-10T16:03:46.309475+0000', 'last_peered': '2024-09-10T16:03:46.309475+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:03:46.309475+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3521, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:12.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:12.557 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21491, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.310055+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:03:46.310055+0000', 'last_peered': '2024-09-10T16:03:46.310055+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:03:46.310055+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:12.557 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-10T16:04:12.557 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'20766", 'reported_seq': 28558, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:06.740961+0000', 'last_change': '2024-09-10T16:03:44.544788+0000', 'last_active': '2024-09-10T16:04:06.740961+0000', 'last_peered': '2024-09-10T16:04:06.740961+0000', 'last_clean': '2024-09-10T15:53:17.323548+0000', 'last_became_active': '2024-09-10T15:53:19.933952+0000', 'last_became_peered': '2024-09-10T15:53:19.933952+0000', 'last_unstale': '2024-09-10T16:04:06.740961+0000', 'last_undegraded': '2024-09-10T15:53:19.529631+0000', 'last_fullsized': '2024-09-10T15:53:19.468229+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3433", 'last_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.687452+0000', 'objects_scrubbed': 3390, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1195, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 67, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17372, 'num_write_kb': 27964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3813, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -67}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:12.557 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:12.557 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21882, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:03:46.311632+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:03:46.311632+0000', 'last_peered': '2024-09-10T16:03:46.311632+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:03:46.311632+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:12.557 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:13.107 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:13.107 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13586/6 objects degraded (226433.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":779,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1536577536,"bytes_avail":288373714944,"bytes_total":289910292480,"degraded_objects":13586,"degraded_total":6,"degraded_ratio":2264.3333333333335,"recovering_objects_per_sec":244,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1955,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:13.122 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:13.122 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:13.454 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:13.454 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:13.467 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-09-10T16:04:12.661504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33064,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500564,"kb_used_data":178404,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281614956,"statfs":{"total":289910292480,"available":288373714944,"internally_reserved":0,"allocated":182685696,"data_stored":181640215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1955,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1955,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001309"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":22999,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:11.745064+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:11.745064+0000","last_peered":"2024-09-10T16:04:11.745064+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:11.745064+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2734}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33062,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180686848,"data_stored":180647105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626568,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":171550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149849,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953620,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472076,"kb_used_data":115580,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93899764,"statfs":{"total":96636764160,"available":96153358336,"internally_reserved":0,"allocated":118353920,"data_stored":118017779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117362688,"data_stored":117359075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:16.469 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:16.784 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:16.785 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:16.796 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-10T16:04:16.662277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33064,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1498164,"kb_used_data":176004,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281617356,"statfs":{"total":289910292480,"available":288376172544,"internally_reserved":0,"allocated":180228096,"data_stored":179182615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":854,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.876862"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":22999,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:11.745064+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:11.745064+0000","last_peered":"2024-09-10T16:04:11.745064+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:11.745064+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2734}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33062,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":178229248,"data_stored":178189505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626568,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":171550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149849,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953621,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469676,"kb_used_data":113180,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93902164,"statfs":{"total":96636764160,"available":96155815936,"internally_reserved":0,"allocated":115896320,"data_stored":115560179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114905088,"data_stored":114901475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:16.798 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:17.128 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:17.128 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:17.143 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-10T16:04:16.662277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33064,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1498164,"kb_used_data":176004,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281617356,"statfs":{"total":289910292480,"available":288376172544,"internally_reserved":0,"allocated":180228096,"data_stored":179182615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":854,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.876862"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":22999,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:11.745064+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:11.745064+0000","last_peered":"2024-09-10T16:04:11.745064+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:11.745064+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2734}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33062,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":178229248,"data_stored":178189505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626568,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":171550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149849,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953621,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469676,"kb_used_data":113180,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93902164,"statfs":{"total":96636764160,"available":96155815936,"internally_reserved":0,"allocated":115896320,"data_stored":115560179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114905088,"data_stored":114901475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:17.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:17.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21078, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.806114+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:08.806114+0000', 'last_peered': '2024-09-10T16:04:08.806114+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:08.806114+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:17.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:17.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 22999, 'reported_epoch': 60, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:11.745064+0000', 'last_change': '2024-09-10T16:04:07.189506+0000', 'last_active': '2024-09-10T16:04:11.745064+0000', 'last_peered': '2024-09-10T16:04:11.745064+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:04:11.745064+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2734, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1131, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2734}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:17.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:17.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21495, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.806200+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:04:08.806200+0000', 'last_peered': '2024-09-10T16:04:08.806200+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:04:08.806200+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:17.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:17.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21886, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.806641+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:04:08.806641+0000', 'last_peered': '2024-09-10T16:04:08.806641+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:04:08.806641+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:17.146 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:17.518 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:17.518 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13586/6 objects degraded (226433.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":783,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1534119936,"bytes_avail":288376172544,"bytes_total":289910292480,"degraded_objects":13586,"degraded_total":6,"degraded_ratio":2264.3333333333335,"recovering_objects_per_sec":96,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":854,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:17.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:17.532 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:17.838 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:17.839 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:17.851 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-10T16:04:16.662277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33064,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1498164,"kb_used_data":176004,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281617356,"statfs":{"total":289910292480,"available":288376172544,"internally_reserved":0,"allocated":180228096,"data_stored":179182615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":854,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.876862"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":22999,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:11.745064+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:11.745064+0000","last_peered":"2024-09-10T16:04:11.745064+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:11.745064+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2734}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33062,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":178229248,"data_stored":178189505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626568,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":171550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149849,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953621,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469676,"kb_used_data":113180,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93902164,"statfs":{"total":96636764160,"available":96155815936,"internally_reserved":0,"allocated":115896320,"data_stored":115560179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114905088,"data_stored":114901475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:20.854 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:21.189 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:21.189 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:21.202 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-09-10T16:04:20.662901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33856,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1506484,"kb_used_data":176004,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281609036,"statfs":{"total":289910292480,"available":288367652864,"internally_reserved":0,"allocated":180228096,"data_stored":179182615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1646,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1646,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.860414"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":24583,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:16.748297+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:16.748297+0000","last_peered":"2024-09-10T16:04:16.748297+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:16.748297+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1923,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1942}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33854,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":178229248,"data_stored":178189505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626569,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862112,"statfs":{"total":96636764160,"available":96114802688,"internally_reserved":0,"allocated":540672,"data_stored":171550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149850,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953621,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469676,"kb_used_data":113180,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93902164,"statfs":{"total":96636764160,"available":96155815936,"internally_reserved":0,"allocated":115896320,"data_stored":115560179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114905088,"data_stored":114901475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:21.203 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:21.514 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:21.529 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-09-10T16:04:20.662901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33856,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1506484,"kb_used_data":176004,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281609036,"statfs":{"total":289910292480,"available":288367652864,"internally_reserved":0,"allocated":180228096,"data_stored":179182615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1646,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1646,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.860414"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":24583,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:16.748297+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:16.748297+0000","last_peered":"2024-09-10T16:04:16.748297+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:16.748297+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1923,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1942}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33854,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":178229248,"data_stored":178189505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626569,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862112,"statfs":{"total":96636764160,"available":96114802688,"internally_reserved":0,"allocated":540672,"data_stored":171550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149850,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953621,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469676,"kb_used_data":113180,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93902164,"statfs":{"total":96636764160,"available":96155815936,"internally_reserved":0,"allocated":115896320,"data_stored":115560179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114905088,"data_stored":114901475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:21.530 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:21.530 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21078, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.806114+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:08.806114+0000', 'last_peered': '2024-09-10T16:04:08.806114+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:08.806114+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:21.530 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:21.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 24583, 'reported_epoch': 60, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:16.748297+0000', 'last_change': '2024-09-10T16:04:07.189506+0000', 'last_active': '2024-09-10T16:04:16.748297+0000', 'last_peered': '2024-09-10T16:04:16.748297+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:04:16.748297+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1923, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1942}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:21.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:21.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21495, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.806200+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:04:08.806200+0000', 'last_peered': '2024-09-10T16:04:08.806200+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:04:08.806200+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:21.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:21.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21886, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.806641+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:04:08.806641+0000', 'last_peered': '2024-09-10T16:04:08.806641+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:04:08.806641+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:21.531 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:21.952 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:21.952 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12794/6 objects degraded (213233.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":787,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1542639616,"bytes_avail":288367652864,"bytes_total":289910292480,"degraded_objects":12794,"degraded_total":6,"degraded_ratio":2132.3333333333335,"recovering_objects_per_sec":138,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1646,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:21.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:21.965 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:22.267 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:22.268 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:22.280 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-09-10T16:04:20.662901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33856,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1506484,"kb_used_data":176004,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281609036,"statfs":{"total":289910292480,"available":288367652864,"internally_reserved":0,"allocated":180228096,"data_stored":179182615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1646,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1646,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.860414"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":24583,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:16.748297+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:16.748297+0000","last_peered":"2024-09-10T16:04:16.748297+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:16.748297+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1923,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1942}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33854,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":178229248,"data_stored":178189505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626569,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862112,"statfs":{"total":96636764160,"available":96114802688,"internally_reserved":0,"allocated":540672,"data_stored":171550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149850,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953621,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469676,"kb_used_data":113180,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93902164,"statfs":{"total":96636764160,"available":96155815936,"internally_reserved":0,"allocated":115896320,"data_stored":115560179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114905088,"data_stored":114901475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:25.282 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:25.590 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:25.590 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:25.602 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-09-10T16:04:24.663627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34680,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495764,"kb_used_data":173604,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281619756,"statfs":{"total":289910292480,"available":288378630144,"internally_reserved":0,"allocated":177770496,"data_stored":176725015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002122"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":26231,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:21.752041+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:21.752041+0000","last_peered":"2024-09-10T16:04:21.752041+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:21.752041+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2747,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1118}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34678,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175771648,"data_stored":175731905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626570,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":171550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953622,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467276,"kb_used_data":110780,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93904564,"statfs":{"total":96636764160,"available":96158273536,"internally_reserved":0,"allocated":113438720,"data_stored":113102579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112447488,"data_stored":112443875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:25.604 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:25.908 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:25.908 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:25.921 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-09-10T16:04:24.663627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34680,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495764,"kb_used_data":173604,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281619756,"statfs":{"total":289910292480,"available":288378630144,"internally_reserved":0,"allocated":177770496,"data_stored":176725015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002122"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":26231,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:21.752041+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:21.752041+0000","last_peered":"2024-09-10T16:04:21.752041+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:21.752041+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2747,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1118}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34678,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":175771648,"data_stored":175731905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626570,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":171550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953622,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467276,"kb_used_data":110780,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93904564,"statfs":{"total":96636764160,"available":96158273536,"internally_reserved":0,"allocated":113438720,"data_stored":113102579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112447488,"data_stored":112443875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:25.923 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:25.924 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21078, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.806114+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:08.806114+0000', 'last_peered': '2024-09-10T16:04:08.806114+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:08.806114+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:25.924 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:25.924 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 26231, 'reported_epoch': 60, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:21.752041+0000', 'last_change': '2024-09-10T16:04:07.189506+0000', 'last_active': '2024-09-10T16:04:21.752041+0000', 'last_peered': '2024-09-10T16:04:21.752041+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:04:21.752041+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1118, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2747, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1118}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:25.924 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:25.924 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21495, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.806200+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:04:08.806200+0000', 'last_peered': '2024-09-10T16:04:08.806200+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:04:08.806200+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:25.925 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:25.925 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21886, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.806641+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:04:08.806641+0000', 'last_peered': '2024-09-10T16:04:08.806641+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:04:08.806641+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:25.925 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:26.330 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:26.331 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11970/6 objects degraded (199500.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":792,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1531662336,"bytes_avail":288378630144,"bytes_total":289910292480,"degraded_objects":11970,"degraded_total":6,"degraded_ratio":1995,"recovering_objects_per_sec":134,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1616,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:26.344 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:26.345 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:26.698 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:26.698 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:26.711 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-09-10T16:04:26.663940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34680,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493004,"kb_used_data":170844,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281622516,"statfs":{"total":289910292480,"available":288381456384,"internally_reserved":0,"allocated":174944256,"data_stored":173898775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1616,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001987"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":26231,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:21.752041+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:21.752041+0000","last_peered":"2024-09-10T16:04:21.752041+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:21.752041+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2747,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1118}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34678,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173314048,"data_stored":173274305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626570,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":171550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953623,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464516,"kb_used_data":108020,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93907324,"statfs":{"total":96636764160,"available":96161099776,"internally_reserved":0,"allocated":110612480,"data_stored":110276339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":109989888,"data_stored":109986275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:29.713 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:30.276 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:30.276 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:30.291 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-10T16:04:28.862607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35741,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501312,"kb_used_data":170832,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281614208,"statfs":{"total":289910292480,"available":288372948992,"internally_reserved":0,"allocated":174931968,"data_stored":173897326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1885,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1885,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.199705"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28353,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:26.753167+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:26.753167+0000","last_peered":"2024-09-10T16:04:26.753167+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:26.753167+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":57,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3808,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-57}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35739,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173301760,"data_stored":173272856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626571,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509716,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862124,"statfs":{"total":96636764160,"available":96114814976,"internally_reserved":0,"allocated":528384,"data_stored":170101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953623,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464516,"kb_used_data":108020,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93907324,"statfs":{"total":96636764160,"available":96161099776,"internally_reserved":0,"allocated":110612480,"data_stored":110276339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":109989888,"data_stored":109986275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:30.293 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:30.650 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:30.650 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:30.663 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-10T16:04:28.862607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35741,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501312,"kb_used_data":170832,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281614208,"statfs":{"total":289910292480,"available":288372948992,"internally_reserved":0,"allocated":174931968,"data_stored":173897326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1885,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1885,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.199705"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28353,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:26.753167+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:26.753167+0000","last_peered":"2024-09-10T16:04:26.753167+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:26.753167+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":57,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3808,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-57}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35739,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173301760,"data_stored":173272856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626571,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509716,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862124,"statfs":{"total":96636764160,"available":96114814976,"internally_reserved":0,"allocated":528384,"data_stored":170101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953623,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464516,"kb_used_data":108020,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93907324,"statfs":{"total":96636764160,"available":96161099776,"internally_reserved":0,"allocated":110612480,"data_stored":110276339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":109989888,"data_stored":109986275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:30.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:30.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21078, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.806114+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:08.806114+0000', 'last_peered': '2024-09-10T16:04:08.806114+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:08.806114+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:30.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-10T16:04:30.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'20578", 'reported_seq': 28353, 'reported_epoch': 60, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:26.753167+0000', 'last_change': '2024-09-10T16:04:07.189506+0000', 'last_active': '2024-09-10T16:04:26.753167+0000', 'last_peered': '2024-09-10T16:04:26.753167+0000', 'last_clean': '2024-09-10T15:53:17.298586+0000', 'last_became_active': '2024-09-10T15:53:19.964310+0000', 'last_became_peered': '2024-09-10T15:53:19.964310+0000', 'last_unstale': '2024-09-10T16:04:26.753167+0000', 'last_undegraded': '2024-09-10T15:53:19.535420+0000', 'last_fullsized': '2024-09-10T15:53:19.468206+0000', 'mapping_epoch': 43, 'log_start': "46'10500", 'ondisk_log_start': "46'10500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3191", 'last_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:05.799133+0000', 'objects_scrubbed': 3128, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1157, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 57, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17442, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3808, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -57}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:30.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:30.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21495, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.806200+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:04:08.806200+0000', 'last_peered': '2024-09-10T16:04:08.806200+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:04:08.806200+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:30.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:30.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 21886, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:08.806641+0000', 'last_change': '2024-09-10T15:53:19.971975+0000', 'last_active': '2024-09-10T16:04:08.806641+0000', 'last_peered': '2024-09-10T16:04:08.806641+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:04:08.806641+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3697, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 309, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:30.665 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:31.145 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:31.145 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10909/6 objects degraded (181816.667%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":797,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1537343488,"bytes_avail":288372948992,"bytes_total":289910292480,"degraded_objects":10909,"degraded_total":6,"degraded_ratio":1818.166666666667,"recovering_objects_per_sec":132,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1061,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:31.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:31.162 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:31.531 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:31.532 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:31.545 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-10T16:04:30.664567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35741,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501312,"kb_used_data":170832,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281614208,"statfs":{"total":289910292480,"available":288372948992,"internally_reserved":0,"allocated":174931968,"data_stored":173897326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1061,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001370"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28832,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808440+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:08.808440+0000","last_peered":"2024-09-10T16:04:08.808440+0000","last_clean":"2024-09-10T16:04:08.808440+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:08.808440+0000","last_undegraded":"2024-09-10T16:04:08.808440+0000","last_fullsized":"2024-09-10T16:04:08.808440+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21213,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999906+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:08.999906+0000","last_peered":"2024-09-10T16:04:08.999906+0000","last_clean":"2024-09-10T16:04:08.999906+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:08.999906+0000","last_undegraded":"2024-09-10T16:04:08.999906+0000","last_fullsized":"2024-09-10T16:04:08.999906+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21128,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809010+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:08.809010+0000","last_peered":"2024-09-10T16:04:08.809010+0000","last_clean":"2024-09-10T16:04:08.809010+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:08.809010+0000","last_undegraded":"2024-09-10T16:04:08.809010+0000","last_fullsized":"2024-09-10T16:04:08.809010+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20793,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999715+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:08.999715+0000","last_peered":"2024-09-10T16:04:08.999715+0000","last_clean":"2024-09-10T16:04:08.999715+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:08.999715+0000","last_undegraded":"2024-09-10T16:04:08.999715+0000","last_fullsized":"2024-09-10T16:04:08.999715+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20489,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808155+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:08.808155+0000","last_peered":"2024-09-10T16:04:08.808155+0000","last_clean":"2024-09-10T16:04:08.808155+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:08.808155+0000","last_undegraded":"2024-09-10T16:04:08.808155+0000","last_fullsized":"2024-09-10T16:04:08.808155+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28786,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.809029+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:08.809029+0000","last_peered":"2024-09-10T16:04:08.809029+0000","last_clean":"2024-09-10T16:04:08.809029+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:08.809029+0000","last_undegraded":"2024-09-10T16:04:08.809029+0000","last_fullsized":"2024-09-10T16:04:08.809029+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806114+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:08.806114+0000","last_peered":"2024-09-10T16:04:08.806114+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:08.806114+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21077,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.805936+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:08.805936+0000","last_peered":"2024-09-10T16:04:08.805936+0000","last_clean":"2024-09-10T16:04:08.805936+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:08.805936+0000","last_undegraded":"2024-09-10T16:04:08.805936+0000","last_fullsized":"2024-09-10T16:04:08.805936+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.808804+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:08.808804+0000","last_peered":"2024-09-10T16:04:08.808804+0000","last_clean":"2024-09-10T16:04:08.808804+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:08.808804+0000","last_undegraded":"2024-09-10T16:04:08.808804+0000","last_fullsized":"2024-09-10T16:04:08.808804+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28878,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810081+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:08.810081+0000","last_peered":"2024-09-10T16:04:08.810081+0000","last_clean":"2024-09-10T16:04:08.810081+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:08.810081+0000","last_undegraded":"2024-09-10T16:04:08.810081+0000","last_fullsized":"2024-09-10T16:04:08.810081+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999683+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:08.999683+0000","last_peered":"2024-09-10T16:04:08.999683+0000","last_clean":"2024-09-10T16:04:08.999683+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:08.999683+0000","last_undegraded":"2024-09-10T16:04:08.999683+0000","last_fullsized":"2024-09-10T16:04:08.999683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28353,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:26.753167+0000","last_change":"2024-09-10T16:04:07.189506+0000","last_active":"2024-09-10T16:04:26.753167+0000","last_peered":"2024-09-10T16:04:26.753167+0000","last_clean":"2024-09-10T15:53:17.298586+0000","last_became_active":"2024-09-10T15:53:19.964310+0000","last_became_peered":"2024-09-10T15:53:19.964310+0000","last_unstale":"2024-09-10T16:04:26.753167+0000","last_undegraded":"2024-09-10T15:53:19.535420+0000","last_fullsized":"2024-09-10T15:53:19.468206+0000","mapping_epoch":43,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":57,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3808,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-57}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.806489+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:08.806489+0000","last_peered":"2024-09-10T16:04:08.806489+0000","last_clean":"2024-09-10T16:04:08.806489+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:08.806489+0000","last_undegraded":"2024-09-10T16:04:08.806489+0000","last_fullsized":"2024-09-10T16:04:08.806489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28616,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999738+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:08.999738+0000","last_peered":"2024-09-10T16:04:08.999738+0000","last_clean":"2024-09-10T16:04:08.999738+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:08.999738+0000","last_undegraded":"2024-09-10T16:04:08.999738+0000","last_fullsized":"2024-09-10T16:04:08.999738+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810023+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:08.810023+0000","last_peered":"2024-09-10T16:04:08.810023+0000","last_clean":"2024-09-10T16:04:08.810023+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:08.810023+0000","last_undegraded":"2024-09-10T16:04:08.810023+0000","last_fullsized":"2024-09-10T16:04:08.810023+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28284,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.998979+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:08.998979+0000","last_peered":"2024-09-10T16:04:08.998979+0000","last_clean":"2024-09-10T16:04:08.998979+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:08.998979+0000","last_undegraded":"2024-09-10T16:04:08.998979+0000","last_fullsized":"2024-09-10T16:04:08.998979+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810301+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:08.810301+0000","last_peered":"2024-09-10T16:04:08.810301+0000","last_clean":"2024-09-10T16:04:08.810301+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:08.810301+0000","last_undegraded":"2024-09-10T16:04:08.810301+0000","last_fullsized":"2024-09-10T16:04:08.810301+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810369+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:08.810369+0000","last_peered":"2024-09-10T16:04:08.810369+0000","last_clean":"2024-09-10T16:04:08.810369+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:08.810369+0000","last_undegraded":"2024-09-10T16:04:08.810369+0000","last_fullsized":"2024-09-10T16:04:08.810369+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21495,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806200+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:08.806200+0000","last_peered":"2024-09-10T16:04:08.806200+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:08.806200+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.999781+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:08.999781+0000","last_peered":"2024-09-10T16:04:08.999781+0000","last_clean":"2024-09-10T16:04:08.999781+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:08.999781+0000","last_undegraded":"2024-09-10T16:04:08.999781+0000","last_fullsized":"2024-09-10T16:04:08.999781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21611,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.910853+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:08.910853+0000","last_peered":"2024-09-10T16:04:08.910853+0000","last_clean":"2024-09-10T16:04:08.910853+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:08.910853+0000","last_undegraded":"2024-09-10T16:04:08.910853+0000","last_fullsized":"2024-09-10T16:04:08.910853+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810297+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:08.810297+0000","last_peered":"2024-09-10T16:04:08.810297+0000","last_clean":"2024-09-10T16:04:08.810297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:08.810297+0000","last_undegraded":"2024-09-10T16:04:08.810297+0000","last_fullsized":"2024-09-10T16:04:08.810297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T16:04:08.810167+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:08.810167+0000","last_peered":"2024-09-10T16:04:08.810167+0000","last_clean":"2024-09-10T16:04:08.810167+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:08.810167+0000","last_undegraded":"2024-09-10T16:04:08.810167+0000","last_fullsized":"2024-09-10T16:04:08.810167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21886,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:08.806641+0000","last_change":"2024-09-10T15:53:19.971975+0000","last_active":"2024-09-10T16:04:08.806641+0000","last_peered":"2024-09-10T16:04:08.806641+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:08.806641+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35739,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173301760,"data_stored":173272856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626571,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509716,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862124,"statfs":{"total":96636764160,"available":96114814976,"internally_reserved":0,"allocated":528384,"data_stored":170101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527080,"kb_used_data":62296,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844760,"statfs":{"total":96636764160,"available":96097034240,"internally_reserved":0,"allocated":63791104,"data_stored":63450886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953623,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464516,"kb_used_data":108020,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93907324,"statfs":{"total":96636764160,"available":96161099776,"internally_reserved":0,"allocated":110612480,"data_stored":110276339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":109989888,"data_stored":109986275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:34.548 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:34.845 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:34.845 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:34.857 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-10T16:04:34.665238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36513,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486244,"kb_used_data":168244,"kb_used_omap":12,"kb_used_meta":1317939,"kb_avail":281629276,"statfs":{"total":289910292480,"available":288388378624,"internally_reserved":0,"allocated":172281856,"data_stored":171214829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1349569541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1833,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001298"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28790,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867473+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867473+0000","last_peered":"2024-09-10T16:04:28.867473+0000","last_clean":"2024-09-10T16:04:28.867473+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867473+0000","last_undegraded":"2024-09-10T16:04:28.867473+0000","last_fullsized":"2024-09-10T16:04:28.867473+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28478,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.962862+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.962862+0000","last_peered":"2024-09-10T16:04:28.962862+0000","last_clean":"2024-09-10T16:04:28.962862+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.962862+0000","last_undegraded":"2024-09-10T16:04:28.962862+0000","last_fullsized":"2024-09-10T16:04:28.962862+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864477+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864477+0000","last_peered":"2024-09-10T16:04:28.864477+0000","last_clean":"2024-09-10T16:04:28.864477+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864477+0000","last_undegraded":"2024-09-10T16:04:28.864477+0000","last_fullsized":"2024-09-10T16:04:28.864477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":23321,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:31.752816+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:31.752816+0000","last_peered":"2024-09-10T16:04:31.752816+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:31.752816+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1024,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36511,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":170479616,"data_stored":170450712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626572,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497252,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874588,"statfs":{"total":96636764160,"available":96127578112,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508620462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953624,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461896,"kb_used_data":105400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93909944,"statfs":{"total":96636764160,"available":96163782656,"internally_reserved":0,"allocated":107929600,"data_stored":107582664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107167744,"data_stored":107164131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:34.859 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:35.163 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:35.163 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:35.175 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-10T16:04:34.665238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36513,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486244,"kb_used_data":168244,"kb_used_omap":12,"kb_used_meta":1317939,"kb_avail":281629276,"statfs":{"total":289910292480,"available":288388378624,"internally_reserved":0,"allocated":172281856,"data_stored":171214829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1349569541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1833,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001298"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28790,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867473+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867473+0000","last_peered":"2024-09-10T16:04:28.867473+0000","last_clean":"2024-09-10T16:04:28.867473+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867473+0000","last_undegraded":"2024-09-10T16:04:28.867473+0000","last_fullsized":"2024-09-10T16:04:28.867473+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28478,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.962862+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.962862+0000","last_peered":"2024-09-10T16:04:28.962862+0000","last_clean":"2024-09-10T16:04:28.962862+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.962862+0000","last_undegraded":"2024-09-10T16:04:28.962862+0000","last_fullsized":"2024-09-10T16:04:28.962862+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864477+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864477+0000","last_peered":"2024-09-10T16:04:28.864477+0000","last_clean":"2024-09-10T16:04:28.864477+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864477+0000","last_undegraded":"2024-09-10T16:04:28.864477+0000","last_fullsized":"2024-09-10T16:04:28.864477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":23321,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:31.752816+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:31.752816+0000","last_peered":"2024-09-10T16:04:31.752816+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:31.752816+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1024,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36511,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":170479616,"data_stored":170450712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626572,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497252,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874588,"statfs":{"total":96636764160,"available":96127578112,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508620462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953624,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461896,"kb_used_data":105400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93909944,"statfs":{"total":96636764160,"available":96163782656,"internally_reserved":0,"allocated":107929600,"data_stored":107582664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107167744,"data_stored":107164131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:35.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:35.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21082, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.864734+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:28.864734+0000', 'last_peered': '2024-09-10T16:04:28.864734+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:28.864734+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:35.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:35.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21499, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.864661+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:04:28.864661+0000', 'last_peered': '2024-09-10T16:04:28.864661+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:04:28.864661+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:35.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:35.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 23321, 'reported_epoch': 62, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:31.752816+0000', 'last_change': '2024-09-10T16:04:26.981489+0000', 'last_active': '2024-09-10T16:04:31.752816+0000', 'last_peered': '2024-09-10T16:04:31.752816+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:04:31.752816+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1024, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2981}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:35.177 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:35.581 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:35.581 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10137/6 objects degraded (168950.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":801,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1521913856,"bytes_avail":288388378624,"bytes_total":289910292480,"degraded_objects":10137,"degraded_total":6,"degraded_ratio":1689.5,"recovering_objects_per_sec":229,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1833,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:35.597 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:35.597 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:35.908 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:35.909 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:35.921 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-10T16:04:34.665238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36513,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486244,"kb_used_data":168244,"kb_used_omap":12,"kb_used_meta":1317939,"kb_avail":281629276,"statfs":{"total":289910292480,"available":288388378624,"internally_reserved":0,"allocated":172281856,"data_stored":171214829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1349569541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1833,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001298"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28790,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867473+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867473+0000","last_peered":"2024-09-10T16:04:28.867473+0000","last_clean":"2024-09-10T16:04:28.867473+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867473+0000","last_undegraded":"2024-09-10T16:04:28.867473+0000","last_fullsized":"2024-09-10T16:04:28.867473+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28478,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.962862+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.962862+0000","last_peered":"2024-09-10T16:04:28.962862+0000","last_clean":"2024-09-10T16:04:28.962862+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.962862+0000","last_undegraded":"2024-09-10T16:04:28.962862+0000","last_fullsized":"2024-09-10T16:04:28.962862+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864477+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864477+0000","last_peered":"2024-09-10T16:04:28.864477+0000","last_clean":"2024-09-10T16:04:28.864477+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864477+0000","last_undegraded":"2024-09-10T16:04:28.864477+0000","last_fullsized":"2024-09-10T16:04:28.864477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":23321,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:31.752816+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:31.752816+0000","last_peered":"2024-09-10T16:04:31.752816+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:31.752816+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1024,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2981}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36511,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":170479616,"data_stored":170450712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626572,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497252,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874588,"statfs":{"total":96636764160,"available":96127578112,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508620462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953624,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461896,"kb_used_data":105400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93909944,"statfs":{"total":96636764160,"available":96163782656,"internally_reserved":0,"allocated":107929600,"data_stored":107582664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107167744,"data_stored":107164131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:38.923 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:39.250 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:39.251 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:39.262 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-10T16:04:38.665885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37303,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487404,"kb_used_data":165244,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281628116,"statfs":{"total":289910292480,"available":288387190784,"internally_reserved":0,"allocated":169209856,"data_stored":168142829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1562,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001603"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28790,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867473+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867473+0000","last_peered":"2024-09-10T16:04:28.867473+0000","last_clean":"2024-09-10T16:04:28.867473+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867473+0000","last_undegraded":"2024-09-10T16:04:28.867473+0000","last_fullsized":"2024-09-10T16:04:28.867473+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28478,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.962862+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.962862+0000","last_peered":"2024-09-10T16:04:28.962862+0000","last_clean":"2024-09-10T16:04:28.962862+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.962862+0000","last_undegraded":"2024-09-10T16:04:28.962862+0000","last_fullsized":"2024-09-10T16:04:28.962862+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864477+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864477+0000","last_peered":"2024-09-10T16:04:28.864477+0000","last_clean":"2024-09-10T16:04:28.864477+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864477+0000","last_undegraded":"2024-09-10T16:04:28.864477+0000","last_fullsized":"2024-09-10T16:04:28.864477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":24901,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:36.756094+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:36.756094+0000","last_peered":"2024-09-10T16:04:36.756094+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:36.756094+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1814,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2191}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37301,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167407616,"data_stored":167378712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501412,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870428,"statfs":{"total":96636764160,"available":96123318272,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953625,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458896,"kb_used_data":102400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93912944,"statfs":{"total":96636764160,"available":96166854656,"internally_reserved":0,"allocated":104857600,"data_stored":104510664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":104095744,"data_stored":104092131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:39.263 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:39.581 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:39.581 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:39.594 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-10T16:04:38.665885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37303,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487404,"kb_used_data":165244,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281628116,"statfs":{"total":289910292480,"available":288387190784,"internally_reserved":0,"allocated":169209856,"data_stored":168142829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1562,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001603"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28790,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867473+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867473+0000","last_peered":"2024-09-10T16:04:28.867473+0000","last_clean":"2024-09-10T16:04:28.867473+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867473+0000","last_undegraded":"2024-09-10T16:04:28.867473+0000","last_fullsized":"2024-09-10T16:04:28.867473+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28478,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.962862+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.962862+0000","last_peered":"2024-09-10T16:04:28.962862+0000","last_clean":"2024-09-10T16:04:28.962862+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.962862+0000","last_undegraded":"2024-09-10T16:04:28.962862+0000","last_fullsized":"2024-09-10T16:04:28.962862+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864477+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864477+0000","last_peered":"2024-09-10T16:04:28.864477+0000","last_clean":"2024-09-10T16:04:28.864477+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864477+0000","last_undegraded":"2024-09-10T16:04:28.864477+0000","last_fullsized":"2024-09-10T16:04:28.864477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":24901,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:36.756094+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:36.756094+0000","last_peered":"2024-09-10T16:04:36.756094+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:36.756094+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1814,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2191}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37301,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167407616,"data_stored":167378712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501412,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870428,"statfs":{"total":96636764160,"available":96123318272,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953625,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458896,"kb_used_data":102400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93912944,"statfs":{"total":96636764160,"available":96166854656,"internally_reserved":0,"allocated":104857600,"data_stored":104510664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":104095744,"data_stored":104092131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:39.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:39.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21082, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.864734+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:28.864734+0000', 'last_peered': '2024-09-10T16:04:28.864734+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:28.864734+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:39.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:39.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21499, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.864661+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:04:28.864661+0000', 'last_peered': '2024-09-10T16:04:28.864661+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:04:28.864661+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:39.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:39.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 24901, 'reported_epoch': 62, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:36.756094+0000', 'last_change': '2024-09-10T16:04:26.981489+0000', 'last_active': '2024-09-10T16:04:36.756094+0000', 'last_peered': '2024-09-10T16:04:36.756094+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:04:36.756094+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2192, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1814, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2191}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:39.596 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:39.986 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:39.987 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9347/6 objects degraded (155783.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":805,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1523101696,"bytes_avail":288387190784,"bytes_total":289910292480,"degraded_objects":9347,"degraded_total":6,"degraded_ratio":1557.833333333333,"recovering_objects_per_sec":156,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1562,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:40.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:40.000 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:40.335 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:40.335 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:40.348 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-10T16:04:38.665885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37303,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487404,"kb_used_data":165244,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281628116,"statfs":{"total":289910292480,"available":288387190784,"internally_reserved":0,"allocated":169209856,"data_stored":168142829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1562,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001603"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28790,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867473+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867473+0000","last_peered":"2024-09-10T16:04:28.867473+0000","last_clean":"2024-09-10T16:04:28.867473+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867473+0000","last_undegraded":"2024-09-10T16:04:28.867473+0000","last_fullsized":"2024-09-10T16:04:28.867473+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28478,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.962862+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.962862+0000","last_peered":"2024-09-10T16:04:28.962862+0000","last_clean":"2024-09-10T16:04:28.962862+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.962862+0000","last_undegraded":"2024-09-10T16:04:28.962862+0000","last_fullsized":"2024-09-10T16:04:28.962862+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864477+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864477+0000","last_peered":"2024-09-10T16:04:28.864477+0000","last_clean":"2024-09-10T16:04:28.864477+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864477+0000","last_undegraded":"2024-09-10T16:04:28.864477+0000","last_fullsized":"2024-09-10T16:04:28.864477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":24901,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:36.756094+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:36.756094+0000","last_peered":"2024-09-10T16:04:36.756094+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:36.756094+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1814,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2191}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37301,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167407616,"data_stored":167378712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501412,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870428,"statfs":{"total":96636764160,"available":96123318272,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953625,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458896,"kb_used_data":102400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93912944,"statfs":{"total":96636764160,"available":96166854656,"internally_reserved":0,"allocated":104857600,"data_stored":104510664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":104095744,"data_stored":104092131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:43.350 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:43.647 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:43.647 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:43.659 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-09-10T16:04:42.666705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38023,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492724,"kb_used_data":162244,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281622796,"statfs":{"total":289910292480,"available":288381743104,"internally_reserved":0,"allocated":166137856,"data_stored":165070829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002138"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28790,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867473+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867473+0000","last_peered":"2024-09-10T16:04:28.867473+0000","last_clean":"2024-09-10T16:04:28.867473+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867473+0000","last_undegraded":"2024-09-10T16:04:28.867473+0000","last_fullsized":"2024-09-10T16:04:28.867473+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.963290+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.963290+0000","last_peered":"2024-09-10T16:04:28.963290+0000","last_clean":"2024-09-10T16:04:28.963290+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.963290+0000","last_undegraded":"2024-09-10T16:04:28.963290+0000","last_fullsized":"2024-09-10T16:04:28.963290+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":137,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864527+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864527+0000","last_peered":"2024-09-10T16:04:28.864527+0000","last_clean":"2024-09-10T16:04:28.864527+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864527+0000","last_undegraded":"2024-09-10T16:04:28.864527+0000","last_fullsized":"2024-09-10T16:04:28.864527+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":26341,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:41.755014+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:41.755014+0000","last_peered":"2024-09-10T16:04:41.755014+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:41.755014+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2534,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38021,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164335616,"data_stored":164306712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626574,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509732,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862108,"statfs":{"total":96636764160,"available":96114798592,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953626,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455896,"kb_used_data":99400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93915944,"statfs":{"total":96636764160,"available":96169926656,"internally_reserved":0,"allocated":101785600,"data_stored":101438664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101023744,"data_stored":101020131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:43.660 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:43.973 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:43.974 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:43.984 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-09-10T16:04:42.666705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38023,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492724,"kb_used_data":162244,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281622796,"statfs":{"total":289910292480,"available":288381743104,"internally_reserved":0,"allocated":166137856,"data_stored":165070829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002138"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28790,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867473+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867473+0000","last_peered":"2024-09-10T16:04:28.867473+0000","last_clean":"2024-09-10T16:04:28.867473+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867473+0000","last_undegraded":"2024-09-10T16:04:28.867473+0000","last_fullsized":"2024-09-10T16:04:28.867473+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.963290+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.963290+0000","last_peered":"2024-09-10T16:04:28.963290+0000","last_clean":"2024-09-10T16:04:28.963290+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.963290+0000","last_undegraded":"2024-09-10T16:04:28.963290+0000","last_fullsized":"2024-09-10T16:04:28.963290+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":137,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864527+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864527+0000","last_peered":"2024-09-10T16:04:28.864527+0000","last_clean":"2024-09-10T16:04:28.864527+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864527+0000","last_undegraded":"2024-09-10T16:04:28.864527+0000","last_fullsized":"2024-09-10T16:04:28.864527+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":26341,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:41.755014+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:41.755014+0000","last_peered":"2024-09-10T16:04:41.755014+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:41.755014+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2534,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38021,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164335616,"data_stored":164306712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626574,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509732,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862108,"statfs":{"total":96636764160,"available":96114798592,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953626,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455896,"kb_used_data":99400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93915944,"statfs":{"total":96636764160,"available":96169926656,"internally_reserved":0,"allocated":101785600,"data_stored":101438664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101023744,"data_stored":101020131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:43.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:43.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21082, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.864734+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:28.864734+0000', 'last_peered': '2024-09-10T16:04:28.864734+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:28.864734+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:43.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:43.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21499, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.864661+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:04:28.864661+0000', 'last_peered': '2024-09-10T16:04:28.864661+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:04:28.864661+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:43.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:43.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 26341, 'reported_epoch': 62, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:41.755014+0000', 'last_change': '2024-09-10T16:04:26.981489+0000', 'last_active': '2024-09-10T16:04:41.755014+0000', 'last_peered': '2024-09-10T16:04:41.755014+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:04:41.755014+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1472, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2534, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1471}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:43.986 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:44.375 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:44.375 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8627/6 objects degraded (143783.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":810,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1528549376,"bytes_avail":288381743104,"bytes_total":289910292480,"degraded_objects":8627,"degraded_total":6,"degraded_ratio":1437.833333333333,"recovering_objects_per_sec":190,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2282,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:44.390 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:44.391 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:44.693 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:44.693 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:44.706 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-10T16:04:44.667071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38023,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492724,"kb_used_data":162244,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281622796,"statfs":{"total":289910292480,"available":288381743104,"internally_reserved":0,"allocated":166137856,"data_stored":165070829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1510,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002180"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28790,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867473+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867473+0000","last_peered":"2024-09-10T16:04:28.867473+0000","last_clean":"2024-09-10T16:04:28.867473+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867473+0000","last_undegraded":"2024-09-10T16:04:28.867473+0000","last_fullsized":"2024-09-10T16:04:28.867473+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.963290+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.963290+0000","last_peered":"2024-09-10T16:04:28.963290+0000","last_clean":"2024-09-10T16:04:28.963290+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.963290+0000","last_undegraded":"2024-09-10T16:04:28.963290+0000","last_fullsized":"2024-09-10T16:04:28.963290+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":137,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864527+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864527+0000","last_peered":"2024-09-10T16:04:28.864527+0000","last_clean":"2024-09-10T16:04:28.864527+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864527+0000","last_undegraded":"2024-09-10T16:04:28.864527+0000","last_fullsized":"2024-09-10T16:04:28.864527+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":26341,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:41.755014+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:41.755014+0000","last_peered":"2024-09-10T16:04:41.755014+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:41.755014+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2534,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38021,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164335616,"data_stored":164306712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626574,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509732,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862108,"statfs":{"total":96636764160,"available":96114798592,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953626,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455896,"kb_used_data":99400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93915944,"statfs":{"total":96636764160,"available":96169926656,"internally_reserved":0,"allocated":101785600,"data_stored":101438664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101023744,"data_stored":101020131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:47.708 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:48.040 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:48.041 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:48.053 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-10T16:04:46.667357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38023,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1489724,"kb_used_data":159244,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281625796,"statfs":{"total":289910292480,"available":288384815104,"internally_reserved":0,"allocated":163065856,"data_stored":161998829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1510,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002118"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28790,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867473+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867473+0000","last_peered":"2024-09-10T16:04:28.867473+0000","last_clean":"2024-09-10T16:04:28.867473+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867473+0000","last_undegraded":"2024-09-10T16:04:28.867473+0000","last_fullsized":"2024-09-10T16:04:28.867473+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.963290+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.963290+0000","last_peered":"2024-09-10T16:04:28.963290+0000","last_clean":"2024-09-10T16:04:28.963290+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.963290+0000","last_undegraded":"2024-09-10T16:04:28.963290+0000","last_fullsized":"2024-09-10T16:04:28.963290+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":137,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864527+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864527+0000","last_peered":"2024-09-10T16:04:28.864527+0000","last_clean":"2024-09-10T16:04:28.864527+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864527+0000","last_undegraded":"2024-09-10T16:04:28.864527+0000","last_fullsized":"2024-09-10T16:04:28.864527+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":26341,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:41.755014+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:41.755014+0000","last_peered":"2024-09-10T16:04:41.755014+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:41.755014+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2534,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38021,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":161263616,"data_stored":161234712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626574,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509732,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862108,"statfs":{"total":96636764160,"available":96114798592,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953627,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452896,"kb_used_data":96400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93918944,"statfs":{"total":96636764160,"available":96172998656,"internally_reserved":0,"allocated":98713600,"data_stored":98366664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":97951744,"data_stored":97948131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:48.054 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:48.368 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:48.368 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:48.381 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-10T16:04:46.667357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38023,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1489724,"kb_used_data":159244,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281625796,"statfs":{"total":289910292480,"available":288384815104,"internally_reserved":0,"allocated":163065856,"data_stored":161998829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1510,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002118"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28790,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867473+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867473+0000","last_peered":"2024-09-10T16:04:28.867473+0000","last_clean":"2024-09-10T16:04:28.867473+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867473+0000","last_undegraded":"2024-09-10T16:04:28.867473+0000","last_fullsized":"2024-09-10T16:04:28.867473+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.963290+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.963290+0000","last_peered":"2024-09-10T16:04:28.963290+0000","last_clean":"2024-09-10T16:04:28.963290+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.963290+0000","last_undegraded":"2024-09-10T16:04:28.963290+0000","last_fullsized":"2024-09-10T16:04:28.963290+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":137,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864527+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864527+0000","last_peered":"2024-09-10T16:04:28.864527+0000","last_clean":"2024-09-10T16:04:28.864527+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864527+0000","last_undegraded":"2024-09-10T16:04:28.864527+0000","last_fullsized":"2024-09-10T16:04:28.864527+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":26341,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:41.755014+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:41.755014+0000","last_peered":"2024-09-10T16:04:41.755014+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:41.755014+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2534,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38021,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":161263616,"data_stored":161234712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626574,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509732,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862108,"statfs":{"total":96636764160,"available":96114798592,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953627,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452896,"kb_used_data":96400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93918944,"statfs":{"total":96636764160,"available":96172998656,"internally_reserved":0,"allocated":98713600,"data_stored":98366664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":97951744,"data_stored":97948131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:48.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:48.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21082, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.864734+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:28.864734+0000', 'last_peered': '2024-09-10T16:04:28.864734+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:28.864734+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:48.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:48.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 21499, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:28.864661+0000', 'last_change': '2024-09-10T15:53:19.975517+0000', 'last_active': '2024-09-10T16:04:28.864661+0000', 'last_peered': '2024-09-10T16:04:28.864661+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:04:28.864661+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3629, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:48.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-10T16:04:48.384 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "50'21098", 'reported_seq': 26341, 'reported_epoch': 62, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:41.755014+0000', 'last_change': '2024-09-10T16:04:26.981489+0000', 'last_active': '2024-09-10T16:04:41.755014+0000', 'last_peered': '2024-09-10T16:04:41.755014+0000', 'last_clean': '2024-09-10T15:53:17.318207+0000', 'last_became_active': '2024-09-10T15:53:19.963829+0000', 'last_became_peered': '2024-09-10T15:53:19.963829+0000', 'last_unstale': '2024-09-10T16:04:41.755014+0000', 'last_undegraded': '2024-09-10T15:53:19.549986+0000', 'last_fullsized': '2024-09-10T15:53:19.472400+0000', 'mapping_epoch': 43, 'log_start': "46'11000", 'ondisk_log_start': "46'11000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3241", 'last_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:04.965466+0000', 'objects_scrubbed': 3170, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1293, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1472, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17917, 'num_write_kb': 29496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2534, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1471}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:48.384 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:48.761 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8627/6 objects degraded (143783.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":814,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1525477376,"bytes_avail":288384815104,"bytes_total":289910292480,"degraded_objects":8627,"degraded_total":6,"degraded_ratio":1437.833333333333,"recovering_objects_per_sec":125,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1510,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:48.775 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:48.775 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:49.085 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:49.088 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:49.099 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-09-10T16:04:48.667661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39001,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481404,"kb_used_data":159244,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281634116,"statfs":{"total":289910292480,"available":288393334784,"internally_reserved":0,"allocated":163065856,"data_stored":161998829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2488,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002102"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28791,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867517+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867517+0000","last_peered":"2024-09-10T16:04:28.867517+0000","last_clean":"2024-09-10T16:04:28.867517+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867517+0000","last_undegraded":"2024-09-10T16:04:28.867517+0000","last_fullsized":"2024-09-10T16:04:28.867517+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.963290+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.963290+0000","last_peered":"2024-09-10T16:04:28.963290+0000","last_clean":"2024-09-10T16:04:28.963290+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.963290+0000","last_undegraded":"2024-09-10T16:04:28.963290+0000","last_fullsized":"2024-09-10T16:04:28.963290+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":137,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864527+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864527+0000","last_peered":"2024-09-10T16:04:28.864527+0000","last_clean":"2024-09-10T16:04:28.864527+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864527+0000","last_undegraded":"2024-09-10T16:04:28.864527+0000","last_fullsized":"2024-09-10T16:04:28.864527+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":28297,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:46.757281+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:46.757281+0000","last_peered":"2024-09-10T16:04:46.757281+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:46.757281+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3512,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-493}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38999,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":161263616,"data_stored":161234712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626575,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501412,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870428,"statfs":{"total":96636764160,"available":96123318272,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953627,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452896,"kb_used_data":96400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93918944,"statfs":{"total":96636764160,"available":96172998656,"internally_reserved":0,"allocated":98713600,"data_stored":98366664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":97951744,"data_stored":97948131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:52.101 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:52.476 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:52.476 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:52.490 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-10T16:04:50.932541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39001,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481404,"kb_used_data":159244,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281634116,"statfs":{"total":289910292480,"available":288393334784,"internally_reserved":0,"allocated":163065856,"data_stored":161998829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":978,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.265836"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28836,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864955+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:28.864955+0000","last_peered":"2024-09-10T16:04:28.864955+0000","last_clean":"2024-09-10T16:04:28.864955+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:28.864955+0000","last_undegraded":"2024-09-10T16:04:28.864955+0000","last_fullsized":"2024-09-10T16:04:28.864955+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21217,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864629+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:28.864629+0000","last_peered":"2024-09-10T16:04:28.864629+0000","last_clean":"2024-09-10T16:04:28.864629+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:28.864629+0000","last_undegraded":"2024-09-10T16:04:28.864629+0000","last_fullsized":"2024-09-10T16:04:28.864629+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21132,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867590+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:28.867590+0000","last_peered":"2024-09-10T16:04:28.867590+0000","last_clean":"2024-09-10T16:04:28.867590+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:28.867590+0000","last_undegraded":"2024-09-10T16:04:28.867590+0000","last_fullsized":"2024-09-10T16:04:28.867590+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20797,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864470+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:28.864470+0000","last_peered":"2024-09-10T16:04:28.864470+0000","last_clean":"2024-09-10T16:04:28.864470+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:28.864470+0000","last_undegraded":"2024-09-10T16:04:28.864470+0000","last_fullsized":"2024-09-10T16:04:28.864470+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20493,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.865362+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:28.865362+0000","last_peered":"2024-09-10T16:04:28.865362+0000","last_clean":"2024-09-10T16:04:28.865362+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:28.865362+0000","last_undegraded":"2024-09-10T16:04:28.865362+0000","last_fullsized":"2024-09-10T16:04:28.865362+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28791,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867517+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:28.867517+0000","last_peered":"2024-09-10T16:04:28.867517+0000","last_clean":"2024-09-10T16:04:28.867517+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:28.867517+0000","last_undegraded":"2024-09-10T16:04:28.867517+0000","last_fullsized":"2024-09-10T16:04:28.867517+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21082,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864734+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:28.864734+0000","last_peered":"2024-09-10T16:04:28.864734+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:28.864734+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21081,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864492+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:28.864492+0000","last_peered":"2024-09-10T16:04:28.864492+0000","last_clean":"2024-09-10T16:04:28.864492+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:28.864492+0000","last_undegraded":"2024-09-10T16:04:28.864492+0000","last_fullsized":"2024-09-10T16:04:28.864492+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.866453+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:28.866453+0000","last_peered":"2024-09-10T16:04:28.866453+0000","last_clean":"2024-09-10T16:04:28.866453+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:28.866453+0000","last_undegraded":"2024-09-10T16:04:28.866453+0000","last_fullsized":"2024-09-10T16:04:28.866453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28882,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867488+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:28.867488+0000","last_peered":"2024-09-10T16:04:28.867488+0000","last_clean":"2024-09-10T16:04:28.867488+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:28.867488+0000","last_undegraded":"2024-09-10T16:04:28.867488+0000","last_fullsized":"2024-09-10T16:04:28.867488+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863812+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:28.863812+0000","last_peered":"2024-09-10T16:04:28.863812+0000","last_clean":"2024-09-10T16:04:28.863812+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:28.863812+0000","last_undegraded":"2024-09-10T16:04:28.863812+0000","last_fullsized":"2024-09-10T16:04:28.863812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.963290+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:28.963290+0000","last_peered":"2024-09-10T16:04:28.963290+0000","last_clean":"2024-09-10T16:04:28.963290+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:28.963290+0000","last_undegraded":"2024-09-10T16:04:28.963290+0000","last_fullsized":"2024-09-10T16:04:28.963290+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":137,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864527+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:28.864527+0000","last_peered":"2024-09-10T16:04:28.864527+0000","last_clean":"2024-09-10T16:04:28.864527+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:28.864527+0000","last_undegraded":"2024-09-10T16:04:28.864527+0000","last_fullsized":"2024-09-10T16:04:28.864527+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863972+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:28.863972+0000","last_peered":"2024-09-10T16:04:28.863972+0000","last_clean":"2024-09-10T16:04:28.863972+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:28.863972+0000","last_undegraded":"2024-09-10T16:04:28.863972+0000","last_fullsized":"2024-09-10T16:04:28.863972+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867476+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:28.867476+0000","last_peered":"2024-09-10T16:04:28.867476+0000","last_clean":"2024-09-10T16:04:28.867476+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:28.867476+0000","last_undegraded":"2024-09-10T16:04:28.867476+0000","last_fullsized":"2024-09-10T16:04:28.867476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28288,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.863855+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:28.863855+0000","last_peered":"2024-09-10T16:04:28.863855+0000","last_clean":"2024-09-10T16:04:28.863855+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:28.863855+0000","last_undegraded":"2024-09-10T16:04:28.863855+0000","last_fullsized":"2024-09-10T16:04:28.863855+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21726,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867730+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:28.867730+0000","last_peered":"2024-09-10T16:04:28.867730+0000","last_clean":"2024-09-10T16:04:28.867730+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:28.867730+0000","last_undegraded":"2024-09-10T16:04:28.867730+0000","last_fullsized":"2024-09-10T16:04:28.867730+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867581+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:28.867581+0000","last_peered":"2024-09-10T16:04:28.867581+0000","last_clean":"2024-09-10T16:04:28.867581+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:28.867581+0000","last_undegraded":"2024-09-10T16:04:28.867581+0000","last_fullsized":"2024-09-10T16:04:28.867581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21499,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:28.864661+0000","last_change":"2024-09-10T15:53:19.975517+0000","last_active":"2024-09-10T16:04:28.864661+0000","last_peered":"2024-09-10T16:04:28.864661+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:28.864661+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3629,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.864775+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:28.864775+0000","last_peered":"2024-09-10T16:04:28.864775+0000","last_clean":"2024-09-10T16:04:28.864775+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:28.864775+0000","last_undegraded":"2024-09-10T16:04:28.864775+0000","last_fullsized":"2024-09-10T16:04:28.864775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21615,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867596+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:28.867596+0000","last_peered":"2024-09-10T16:04:28.867596+0000","last_clean":"2024-09-10T16:04:28.867596+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:28.867596+0000","last_undegraded":"2024-09-10T16:04:28.867596+0000","last_fullsized":"2024-09-10T16:04:28.867596+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867671+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:28.867671+0000","last_peered":"2024-09-10T16:04:28.867671+0000","last_clean":"2024-09-10T16:04:28.867671+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:28.867671+0000","last_undegraded":"2024-09-10T16:04:28.867671+0000","last_fullsized":"2024-09-10T16:04:28.867671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T16:04:28.867715+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:28.867715+0000","last_peered":"2024-09-10T16:04:28.867715+0000","last_clean":"2024-09-10T16:04:28.867715+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:28.867715+0000","last_undegraded":"2024-09-10T16:04:28.867715+0000","last_fullsized":"2024-09-10T16:04:28.867715+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":28297,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:46.757281+0000","last_change":"2024-09-10T16:04:26.981489+0000","last_active":"2024-09-10T16:04:46.757281+0000","last_peered":"2024-09-10T16:04:46.757281+0000","last_clean":"2024-09-10T15:53:17.318207+0000","last_became_active":"2024-09-10T15:53:19.963829+0000","last_became_peered":"2024-09-10T15:53:19.963829+0000","last_unstale":"2024-09-10T16:04:46.757281+0000","last_undegraded":"2024-09-10T15:53:19.549986+0000","last_fullsized":"2024-09-10T15:53:19.472400+0000","mapping_epoch":43,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3512,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-493}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7649,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38999,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":161263616,"data_stored":161234712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626575,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501412,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870428,"statfs":{"total":96636764160,"available":96123318272,"internally_reserved":0,"allocated":544768,"data_stored":175690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527096,"kb_used_data":62312,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844744,"statfs":{"total":96636764160,"available":96097017856,"internally_reserved":0,"allocated":63807488,"data_stored":63456475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953627,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452896,"kb_used_data":96400,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93918944,"statfs":{"total":96636764160,"available":96172998656,"internally_reserved":0,"allocated":98713600,"data_stored":98366664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":97951744,"data_stored":97948131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:52.492 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:52.849 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:52.850 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:52.862 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-09-10T16:04:52.668370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39932,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486744,"kb_used_data":156264,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281628776,"statfs":{"total":289910292480,"available":288387866624,"internally_reserved":0,"allocated":160014336,"data_stored":158936479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1909,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001299"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21219,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.930905+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:49.930905+0000","last_peered":"2024-09-10T16:04:49.930905+0000","last_clean":"2024-09-10T16:04:49.930905+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:49.930905+0000","last_undegraded":"2024-09-10T16:04:49.930905+0000","last_fullsized":"2024-09-10T16:04:49.930905+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20799,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.930462+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:49.930462+0000","last_peered":"2024-09-10T16:04:49.930462+0000","last_clean":"2024-09-10T16:04:49.930462+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:49.930462+0000","last_undegraded":"2024-09-10T16:04:49.930462+0000","last_fullsized":"2024-09-10T16:04:49.930462+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":131,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.929363+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:49.929363+0000","last_peered":"2024-09-10T16:04:49.929363+0000","last_clean":"2024-09-10T16:04:49.929363+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:49.929363+0000","last_undegraded":"2024-09-10T16:04:49.929363+0000","last_fullsized":"2024-09-10T16:04:49.929363+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.930378+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:49.930378+0000","last_peered":"2024-09-10T16:04:49.930378+0000","last_clean":"2024-09-10T16:04:49.930378+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:49.930378+0000","last_undegraded":"2024-09-10T16:04:49.930378+0000","last_fullsized":"2024-09-10T16:04:49.930378+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28290,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.929136+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:49.929136+0000","last_peered":"2024-09-10T16:04:49.929136+0000","last_clean":"2024-09-10T16:04:49.929136+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:49.929136+0000","last_undegraded":"2024-09-10T16:04:49.929136+0000","last_fullsized":"2024-09-10T16:04:49.929136+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":22378,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:51.756952+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:04:51.756952+0000","last_peered":"2024-09-10T16:04:51.756952+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:51.756952+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":131,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.931068+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:49.931068+0000","last_peered":"2024-09-10T16:04:49.931068+0000","last_clean":"2024-09-10T16:04:49.931068+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:49.931068+0000","last_undegraded":"2024-09-10T16:04:49.931068+0000","last_fullsized":"2024-09-10T16:04:49.931068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39930,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158179328,"data_stored":158161263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626576,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527104,"kb_used_data":62320,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844736,"statfs":{"total":96636764160,"available":96097009664,"internally_reserved":0,"allocated":63815680,"data_stored":63459255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953628,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":449904,"kb_used_data":93408,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93921936,"statfs":{"total":96636764160,"available":96176062464,"internally_reserved":0,"allocated":95649792,"data_stored":95297444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94879744,"data_stored":94876131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:52.864 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:52.864 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21086, 'reported_epoch': 64, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:50.934696+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:50.934696+0000', 'last_peered': '2024-09-10T16:04:50.934696+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:50.934696+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:52.864 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:52.864 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 22378, 'reported_epoch': 64, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:51.756952+0000', 'last_change': '2024-09-10T16:04:49.009212+0000', 'last_active': '2024-09-10T16:04:51.756952+0000', 'last_peered': '2024-09-10T16:04:51.756952+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:04:51.756952+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3192, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 728, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:52.865 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:53.339 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:53.339 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6718/6 objects degraded (111966.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":819,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1522425856,"bytes_avail":288387866624,"bytes_total":289910292480,"degraded_objects":6718,"degraded_total":6,"degraded_ratio":1119.6666666666667,"recovering_objects_per_sec":238,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1909,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:53.352 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:53.352 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:53.675 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:53.675 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:53.689 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-09-10T16:04:52.668370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39932,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486744,"kb_used_data":156264,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281628776,"statfs":{"total":289910292480,"available":288387866624,"internally_reserved":0,"allocated":160014336,"data_stored":158936479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1909,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001299"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21219,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.930905+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:49.930905+0000","last_peered":"2024-09-10T16:04:49.930905+0000","last_clean":"2024-09-10T16:04:49.930905+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:49.930905+0000","last_undegraded":"2024-09-10T16:04:49.930905+0000","last_fullsized":"2024-09-10T16:04:49.930905+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20799,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.930462+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:49.930462+0000","last_peered":"2024-09-10T16:04:49.930462+0000","last_clean":"2024-09-10T16:04:49.930462+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:49.930462+0000","last_undegraded":"2024-09-10T16:04:49.930462+0000","last_fullsized":"2024-09-10T16:04:49.930462+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":131,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.929363+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:49.929363+0000","last_peered":"2024-09-10T16:04:49.929363+0000","last_clean":"2024-09-10T16:04:49.929363+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:49.929363+0000","last_undegraded":"2024-09-10T16:04:49.929363+0000","last_fullsized":"2024-09-10T16:04:49.929363+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.930378+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:49.930378+0000","last_peered":"2024-09-10T16:04:49.930378+0000","last_clean":"2024-09-10T16:04:49.930378+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:49.930378+0000","last_undegraded":"2024-09-10T16:04:49.930378+0000","last_fullsized":"2024-09-10T16:04:49.930378+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28290,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.929136+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:49.929136+0000","last_peered":"2024-09-10T16:04:49.929136+0000","last_clean":"2024-09-10T16:04:49.929136+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:49.929136+0000","last_undegraded":"2024-09-10T16:04:49.929136+0000","last_fullsized":"2024-09-10T16:04:49.929136+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":22378,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:51.756952+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:04:51.756952+0000","last_peered":"2024-09-10T16:04:51.756952+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:51.756952+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":131,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T16:04:49.931068+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:49.931068+0000","last_peered":"2024-09-10T16:04:49.931068+0000","last_clean":"2024-09-10T16:04:49.931068+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:49.931068+0000","last_undegraded":"2024-09-10T16:04:49.931068+0000","last_fullsized":"2024-09-10T16:04:49.931068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39930,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158179328,"data_stored":158161263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626576,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527104,"kb_used_data":62320,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844736,"statfs":{"total":96636764160,"available":96097009664,"internally_reserved":0,"allocated":63815680,"data_stored":63459255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953628,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":449904,"kb_used_data":93408,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93921936,"statfs":{"total":96636764160,"available":96176062464,"internally_reserved":0,"allocated":95649792,"data_stored":95297444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94879744,"data_stored":94876131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:56.691 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:57.026 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:57.027 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:57.039 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-09-10T16:04:56.669014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39932,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483156,"kb_used_data":152676,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281632364,"statfs":{"total":289910292480,"available":288391540736,"internally_reserved":0,"allocated":156340224,"data_stored":155256934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":931,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001353"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439460+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:51.439460+0000","last_peered":"2024-09-10T16:04:51.439460+0000","last_clean":"2024-09-10T16:04:51.439460+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:51.439460+0000","last_undegraded":"2024-09-10T16:04:51.439460+0000","last_fullsized":"2024-09-10T16:04:51.439460+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439440+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:51.439440+0000","last_peered":"2024-09-10T16:04:51.439440+0000","last_clean":"2024-09-10T16:04:51.439440+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:51.439440+0000","last_undegraded":"2024-09-10T16:04:51.439440+0000","last_fullsized":"2024-09-10T16:04:51.439440+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439237+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:51.439237+0000","last_peered":"2024-09-10T16:04:51.439237+0000","last_clean":"2024-09-10T16:04:51.439237+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:51.439237+0000","last_undegraded":"2024-09-10T16:04:51.439237+0000","last_fullsized":"2024-09-10T16:04:51.439237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28625,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439338+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:51.439338+0000","last_peered":"2024-09-10T16:04:51.439338+0000","last_clean":"2024-09-10T16:04:51.439338+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:51.439338+0000","last_undegraded":"2024-09-10T16:04:51.439338+0000","last_fullsized":"2024-09-10T16:04:51.439338+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28293,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439155+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:51.439155+0000","last_peered":"2024-09-10T16:04:51.439155+0000","last_clean":"2024-09-10T16:04:51.439155+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:51.439155+0000","last_undegraded":"2024-09-10T16:04:51.439155+0000","last_fullsized":"2024-09-10T16:04:51.439155+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":22378,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:51.756952+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:04:51.756952+0000","last_peered":"2024-09-10T16:04:51.756952+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:51.756952+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439547+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:51.439547+0000","last_peered":"2024-09-10T16:04:51.439547+0000","last_clean":"2024-09-10T16:04:51.439547+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:51.439547+0000","last_undegraded":"2024-09-10T16:04:51.439547+0000","last_fullsized":"2024-09-10T16:04:51.439547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39930,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154497024,"data_stored":154478959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626576,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527104,"kb_used_data":62320,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844736,"statfs":{"total":96636764160,"available":96097009664,"internally_reserved":0,"allocated":63815680,"data_stored":63459255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953629,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446316,"kb_used_data":89820,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93925524,"statfs":{"total":96636764160,"available":96179736576,"internally_reserved":0,"allocated":91975680,"data_stored":91617899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91197440,"data_stored":91193827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:57.041 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:57.418 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:57.429 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-09-10T16:04:56.669014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39932,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483156,"kb_used_data":152676,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281632364,"statfs":{"total":289910292480,"available":288391540736,"internally_reserved":0,"allocated":156340224,"data_stored":155256934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":931,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001353"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439460+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:51.439460+0000","last_peered":"2024-09-10T16:04:51.439460+0000","last_clean":"2024-09-10T16:04:51.439460+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:51.439460+0000","last_undegraded":"2024-09-10T16:04:51.439460+0000","last_fullsized":"2024-09-10T16:04:51.439460+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439440+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:51.439440+0000","last_peered":"2024-09-10T16:04:51.439440+0000","last_clean":"2024-09-10T16:04:51.439440+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:51.439440+0000","last_undegraded":"2024-09-10T16:04:51.439440+0000","last_fullsized":"2024-09-10T16:04:51.439440+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439237+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:51.439237+0000","last_peered":"2024-09-10T16:04:51.439237+0000","last_clean":"2024-09-10T16:04:51.439237+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:51.439237+0000","last_undegraded":"2024-09-10T16:04:51.439237+0000","last_fullsized":"2024-09-10T16:04:51.439237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28625,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439338+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:51.439338+0000","last_peered":"2024-09-10T16:04:51.439338+0000","last_clean":"2024-09-10T16:04:51.439338+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:51.439338+0000","last_undegraded":"2024-09-10T16:04:51.439338+0000","last_fullsized":"2024-09-10T16:04:51.439338+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28293,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439155+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:51.439155+0000","last_peered":"2024-09-10T16:04:51.439155+0000","last_clean":"2024-09-10T16:04:51.439155+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:51.439155+0000","last_undegraded":"2024-09-10T16:04:51.439155+0000","last_fullsized":"2024-09-10T16:04:51.439155+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":22378,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:51.756952+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:04:51.756952+0000","last_peered":"2024-09-10T16:04:51.756952+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:51.756952+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439547+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:51.439547+0000","last_peered":"2024-09-10T16:04:51.439547+0000","last_clean":"2024-09-10T16:04:51.439547+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:51.439547+0000","last_undegraded":"2024-09-10T16:04:51.439547+0000","last_fullsized":"2024-09-10T16:04:51.439547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39930,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154497024,"data_stored":154478959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626576,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527104,"kb_used_data":62320,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844736,"statfs":{"total":96636764160,"available":96097009664,"internally_reserved":0,"allocated":63815680,"data_stored":63459255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953629,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446316,"kb_used_data":89820,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93925524,"statfs":{"total":96636764160,"available":96179736576,"internally_reserved":0,"allocated":91975680,"data_stored":91617899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91197440,"data_stored":91193827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:04:57.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:04:57.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21086, 'reported_epoch': 64, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:50.934696+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:50.934696+0000', 'last_peered': '2024-09-10T16:04:50.934696+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:50.934696+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:57.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:04:57.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 22378, 'reported_epoch': 64, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:51.756952+0000', 'last_change': '2024-09-10T16:04:49.009212+0000', 'last_active': '2024-09-10T16:04:51.756952+0000', 'last_peered': '2024-09-10T16:04:51.756952+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:04:51.756952+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3192, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 728, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:04:57.431 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:04:57.815 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:57.815 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6718/6 objects degraded (111966.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":823,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1518751744,"bytes_avail":288391540736,"bytes_total":289910292480,"degraded_objects":6718,"degraded_total":6,"degraded_ratio":1119.6666666666667,"recovering_objects_per_sec":116,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":931,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:04:57.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:04:57.829 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:04:58.183 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:04:58.184 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:04:58.197 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-09-10T16:04:56.669014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39932,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483156,"kb_used_data":152676,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281632364,"statfs":{"total":289910292480,"available":288391540736,"internally_reserved":0,"allocated":156340224,"data_stored":155256934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":931,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001353"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439460+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:51.439460+0000","last_peered":"2024-09-10T16:04:51.439460+0000","last_clean":"2024-09-10T16:04:51.439460+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:51.439460+0000","last_undegraded":"2024-09-10T16:04:51.439460+0000","last_fullsized":"2024-09-10T16:04:51.439460+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439440+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:51.439440+0000","last_peered":"2024-09-10T16:04:51.439440+0000","last_clean":"2024-09-10T16:04:51.439440+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:51.439440+0000","last_undegraded":"2024-09-10T16:04:51.439440+0000","last_fullsized":"2024-09-10T16:04:51.439440+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439237+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:51.439237+0000","last_peered":"2024-09-10T16:04:51.439237+0000","last_clean":"2024-09-10T16:04:51.439237+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:51.439237+0000","last_undegraded":"2024-09-10T16:04:51.439237+0000","last_fullsized":"2024-09-10T16:04:51.439237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28625,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439338+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:51.439338+0000","last_peered":"2024-09-10T16:04:51.439338+0000","last_clean":"2024-09-10T16:04:51.439338+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:51.439338+0000","last_undegraded":"2024-09-10T16:04:51.439338+0000","last_fullsized":"2024-09-10T16:04:51.439338+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28293,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439155+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:51.439155+0000","last_peered":"2024-09-10T16:04:51.439155+0000","last_clean":"2024-09-10T16:04:51.439155+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:51.439155+0000","last_undegraded":"2024-09-10T16:04:51.439155+0000","last_fullsized":"2024-09-10T16:04:51.439155+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":22378,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:51.756952+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:04:51.756952+0000","last_peered":"2024-09-10T16:04:51.756952+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:51.756952+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":728,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439547+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:51.439547+0000","last_peered":"2024-09-10T16:04:51.439547+0000","last_clean":"2024-09-10T16:04:51.439547+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:51.439547+0000","last_undegraded":"2024-09-10T16:04:51.439547+0000","last_fullsized":"2024-09-10T16:04:51.439547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6718,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39930,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154497024,"data_stored":154478959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626576,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527104,"kb_used_data":62320,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844736,"statfs":{"total":96636764160,"available":96097009664,"internally_reserved":0,"allocated":63815680,"data_stored":63459255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953629,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446316,"kb_used_data":89820,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93925524,"statfs":{"total":96636764160,"available":96179736576,"internally_reserved":0,"allocated":91975680,"data_stored":91617899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91197440,"data_stored":91193827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:01.200 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:01.503 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:01.504 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:01.515 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-10T16:05:00.669634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40903,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1474844,"kb_used_data":152684,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281640676,"statfs":{"total":289910292480,"available":288400052224,"internally_reserved":0,"allocated":156348416,"data_stored":155259693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1902,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1902,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001546"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439460+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:51.439460+0000","last_peered":"2024-09-10T16:04:51.439460+0000","last_clean":"2024-09-10T16:04:51.439460+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:51.439460+0000","last_undegraded":"2024-09-10T16:04:51.439460+0000","last_fullsized":"2024-09-10T16:04:51.439460+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439440+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:51.439440+0000","last_peered":"2024-09-10T16:04:51.439440+0000","last_clean":"2024-09-10T16:04:51.439440+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:51.439440+0000","last_undegraded":"2024-09-10T16:04:51.439440+0000","last_fullsized":"2024-09-10T16:04:51.439440+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439237+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:51.439237+0000","last_peered":"2024-09-10T16:04:51.439237+0000","last_clean":"2024-09-10T16:04:51.439237+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:51.439237+0000","last_undegraded":"2024-09-10T16:04:51.439237+0000","last_fullsized":"2024-09-10T16:04:51.439237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28625,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439338+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:51.439338+0000","last_peered":"2024-09-10T16:04:51.439338+0000","last_clean":"2024-09-10T16:04:51.439338+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:51.439338+0000","last_undegraded":"2024-09-10T16:04:51.439338+0000","last_fullsized":"2024-09-10T16:04:51.439338+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28293,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439155+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:51.439155+0000","last_peered":"2024-09-10T16:04:51.439155+0000","last_clean":"2024-09-10T16:04:51.439155+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:51.439155+0000","last_undegraded":"2024-09-10T16:04:51.439155+0000","last_fullsized":"2024-09-10T16:04:51.439155+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":24320,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:56.755082+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:04:56.755082+0000","last_peered":"2024-09-10T16:04:56.755082+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:56.755082+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1699,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439547+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:51.439547+0000","last_peered":"2024-09-10T16:04:51.439547+0000","last_clean":"2024-09-10T16:04:51.439547+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:51.439547+0000","last_undegraded":"2024-09-10T16:04:51.439547+0000","last_fullsized":"2024-09-10T16:04:51.439547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40901,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154497024,"data_stored":154478959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626577,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501416,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870424,"statfs":{"total":96636764160,"available":96123314176,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149858,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527112,"kb_used_data":62328,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844728,"statfs":{"total":96636764160,"available":96097001472,"internally_reserved":0,"allocated":63823872,"data_stored":63462014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953629,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446316,"kb_used_data":89820,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93925524,"statfs":{"total":96636764160,"available":96179736576,"internally_reserved":0,"allocated":91975680,"data_stored":91617899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91197440,"data_stored":91193827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:01.517 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:01.825 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:01.838 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-10T16:05:00.669634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40903,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1474844,"kb_used_data":152684,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281640676,"statfs":{"total":289910292480,"available":288400052224,"internally_reserved":0,"allocated":156348416,"data_stored":155259693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1902,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1902,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001546"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439460+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:51.439460+0000","last_peered":"2024-09-10T16:04:51.439460+0000","last_clean":"2024-09-10T16:04:51.439460+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:51.439460+0000","last_undegraded":"2024-09-10T16:04:51.439460+0000","last_fullsized":"2024-09-10T16:04:51.439460+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439440+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:51.439440+0000","last_peered":"2024-09-10T16:04:51.439440+0000","last_clean":"2024-09-10T16:04:51.439440+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:51.439440+0000","last_undegraded":"2024-09-10T16:04:51.439440+0000","last_fullsized":"2024-09-10T16:04:51.439440+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439237+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:51.439237+0000","last_peered":"2024-09-10T16:04:51.439237+0000","last_clean":"2024-09-10T16:04:51.439237+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:51.439237+0000","last_undegraded":"2024-09-10T16:04:51.439237+0000","last_fullsized":"2024-09-10T16:04:51.439237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28625,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439338+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:51.439338+0000","last_peered":"2024-09-10T16:04:51.439338+0000","last_clean":"2024-09-10T16:04:51.439338+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:51.439338+0000","last_undegraded":"2024-09-10T16:04:51.439338+0000","last_fullsized":"2024-09-10T16:04:51.439338+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28293,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439155+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:51.439155+0000","last_peered":"2024-09-10T16:04:51.439155+0000","last_clean":"2024-09-10T16:04:51.439155+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:51.439155+0000","last_undegraded":"2024-09-10T16:04:51.439155+0000","last_fullsized":"2024-09-10T16:04:51.439155+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":24320,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:56.755082+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:04:56.755082+0000","last_peered":"2024-09-10T16:04:56.755082+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:56.755082+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1699,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439547+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:51.439547+0000","last_peered":"2024-09-10T16:04:51.439547+0000","last_clean":"2024-09-10T16:04:51.439547+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:51.439547+0000","last_undegraded":"2024-09-10T16:04:51.439547+0000","last_fullsized":"2024-09-10T16:04:51.439547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40901,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154497024,"data_stored":154478959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626577,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501416,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870424,"statfs":{"total":96636764160,"available":96123314176,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149858,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527112,"kb_used_data":62328,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844728,"statfs":{"total":96636764160,"available":96097001472,"internally_reserved":0,"allocated":63823872,"data_stored":63462014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953629,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446316,"kb_used_data":89820,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93925524,"statfs":{"total":96636764160,"available":96179736576,"internally_reserved":0,"allocated":91975680,"data_stored":91617899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91197440,"data_stored":91193827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:01.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:01.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21086, 'reported_epoch': 64, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:50.934696+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:50.934696+0000', 'last_peered': '2024-09-10T16:04:50.934696+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:50.934696+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:01.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:05:01.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 24320, 'reported_epoch': 64, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:56.755082+0000', 'last_change': '2024-09-10T16:04:49.009212+0000', 'last_active': '2024-09-10T16:04:56.755082+0000', 'last_peered': '2024-09-10T16:04:56.755082+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:04:56.755082+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2221, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1699, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2221}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:01.840 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:02.209 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:02.209 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5747/6 objects degraded (95783.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":828,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1510240256,"bytes_avail":288400052224,"bytes_total":289910292480,"degraded_objects":5747,"degraded_total":6,"degraded_ratio":957.83333333333348,"recovering_objects_per_sec":190,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1902,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:02.222 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:02.222 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:02.604 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:02.604 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:02.618 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-10T16:05:00.669634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40903,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1474844,"kb_used_data":152684,"kb_used_omap":12,"kb_used_meta":1322099,"kb_avail":281640676,"statfs":{"total":289910292480,"available":288400052224,"internally_reserved":0,"allocated":156348416,"data_stored":155259693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1353829381},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1902,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1902,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001546"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439460+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:51.439460+0000","last_peered":"2024-09-10T16:04:51.439460+0000","last_clean":"2024-09-10T16:04:51.439460+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:51.439460+0000","last_undegraded":"2024-09-10T16:04:51.439460+0000","last_fullsized":"2024-09-10T16:04:51.439460+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439440+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:51.439440+0000","last_peered":"2024-09-10T16:04:51.439440+0000","last_clean":"2024-09-10T16:04:51.439440+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:51.439440+0000","last_undegraded":"2024-09-10T16:04:51.439440+0000","last_fullsized":"2024-09-10T16:04:51.439440+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439237+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:51.439237+0000","last_peered":"2024-09-10T16:04:51.439237+0000","last_clean":"2024-09-10T16:04:51.439237+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:51.439237+0000","last_undegraded":"2024-09-10T16:04:51.439237+0000","last_fullsized":"2024-09-10T16:04:51.439237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28625,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439338+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:51.439338+0000","last_peered":"2024-09-10T16:04:51.439338+0000","last_clean":"2024-09-10T16:04:51.439338+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:51.439338+0000","last_undegraded":"2024-09-10T16:04:51.439338+0000","last_fullsized":"2024-09-10T16:04:51.439338+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28293,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439155+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:51.439155+0000","last_peered":"2024-09-10T16:04:51.439155+0000","last_clean":"2024-09-10T16:04:51.439155+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:51.439155+0000","last_undegraded":"2024-09-10T16:04:51.439155+0000","last_fullsized":"2024-09-10T16:04:51.439155+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":24320,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:56.755082+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:04:56.755082+0000","last_peered":"2024-09-10T16:04:56.755082+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:04:56.755082+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1699,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439547+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:51.439547+0000","last_peered":"2024-09-10T16:04:51.439547+0000","last_clean":"2024-09-10T16:04:51.439547+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:51.439547+0000","last_undegraded":"2024-09-10T16:04:51.439547+0000","last_fullsized":"2024-09-10T16:04:51.439547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40901,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":154497024,"data_stored":154478959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626577,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501416,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870424,"statfs":{"total":96636764160,"available":96123314176,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512880302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149858,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527112,"kb_used_data":62328,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844728,"statfs":{"total":96636764160,"available":96097001472,"internally_reserved":0,"allocated":63823872,"data_stored":63462014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953629,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446316,"kb_used_data":89820,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93925524,"statfs":{"total":96636764160,"available":96179736576,"internally_reserved":0,"allocated":91975680,"data_stored":91617899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91197440,"data_stored":91193827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:05.621 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:05.937 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:05.937 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:05.949 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-10T16:05:04.670327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41688,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1475404,"kb_used_data":149084,"kb_used_omap":12,"kb_used_meta":1326259,"kb_avail":281640116,"statfs":{"total":289910292480,"available":288399478784,"internally_reserved":0,"allocated":152662016,"data_stored":151573293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1358089221},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1756,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1756,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001957"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439460+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:51.439460+0000","last_peered":"2024-09-10T16:04:51.439460+0000","last_clean":"2024-09-10T16:04:51.439460+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:51.439460+0000","last_undegraded":"2024-09-10T16:04:51.439460+0000","last_fullsized":"2024-09-10T16:04:51.439460+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439440+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:51.439440+0000","last_peered":"2024-09-10T16:04:51.439440+0000","last_clean":"2024-09-10T16:04:51.439440+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:51.439440+0000","last_undegraded":"2024-09-10T16:04:51.439440+0000","last_fullsized":"2024-09-10T16:04:51.439440+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439237+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:51.439237+0000","last_peered":"2024-09-10T16:04:51.439237+0000","last_clean":"2024-09-10T16:04:51.439237+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:51.439237+0000","last_undegraded":"2024-09-10T16:04:51.439237+0000","last_fullsized":"2024-09-10T16:04:51.439237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28625,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439338+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:51.439338+0000","last_peered":"2024-09-10T16:04:51.439338+0000","last_clean":"2024-09-10T16:04:51.439338+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:51.439338+0000","last_undegraded":"2024-09-10T16:04:51.439338+0000","last_fullsized":"2024-09-10T16:04:51.439338+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28293,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439155+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:51.439155+0000","last_peered":"2024-09-10T16:04:51.439155+0000","last_clean":"2024-09-10T16:04:51.439155+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:51.439155+0000","last_undegraded":"2024-09-10T16:04:51.439155+0000","last_fullsized":"2024-09-10T16:04:51.439155+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":25890,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:01.759743+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:05:01.759743+0000","last_peered":"2024-09-10T16:05:01.759743+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:05:01.759743+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2484,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1436}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439547+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:51.439547+0000","last_peered":"2024-09-10T16:04:51.439547+0000","last_clean":"2024-09-10T16:04:51.439547+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:51.439547+0000","last_undegraded":"2024-09-10T16:04:51.439547+0000","last_fullsized":"2024-09-10T16:04:51.439547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41686,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":150810624,"data_stored":150792559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626578,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505576,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866264,"statfs":{"total":96636764160,"available":96119054336,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517140142},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149859,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527112,"kb_used_data":62328,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844728,"statfs":{"total":96636764160,"available":96097001472,"internally_reserved":0,"allocated":63823872,"data_stored":63462014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953630,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442716,"kb_used_data":86220,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93929124,"statfs":{"total":96636764160,"available":96183422976,"internally_reserved":0,"allocated":88289280,"data_stored":87931499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":87511040,"data_stored":87507427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:05.951 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:06.258 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:06.258 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:06.273 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-10T16:05:04.670327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41688,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1475404,"kb_used_data":149084,"kb_used_omap":12,"kb_used_meta":1326259,"kb_avail":281640116,"statfs":{"total":289910292480,"available":288399478784,"internally_reserved":0,"allocated":152662016,"data_stored":151573293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1358089221},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1756,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1756,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001957"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439460+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:51.439460+0000","last_peered":"2024-09-10T16:04:51.439460+0000","last_clean":"2024-09-10T16:04:51.439460+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:51.439460+0000","last_undegraded":"2024-09-10T16:04:51.439460+0000","last_fullsized":"2024-09-10T16:04:51.439460+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439440+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:51.439440+0000","last_peered":"2024-09-10T16:04:51.439440+0000","last_clean":"2024-09-10T16:04:51.439440+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:51.439440+0000","last_undegraded":"2024-09-10T16:04:51.439440+0000","last_fullsized":"2024-09-10T16:04:51.439440+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439237+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:51.439237+0000","last_peered":"2024-09-10T16:04:51.439237+0000","last_clean":"2024-09-10T16:04:51.439237+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:51.439237+0000","last_undegraded":"2024-09-10T16:04:51.439237+0000","last_fullsized":"2024-09-10T16:04:51.439237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28625,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439338+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:51.439338+0000","last_peered":"2024-09-10T16:04:51.439338+0000","last_clean":"2024-09-10T16:04:51.439338+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:51.439338+0000","last_undegraded":"2024-09-10T16:04:51.439338+0000","last_fullsized":"2024-09-10T16:04:51.439338+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28293,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439155+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:51.439155+0000","last_peered":"2024-09-10T16:04:51.439155+0000","last_clean":"2024-09-10T16:04:51.439155+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:51.439155+0000","last_undegraded":"2024-09-10T16:04:51.439155+0000","last_fullsized":"2024-09-10T16:04:51.439155+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":25890,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:01.759743+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:05:01.759743+0000","last_peered":"2024-09-10T16:05:01.759743+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:05:01.759743+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2484,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1436}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439547+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:51.439547+0000","last_peered":"2024-09-10T16:04:51.439547+0000","last_clean":"2024-09-10T16:04:51.439547+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:51.439547+0000","last_undegraded":"2024-09-10T16:04:51.439547+0000","last_fullsized":"2024-09-10T16:04:51.439547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41686,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":150810624,"data_stored":150792559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626578,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505576,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866264,"statfs":{"total":96636764160,"available":96119054336,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517140142},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149859,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527112,"kb_used_data":62328,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844728,"statfs":{"total":96636764160,"available":96097001472,"internally_reserved":0,"allocated":63823872,"data_stored":63462014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953630,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442716,"kb_used_data":86220,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93929124,"statfs":{"total":96636764160,"available":96183422976,"internally_reserved":0,"allocated":88289280,"data_stored":87931499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":87511040,"data_stored":87507427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:06.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:06.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21086, 'reported_epoch': 64, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:50.934696+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:50.934696+0000', 'last_peered': '2024-09-10T16:04:50.934696+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:50.934696+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:06.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:05:06.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 25890, 'reported_epoch': 64, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:01.759743+0000', 'last_change': '2024-09-10T16:04:49.009212+0000', 'last_active': '2024-09-10T16:05:01.759743+0000', 'last_peered': '2024-09-10T16:05:01.759743+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:05:01.759743+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1436, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2484, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1436}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:06.275 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:06.656 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:06.657 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4962/6 objects degraded (82700.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":832,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1510813696,"bytes_avail":288399478784,"bytes_total":289910292480,"degraded_objects":4962,"degraded_total":6,"degraded_ratio":827,"recovering_objects_per_sec":146,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1756,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:06.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:06.670 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:06.990 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:06.991 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:07.003 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-09-10T16:05:06.670634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41688,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1471804,"kb_used_data":145484,"kb_used_omap":12,"kb_used_meta":1326259,"kb_avail":281643716,"statfs":{"total":289910292480,"available":288403165184,"internally_reserved":0,"allocated":148975616,"data_stored":147886893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1358089221},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1756,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1756,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001929"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439460+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:51.439460+0000","last_peered":"2024-09-10T16:04:51.439460+0000","last_clean":"2024-09-10T16:04:51.439460+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:51.439460+0000","last_undegraded":"2024-09-10T16:04:51.439460+0000","last_fullsized":"2024-09-10T16:04:51.439460+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439440+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:51.439440+0000","last_peered":"2024-09-10T16:04:51.439440+0000","last_clean":"2024-09-10T16:04:51.439440+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:51.439440+0000","last_undegraded":"2024-09-10T16:04:51.439440+0000","last_fullsized":"2024-09-10T16:04:51.439440+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439237+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:51.439237+0000","last_peered":"2024-09-10T16:04:51.439237+0000","last_clean":"2024-09-10T16:04:51.439237+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:51.439237+0000","last_undegraded":"2024-09-10T16:04:51.439237+0000","last_fullsized":"2024-09-10T16:04:51.439237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28625,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439338+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:51.439338+0000","last_peered":"2024-09-10T16:04:51.439338+0000","last_clean":"2024-09-10T16:04:51.439338+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:51.439338+0000","last_undegraded":"2024-09-10T16:04:51.439338+0000","last_fullsized":"2024-09-10T16:04:51.439338+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28293,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439155+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:51.439155+0000","last_peered":"2024-09-10T16:04:51.439155+0000","last_clean":"2024-09-10T16:04:51.439155+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:51.439155+0000","last_undegraded":"2024-09-10T16:04:51.439155+0000","last_fullsized":"2024-09-10T16:04:51.439155+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":25890,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:01.759743+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:05:01.759743+0000","last_peered":"2024-09-10T16:05:01.759743+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:05:01.759743+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2484,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1436}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439547+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:51.439547+0000","last_peered":"2024-09-10T16:04:51.439547+0000","last_clean":"2024-09-10T16:04:51.439547+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:51.439547+0000","last_undegraded":"2024-09-10T16:04:51.439547+0000","last_fullsized":"2024-09-10T16:04:51.439547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41686,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147124224,"data_stored":147106159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626578,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505576,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866264,"statfs":{"total":96636764160,"available":96119054336,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517140142},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149859,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527112,"kb_used_data":62328,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844728,"statfs":{"total":96636764160,"available":96097001472,"internally_reserved":0,"allocated":63823872,"data_stored":63462014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953631,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439116,"kb_used_data":82620,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93932724,"statfs":{"total":96636764160,"available":96187109376,"internally_reserved":0,"allocated":84602880,"data_stored":84245099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":83824640,"data_stored":83821027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:10.006 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:10.407 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:10.407 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:10.419 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-10T16:05:09.955041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42641,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463484,"kb_used_data":145484,"kb_used_omap":12,"kb_used_meta":1317939,"kb_avail":281652036,"statfs":{"total":289910292480,"available":288411684864,"internally_reserved":0,"allocated":148975616,"data_stored":147886893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1349569541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1738,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1738,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.285406"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439460+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:51.439460+0000","last_peered":"2024-09-10T16:04:51.439460+0000","last_clean":"2024-09-10T16:04:51.439460+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:51.439460+0000","last_undegraded":"2024-09-10T16:04:51.439460+0000","last_fullsized":"2024-09-10T16:04:51.439460+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439440+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:51.439440+0000","last_peered":"2024-09-10T16:04:51.439440+0000","last_clean":"2024-09-10T16:04:51.439440+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:51.439440+0000","last_undegraded":"2024-09-10T16:04:51.439440+0000","last_fullsized":"2024-09-10T16:04:51.439440+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439237+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:51.439237+0000","last_peered":"2024-09-10T16:04:51.439237+0000","last_clean":"2024-09-10T16:04:51.439237+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:51.439237+0000","last_undegraded":"2024-09-10T16:04:51.439237+0000","last_fullsized":"2024-09-10T16:04:51.439237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28625,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439338+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:51.439338+0000","last_peered":"2024-09-10T16:04:51.439338+0000","last_clean":"2024-09-10T16:04:51.439338+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:51.439338+0000","last_undegraded":"2024-09-10T16:04:51.439338+0000","last_fullsized":"2024-09-10T16:04:51.439338+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28293,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439155+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:51.439155+0000","last_peered":"2024-09-10T16:04:51.439155+0000","last_clean":"2024-09-10T16:04:51.439155+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:51.439155+0000","last_undegraded":"2024-09-10T16:04:51.439155+0000","last_fullsized":"2024-09-10T16:04:51.439155+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":27796,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:06.756935+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:05:06.756935+0000","last_peered":"2024-09-10T16:05:06.756935+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:05:06.756935+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3437,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-483}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439547+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:51.439547+0000","last_peered":"2024-09-10T16:04:51.439547+0000","last_clean":"2024-09-10T16:04:51.439547+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:51.439547+0000","last_undegraded":"2024-09-10T16:04:51.439547+0000","last_fullsized":"2024-09-10T16:04:51.439547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42639,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147124224,"data_stored":147106159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626579,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497256,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874584,"statfs":{"total":96636764160,"available":96127574016,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508620462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149860,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527112,"kb_used_data":62328,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844728,"statfs":{"total":96636764160,"available":96097001472,"internally_reserved":0,"allocated":63823872,"data_stored":63462014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953631,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439116,"kb_used_data":82620,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93932724,"statfs":{"total":96636764160,"available":96187109376,"internally_reserved":0,"allocated":84602880,"data_stored":84245099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":83824640,"data_stored":83821027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:10.421 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:10.715 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:10.715 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:10.726 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-09-10T16:05:10.671284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42641,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463484,"kb_used_data":145484,"kb_used_omap":12,"kb_used_meta":1317939,"kb_avail":281652036,"statfs":{"total":289910292480,"available":288411684864,"internally_reserved":0,"allocated":148975616,"data_stored":147886893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1349569541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001354"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439460+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:51.439460+0000","last_peered":"2024-09-10T16:04:51.439460+0000","last_clean":"2024-09-10T16:04:51.439460+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:51.439460+0000","last_undegraded":"2024-09-10T16:04:51.439460+0000","last_fullsized":"2024-09-10T16:04:51.439460+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439440+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:51.439440+0000","last_peered":"2024-09-10T16:04:51.439440+0000","last_clean":"2024-09-10T16:04:51.439440+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:51.439440+0000","last_undegraded":"2024-09-10T16:04:51.439440+0000","last_fullsized":"2024-09-10T16:04:51.439440+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439237+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:51.439237+0000","last_peered":"2024-09-10T16:04:51.439237+0000","last_clean":"2024-09-10T16:04:51.439237+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:51.439237+0000","last_undegraded":"2024-09-10T16:04:51.439237+0000","last_fullsized":"2024-09-10T16:04:51.439237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28625,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439338+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:51.439338+0000","last_peered":"2024-09-10T16:04:51.439338+0000","last_clean":"2024-09-10T16:04:51.439338+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:51.439338+0000","last_undegraded":"2024-09-10T16:04:51.439338+0000","last_fullsized":"2024-09-10T16:04:51.439338+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28293,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439155+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:51.439155+0000","last_peered":"2024-09-10T16:04:51.439155+0000","last_clean":"2024-09-10T16:04:51.439155+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:51.439155+0000","last_undegraded":"2024-09-10T16:04:51.439155+0000","last_fullsized":"2024-09-10T16:04:51.439155+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":27796,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:06.756935+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:05:06.756935+0000","last_peered":"2024-09-10T16:05:06.756935+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:05:06.756935+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3437,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-483}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439547+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:51.439547+0000","last_peered":"2024-09-10T16:04:51.439547+0000","last_clean":"2024-09-10T16:04:51.439547+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:51.439547+0000","last_undegraded":"2024-09-10T16:04:51.439547+0000","last_fullsized":"2024-09-10T16:04:51.439547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42639,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147124224,"data_stored":147106159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626579,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497256,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874584,"statfs":{"total":96636764160,"available":96127574016,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508620462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149860,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527112,"kb_used_data":62328,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844728,"statfs":{"total":96636764160,"available":96097001472,"internally_reserved":0,"allocated":63823872,"data_stored":63462014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953631,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439116,"kb_used_data":82620,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93932724,"statfs":{"total":96636764160,"available":96187109376,"internally_reserved":0,"allocated":84602880,"data_stored":84245099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":83824640,"data_stored":83821027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:10.728 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:10.728 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21086, 'reported_epoch': 64, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:04:50.934696+0000', 'last_change': '2024-09-10T15:53:19.978937+0000', 'last_active': '2024-09-10T16:04:50.934696+0000', 'last_peered': '2024-09-10T16:04:50.934696+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:04:50.934696+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3526, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 311, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:10.728 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-10T16:05:10.728 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'20718", 'reported_seq': 27796, 'reported_epoch': 64, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:06.756935+0000', 'last_change': '2024-09-10T16:04:49.009212+0000', 'last_active': '2024-09-10T16:05:06.756935+0000', 'last_peered': '2024-09-10T16:05:06.756935+0000', 'last_clean': '2024-09-10T15:53:17.314394+0000', 'last_became_active': '2024-09-10T15:53:19.963477+0000', 'last_became_peered': '2024-09-10T15:53:19.963477+0000', 'last_unstale': '2024-09-10T16:05:06.756935+0000', 'last_undegraded': '2024-09-10T15:53:19.551617+0000', 'last_fullsized': '2024-09-10T15:53:19.468241+0000', 'mapping_epoch': 43, 'log_start': "46'10700", 'ondisk_log_start': "46'10700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3640", 'last_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:13.900343+0000', 'objects_scrubbed': 3574, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1438, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 483, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17136, 'num_write_kb': 27124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3437, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -483}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:10.728 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:11.079 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:11.079 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4009/6 objects degraded (66816.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":837,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1498607616,"bytes_avail":288411684864,"bytes_total":289910292480,"degraded_objects":4009,"degraded_total":6,"degraded_ratio":668.16666666666652,"recovering_objects_per_sec":119,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":953,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:11.092 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:11.092 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:11.395 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:11.396 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:11.409 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-09-10T16:05:10.671284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42641,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463484,"kb_used_data":145484,"kb_used_omap":12,"kb_used_meta":1317939,"kb_avail":281652036,"statfs":{"total":289910292480,"available":288411684864,"internally_reserved":0,"allocated":148975616,"data_stored":147886893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1349569541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001354"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28840,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935129+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:04:50.935129+0000","last_peered":"2024-09-10T16:04:50.935129+0000","last_clean":"2024-09-10T16:04:50.935129+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:04:50.935129+0000","last_undegraded":"2024-09-10T16:04:50.935129+0000","last_fullsized":"2024-09-10T16:04:50.935129+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439460+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:04:51.439460+0000","last_peered":"2024-09-10T16:04:51.439460+0000","last_clean":"2024-09-10T16:04:51.439460+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:04:51.439460+0000","last_undegraded":"2024-09-10T16:04:51.439460+0000","last_fullsized":"2024-09-10T16:04:51.439460+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934145+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:04:50.934145+0000","last_peered":"2024-09-10T16:04:50.934145+0000","last_clean":"2024-09-10T16:04:50.934145+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:04:50.934145+0000","last_undegraded":"2024-09-10T16:04:50.934145+0000","last_fullsized":"2024-09-10T16:04:50.934145+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439440+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:04:51.439440+0000","last_peered":"2024-09-10T16:04:51.439440+0000","last_clean":"2024-09-10T16:04:51.439440+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:04:51.439440+0000","last_undegraded":"2024-09-10T16:04:51.439440+0000","last_fullsized":"2024-09-10T16:04:51.439440+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935603+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:04:50.935603+0000","last_peered":"2024-09-10T16:04:50.935603+0000","last_clean":"2024-09-10T16:04:50.935603+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:04:50.935603+0000","last_undegraded":"2024-09-10T16:04:50.935603+0000","last_fullsized":"2024-09-10T16:04:50.935603+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28795,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935166+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:04:50.935166+0000","last_peered":"2024-09-10T16:04:50.935166+0000","last_clean":"2024-09-10T16:04:50.935166+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:04:50.935166+0000","last_undegraded":"2024-09-10T16:04:50.935166+0000","last_fullsized":"2024-09-10T16:04:50.935166+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21086,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-10T16:04:50.934696+0000","last_change":"2024-09-10T15:53:19.978937+0000","last_active":"2024-09-10T16:04:50.934696+0000","last_peered":"2024-09-10T16:04:50.934696+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:04:50.934696+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21085,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934488+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:04:50.934488+0000","last_peered":"2024-09-10T16:04:50.934488+0000","last_clean":"2024-09-10T16:04:50.934488+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:04:50.934488+0000","last_undegraded":"2024-09-10T16:04:50.934488+0000","last_fullsized":"2024-09-10T16:04:50.934488+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934327+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:04:50.934327+0000","last_peered":"2024-09-10T16:04:50.934327+0000","last_clean":"2024-09-10T16:04:50.934327+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:04:50.934327+0000","last_undegraded":"2024-09-10T16:04:50.934327+0000","last_fullsized":"2024-09-10T16:04:50.934327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28886,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935289+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:04:50.935289+0000","last_peered":"2024-09-10T16:04:50.935289+0000","last_clean":"2024-09-10T16:04:50.935289+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:04:50.935289+0000","last_undegraded":"2024-09-10T16:04:50.935289+0000","last_fullsized":"2024-09-10T16:04:50.935289+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439237+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:04:51.439237+0000","last_peered":"2024-09-10T16:04:51.439237+0000","last_clean":"2024-09-10T16:04:51.439237+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:04:51.439237+0000","last_undegraded":"2024-09-10T16:04:51.439237+0000","last_fullsized":"2024-09-10T16:04:51.439237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935734+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:04:50.935734+0000","last_peered":"2024-09-10T16:04:50.935734+0000","last_clean":"2024-09-10T16:04:50.935734+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:04:50.935734+0000","last_undegraded":"2024-09-10T16:04:50.935734+0000","last_fullsized":"2024-09-10T16:04:50.935734+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935418+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:04:50.935418+0000","last_peered":"2024-09-10T16:04:50.935418+0000","last_clean":"2024-09-10T16:04:50.935418+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:04:50.935418+0000","last_undegraded":"2024-09-10T16:04:50.935418+0000","last_fullsized":"2024-09-10T16:04:50.935418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28625,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439338+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:04:51.439338+0000","last_peered":"2024-09-10T16:04:51.439338+0000","last_clean":"2024-09-10T16:04:51.439338+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:04:51.439338+0000","last_undegraded":"2024-09-10T16:04:51.439338+0000","last_fullsized":"2024-09-10T16:04:51.439338+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":118,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935237+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:04:50.935237+0000","last_peered":"2024-09-10T16:04:50.935237+0000","last_clean":"2024-09-10T16:04:50.935237+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:04:50.935237+0000","last_undegraded":"2024-09-10T16:04:50.935237+0000","last_fullsized":"2024-09-10T16:04:50.935237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28293,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439155+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:04:51.439155+0000","last_peered":"2024-09-10T16:04:51.439155+0000","last_clean":"2024-09-10T16:04:51.439155+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:04:51.439155+0000","last_undegraded":"2024-09-10T16:04:51.439155+0000","last_fullsized":"2024-09-10T16:04:51.439155+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21730,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934197+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:04:50.934197+0000","last_peered":"2024-09-10T16:04:50.934197+0000","last_clean":"2024-09-10T16:04:50.934197+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:04:50.934197+0000","last_undegraded":"2024-09-10T16:04:50.934197+0000","last_fullsized":"2024-09-10T16:04:50.934197+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934756+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:04:50.934756+0000","last_peered":"2024-09-10T16:04:50.934756+0000","last_clean":"2024-09-10T16:04:50.934756+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:04:50.934756+0000","last_undegraded":"2024-09-10T16:04:50.934756+0000","last_fullsized":"2024-09-10T16:04:50.934756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":27796,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:06.756935+0000","last_change":"2024-09-10T16:04:49.009212+0000","last_active":"2024-09-10T16:05:06.756935+0000","last_peered":"2024-09-10T16:05:06.756935+0000","last_clean":"2024-09-10T15:53:17.314394+0000","last_became_active":"2024-09-10T15:53:19.963477+0000","last_became_peered":"2024-09-10T15:53:19.963477+0000","last_unstale":"2024-09-10T16:05:06.756935+0000","last_undegraded":"2024-09-10T15:53:19.551617+0000","last_fullsized":"2024-09-10T15:53:19.468241+0000","mapping_epoch":43,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3437,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-483}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.439547+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:04:51.439547+0000","last_peered":"2024-09-10T16:04:51.439547+0000","last_clean":"2024-09-10T16:04:51.439547+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:04:51.439547+0000","last_undegraded":"2024-09-10T16:04:51.439547+0000","last_fullsized":"2024-09-10T16:04:51.439547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934246+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:04:50.934246+0000","last_peered":"2024-09-10T16:04:50.934246+0000","last_clean":"2024-09-10T16:04:50.934246+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:04:50.934246+0000","last_undegraded":"2024-09-10T16:04:50.934246+0000","last_fullsized":"2024-09-10T16:04:50.934246+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.935283+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:04:50.935283+0000","last_peered":"2024-09-10T16:04:50.935283+0000","last_clean":"2024-09-10T16:04:50.935283+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:04:50.935283+0000","last_undegraded":"2024-09-10T16:04:50.935283+0000","last_fullsized":"2024-09-10T16:04:50.935283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":113,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:50.934097+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:04:50.934097+0000","last_peered":"2024-09-10T16:04:50.934097+0000","last_clean":"2024-09-10T16:04:50.934097+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:04:50.934097+0000","last_undegraded":"2024-09-10T16:04:50.934097+0000","last_fullsized":"2024-09-10T16:04:50.934097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21877,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-10T16:04:51.057122+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:04:51.057122+0000","last_peered":"2024-09-10T16:04:51.057122+0000","last_clean":"2024-09-10T16:04:51.057122+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:04:51.057122+0000","last_undegraded":"2024-09-10T16:04:51.057122+0000","last_fullsized":"2024-09-10T16:04:51.057122+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42639,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147124224,"data_stored":147106159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626579,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497256,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874584,"statfs":{"total":96636764160,"available":96127574016,"internally_reserved":0,"allocated":548864,"data_stored":179780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508620462},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149860,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527112,"kb_used_data":62328,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844728,"statfs":{"total":96636764160,"available":96097001472,"internally_reserved":0,"allocated":63823872,"data_stored":63462014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953631,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439116,"kb_used_data":82620,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93932724,"statfs":{"total":96636764160,"available":96187109376,"internally_reserved":0,"allocated":84602880,"data_stored":84245099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":83824640,"data_stored":83821027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:14.411 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:14.748 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:14.748 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:14.760 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-10T16:05:14.671909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43503,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472288,"kb_used_data":141808,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281643232,"statfs":{"total":289910292480,"available":288402669568,"internally_reserved":0,"allocated":145211392,"data_stored":144097210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1815,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001275"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20501,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961194+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961194+0000","last_peered":"2024-09-10T16:05:09.961194+0000","last_clean":"2024-09-10T16:05:09.961194+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961194+0000","last_undegraded":"2024-09-10T16:05:09.961194+0000","last_fullsized":"2024-09-10T16:05:09.961194+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21849,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:11.760031+0000","last_change":"2024-09-10T16:05:08.649390+0000","last_active":"2024-09-10T16:05:11.760031+0000","last_peered":"2024-09-10T16:05:11.760031+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:11.760031+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":690,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963533+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963533+0000","last_peered":"2024-09-10T16:05:09.963533+0000","last_clean":"2024-09-10T16:05:09.963533+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963533+0000","last_undegraded":"2024-09-10T16:05:09.963533+0000","last_fullsized":"2024-09-10T16:05:09.963533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43501,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143187968,"data_stored":143177129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626580,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505584,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866256,"statfs":{"total":96636764160,"available":96119046144,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517140142},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149861,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953632,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439576,"kb_used_data":78920,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93932264,"statfs":{"total":96636764160,"available":96186638336,"internally_reserved":0,"allocated":80814080,"data_stored":80445404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79896576,"data_stored":79892963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:14.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:15.123 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:15.123 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:15.136 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-10T16:05:14.671909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43503,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472288,"kb_used_data":141808,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281643232,"statfs":{"total":289910292480,"available":288402669568,"internally_reserved":0,"allocated":145211392,"data_stored":144097210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1815,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001275"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20501,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961194+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961194+0000","last_peered":"2024-09-10T16:05:09.961194+0000","last_clean":"2024-09-10T16:05:09.961194+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961194+0000","last_undegraded":"2024-09-10T16:05:09.961194+0000","last_fullsized":"2024-09-10T16:05:09.961194+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21849,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:11.760031+0000","last_change":"2024-09-10T16:05:08.649390+0000","last_active":"2024-09-10T16:05:11.760031+0000","last_peered":"2024-09-10T16:05:11.760031+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:11.760031+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":690,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963533+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963533+0000","last_peered":"2024-09-10T16:05:09.963533+0000","last_clean":"2024-09-10T16:05:09.963533+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963533+0000","last_undegraded":"2024-09-10T16:05:09.963533+0000","last_fullsized":"2024-09-10T16:05:09.963533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43501,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143187968,"data_stored":143177129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626580,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505584,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866256,"statfs":{"total":96636764160,"available":96119046144,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517140142},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149861,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953632,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439576,"kb_used_data":78920,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93932264,"statfs":{"total":96636764160,"available":96186638336,"internally_reserved":0,"allocated":80814080,"data_stored":80445404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79896576,"data_stored":79892963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:15.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:15.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21849, 'reported_epoch': 66, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:11.760031+0000', 'last_change': '2024-09-10T16:05:08.649390+0000', 'last_active': '2024-09-10T16:05:11.760031+0000', 'last_peered': '2024-09-10T16:05:11.760031+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:05:11.760031+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3147, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 690, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:15.137 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:15.508 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:15.509 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3147/6 objects degraded (52450.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":841,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1507622912,"bytes_avail":288402669568,"bytes_total":289910292480,"degraded_objects":3147,"degraded_total":6,"degraded_ratio":524.5,"recovering_objects_per_sec":226,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1815,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:15.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:15.522 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:15.853 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:15.854 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:15.867 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-10T16:05:14.671909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43503,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472288,"kb_used_data":141808,"kb_used_omap":12,"kb_used_meta":1330419,"kb_avail":281643232,"statfs":{"total":289910292480,"available":288402669568,"internally_reserved":0,"allocated":145211392,"data_stored":144097210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1362349061},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1815,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001275"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20501,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961194+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961194+0000","last_peered":"2024-09-10T16:05:09.961194+0000","last_clean":"2024-09-10T16:05:09.961194+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961194+0000","last_undegraded":"2024-09-10T16:05:09.961194+0000","last_fullsized":"2024-09-10T16:05:09.961194+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21849,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:11.760031+0000","last_change":"2024-09-10T16:05:08.649390+0000","last_active":"2024-09-10T16:05:11.760031+0000","last_peered":"2024-09-10T16:05:11.760031+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:11.760031+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":690,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963533+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963533+0000","last_peered":"2024-09-10T16:05:09.963533+0000","last_clean":"2024-09-10T16:05:09.963533+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963533+0000","last_undegraded":"2024-09-10T16:05:09.963533+0000","last_fullsized":"2024-09-10T16:05:09.963533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43501,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143187968,"data_stored":143177129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626580,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505584,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866256,"statfs":{"total":96636764160,"available":96119046144,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517140142},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149861,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953632,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439576,"kb_used_data":78920,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93932264,"statfs":{"total":96636764160,"available":96186638336,"internally_reserved":0,"allocated":80814080,"data_stored":80445404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79896576,"data_stored":79892963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:18.869 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:19.186 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:19.186 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:19.199 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-10T16:05:18.672473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44081,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472248,"kb_used_data":137608,"kb_used_omap":12,"kb_used_meta":1334579,"kb_avail":281643272,"statfs":{"total":289910292480,"available":288402710528,"internally_reserved":0,"allocated":140910592,"data_stored":139796410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1366608901},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1440,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.722326"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20501,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961194+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961194+0000","last_peered":"2024-09-10T16:05:09.961194+0000","last_clean":"2024-09-10T16:05:09.961194+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961194+0000","last_undegraded":"2024-09-10T16:05:09.961194+0000","last_fullsized":"2024-09-10T16:05:09.961194+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":23005,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:16.761323+0000","last_change":"2024-09-10T16:05:08.649390+0000","last_active":"2024-09-10T16:05:16.761323+0000","last_peered":"2024-09-10T16:05:16.761323+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:16.761323+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1268,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963533+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963533+0000","last_peered":"2024-09-10T16:05:09.963533+0000","last_clean":"2024-09-10T16:05:09.963533+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963533+0000","last_undegraded":"2024-09-10T16:05:09.963533+0000","last_fullsized":"2024-09-10T16:05:09.963533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44079,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138887168,"data_stored":138876329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509744,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862096,"statfs":{"total":96636764160,"available":96114786304,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149862,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953633,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435376,"kb_used_data":74720,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93936464,"statfs":{"total":96636764160,"available":96190939136,"internally_reserved":0,"allocated":76513280,"data_stored":76144604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75595776,"data_stored":75592163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:19.201 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:19.502 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:19.503 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:19.515 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-10T16:05:18.672473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44081,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472248,"kb_used_data":137608,"kb_used_omap":12,"kb_used_meta":1334579,"kb_avail":281643272,"statfs":{"total":289910292480,"available":288402710528,"internally_reserved":0,"allocated":140910592,"data_stored":139796410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1366608901},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1440,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.722326"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20501,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961194+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961194+0000","last_peered":"2024-09-10T16:05:09.961194+0000","last_clean":"2024-09-10T16:05:09.961194+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961194+0000","last_undegraded":"2024-09-10T16:05:09.961194+0000","last_fullsized":"2024-09-10T16:05:09.961194+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":23005,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:16.761323+0000","last_change":"2024-09-10T16:05:08.649390+0000","last_active":"2024-09-10T16:05:16.761323+0000","last_peered":"2024-09-10T16:05:16.761323+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:16.761323+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1268,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963533+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963533+0000","last_peered":"2024-09-10T16:05:09.963533+0000","last_clean":"2024-09-10T16:05:09.963533+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963533+0000","last_undegraded":"2024-09-10T16:05:09.963533+0000","last_fullsized":"2024-09-10T16:05:09.963533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44079,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138887168,"data_stored":138876329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509744,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862096,"statfs":{"total":96636764160,"available":96114786304,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149862,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953633,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435376,"kb_used_data":74720,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93936464,"statfs":{"total":96636764160,"available":96190939136,"internally_reserved":0,"allocated":76513280,"data_stored":76144604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75595776,"data_stored":75592163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:19.516 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:19.516 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 23005, 'reported_epoch': 66, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:16.761323+0000', 'last_change': '2024-09-10T16:05:08.649390+0000', 'last_active': '2024-09-10T16:05:16.761323+0000', 'last_peered': '2024-09-10T16:05:16.761323+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:05:16.761323+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2569, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1268, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2569}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:19.516 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:19.903 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:19.904 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2569/6 objects degraded (42816.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":845,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1507581952,"bytes_avail":288402710528,"bytes_total":289910292480,"degraded_objects":2569,"degraded_total":6,"degraded_ratio":428.16666666666674,"recovering_objects_per_sec":148,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1440,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:19.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:19.918 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:20.229 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:20.229 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:20.238 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-10T16:05:18.672473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44081,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472248,"kb_used_data":137608,"kb_used_omap":12,"kb_used_meta":1334579,"kb_avail":281643272,"statfs":{"total":289910292480,"available":288402710528,"internally_reserved":0,"allocated":140910592,"data_stored":139796410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1366608901},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1440,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.722326"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20501,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961194+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961194+0000","last_peered":"2024-09-10T16:05:09.961194+0000","last_clean":"2024-09-10T16:05:09.961194+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961194+0000","last_undegraded":"2024-09-10T16:05:09.961194+0000","last_fullsized":"2024-09-10T16:05:09.961194+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":23005,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:16.761323+0000","last_change":"2024-09-10T16:05:08.649390+0000","last_active":"2024-09-10T16:05:16.761323+0000","last_peered":"2024-09-10T16:05:16.761323+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:16.761323+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1268,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963533+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963533+0000","last_peered":"2024-09-10T16:05:09.963533+0000","last_clean":"2024-09-10T16:05:09.963533+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963533+0000","last_undegraded":"2024-09-10T16:05:09.963533+0000","last_fullsized":"2024-09-10T16:05:09.963533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44079,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138887168,"data_stored":138876329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509744,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862096,"statfs":{"total":96636764160,"available":96114786304,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521399982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149862,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953633,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435376,"kb_used_data":74720,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93936464,"statfs":{"total":96636764160,"available":96190939136,"internally_reserved":0,"allocated":76513280,"data_stored":76144604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75595776,"data_stored":75592163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:23.240 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:23.546 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:23.558 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-10T16:05:22.673088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44790,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477776,"kb_used_data":133408,"kb_used_omap":12,"kb_used_meta":1344307,"kb_avail":281637744,"statfs":{"total":289910292480,"available":288397049856,"internally_reserved":0,"allocated":136609792,"data_stored":135495610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1376570373},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001803"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20501,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961194+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961194+0000","last_peered":"2024-09-10T16:05:09.961194+0000","last_clean":"2024-09-10T16:05:09.961194+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961194+0000","last_undegraded":"2024-09-10T16:05:09.961194+0000","last_fullsized":"2024-09-10T16:05:09.961194+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":24423,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:21.760135+0000","last_change":"2024-09-10T16:05:08.649390+0000","last_active":"2024-09-10T16:05:21.760135+0000","last_peered":"2024-09-10T16:05:21.760135+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:21.760135+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1977,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963580+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963580+0000","last_peered":"2024-09-10T16:05:09.963580+0000","last_clean":"2024-09-10T16:05:09.963580+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963580+0000","last_undegraded":"2024-09-10T16:05:09.963580+0000","last_fullsized":"2024-09-10T16:05:09.963580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44788,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134586368,"data_stored":134575529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626582,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":518907,"kb_avail":93852368,"statfs":{"total":96636764160,"available":96104824832,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":531361454},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149863,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953634,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431176,"kb_used_data":70520,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93940664,"statfs":{"total":96636764160,"available":96195239936,"internally_reserved":0,"allocated":72212480,"data_stored":71843804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":71294976,"data_stored":71291363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:23.559 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:23.884 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:23.885 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:23.898 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-10T16:05:22.673088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44790,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477776,"kb_used_data":133408,"kb_used_omap":12,"kb_used_meta":1344307,"kb_avail":281637744,"statfs":{"total":289910292480,"available":288397049856,"internally_reserved":0,"allocated":136609792,"data_stored":135495610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1376570373},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001803"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20501,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961194+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961194+0000","last_peered":"2024-09-10T16:05:09.961194+0000","last_clean":"2024-09-10T16:05:09.961194+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961194+0000","last_undegraded":"2024-09-10T16:05:09.961194+0000","last_fullsized":"2024-09-10T16:05:09.961194+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":24423,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:21.760135+0000","last_change":"2024-09-10T16:05:08.649390+0000","last_active":"2024-09-10T16:05:21.760135+0000","last_peered":"2024-09-10T16:05:21.760135+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:21.760135+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1977,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963580+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963580+0000","last_peered":"2024-09-10T16:05:09.963580+0000","last_clean":"2024-09-10T16:05:09.963580+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963580+0000","last_undegraded":"2024-09-10T16:05:09.963580+0000","last_fullsized":"2024-09-10T16:05:09.963580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44788,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134586368,"data_stored":134575529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626582,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":518907,"kb_avail":93852368,"statfs":{"total":96636764160,"available":96104824832,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":531361454},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149863,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953634,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431176,"kb_used_data":70520,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93940664,"statfs":{"total":96636764160,"available":96195239936,"internally_reserved":0,"allocated":72212480,"data_stored":71843804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":71294976,"data_stored":71291363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:23.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:23.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 24423, 'reported_epoch': 66, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:21.760135+0000', 'last_change': '2024-09-10T16:05:08.649390+0000', 'last_active': '2024-09-10T16:05:21.760135+0000', 'last_peered': '2024-09-10T16:05:21.760135+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:05:21.760135+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1860, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1977, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1860}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:23.900 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:24.275 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:24.275 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1860/6 objects degraded (31000.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":850,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1513242624,"bytes_avail":288397049856,"bytes_total":289910292480,"degraded_objects":1860,"degraded_total":6,"degraded_ratio":310,"recovering_objects_per_sec":179,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2149,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:24.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:24.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:24.589 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:24.589 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:24.602 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-10T16:05:22.673088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44790,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477776,"kb_used_data":133408,"kb_used_omap":12,"kb_used_meta":1344307,"kb_avail":281637744,"statfs":{"total":289910292480,"available":288397049856,"internally_reserved":0,"allocated":136609792,"data_stored":135495610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1376570373},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001803"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20501,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961194+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961194+0000","last_peered":"2024-09-10T16:05:09.961194+0000","last_clean":"2024-09-10T16:05:09.961194+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961194+0000","last_undegraded":"2024-09-10T16:05:09.961194+0000","last_fullsized":"2024-09-10T16:05:09.961194+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":24423,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:21.760135+0000","last_change":"2024-09-10T16:05:08.649390+0000","last_active":"2024-09-10T16:05:21.760135+0000","last_peered":"2024-09-10T16:05:21.760135+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:21.760135+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1977,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963580+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963580+0000","last_peered":"2024-09-10T16:05:09.963580+0000","last_clean":"2024-09-10T16:05:09.963580+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963580+0000","last_undegraded":"2024-09-10T16:05:09.963580+0000","last_fullsized":"2024-09-10T16:05:09.963580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44788,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134586368,"data_stored":134575529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626582,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":518907,"kb_avail":93852368,"statfs":{"total":96636764160,"available":96104824832,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":531361454},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149863,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953634,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431176,"kb_used_data":70520,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93940664,"statfs":{"total":96636764160,"available":96195239936,"internally_reserved":0,"allocated":72212480,"data_stored":71843804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":71294976,"data_stored":71291363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:27.604 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:27.915 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:27.916 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:27.927 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-10T16:05:26.673715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44790,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473576,"kb_used_data":129208,"kb_used_omap":12,"kb_used_meta":1344307,"kb_avail":281641944,"statfs":{"total":289910292480,"available":288401350656,"internally_reserved":0,"allocated":132308992,"data_stored":131194810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1376570373},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001805"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20501,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961194+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961194+0000","last_peered":"2024-09-10T16:05:09.961194+0000","last_clean":"2024-09-10T16:05:09.961194+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961194+0000","last_undegraded":"2024-09-10T16:05:09.961194+0000","last_fullsized":"2024-09-10T16:05:09.961194+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":24423,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:21.760135+0000","last_change":"2024-09-10T16:05:08.649390+0000","last_active":"2024-09-10T16:05:21.760135+0000","last_peered":"2024-09-10T16:05:21.760135+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:21.760135+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1977,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963580+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963580+0000","last_peered":"2024-09-10T16:05:09.963580+0000","last_clean":"2024-09-10T16:05:09.963580+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963580+0000","last_undegraded":"2024-09-10T16:05:09.963580+0000","last_fullsized":"2024-09-10T16:05:09.963580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44788,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130285568,"data_stored":130274729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626582,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":518907,"kb_avail":93852368,"statfs":{"total":96636764160,"available":96104824832,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":531361454},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149863,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953635,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426976,"kb_used_data":66320,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93944864,"statfs":{"total":96636764160,"available":96199540736,"internally_reserved":0,"allocated":67911680,"data_stored":67543004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66994176,"data_stored":66990563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:27.929 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:28.233 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:28.234 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:28.246 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-10T16:05:26.673715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44790,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473576,"kb_used_data":129208,"kb_used_omap":12,"kb_used_meta":1344307,"kb_avail":281641944,"statfs":{"total":289910292480,"available":288401350656,"internally_reserved":0,"allocated":132308992,"data_stored":131194810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1376570373},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001805"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20501,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961194+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961194+0000","last_peered":"2024-09-10T16:05:09.961194+0000","last_clean":"2024-09-10T16:05:09.961194+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961194+0000","last_undegraded":"2024-09-10T16:05:09.961194+0000","last_fullsized":"2024-09-10T16:05:09.961194+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":24423,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:21.760135+0000","last_change":"2024-09-10T16:05:08.649390+0000","last_active":"2024-09-10T16:05:21.760135+0000","last_peered":"2024-09-10T16:05:21.760135+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:21.760135+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1977,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963580+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963580+0000","last_peered":"2024-09-10T16:05:09.963580+0000","last_clean":"2024-09-10T16:05:09.963580+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963580+0000","last_undegraded":"2024-09-10T16:05:09.963580+0000","last_fullsized":"2024-09-10T16:05:09.963580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44788,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130285568,"data_stored":130274729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626582,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":518907,"kb_avail":93852368,"statfs":{"total":96636764160,"available":96104824832,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":531361454},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149863,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953635,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426976,"kb_used_data":66320,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93944864,"statfs":{"total":96636764160,"available":96199540736,"internally_reserved":0,"allocated":67911680,"data_stored":67543004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66994176,"data_stored":66990563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:28.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:28.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 24423, 'reported_epoch': 66, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-10T16:05:21.760135+0000', 'last_change': '2024-09-10T16:05:08.649390+0000', 'last_active': '2024-09-10T16:05:21.760135+0000', 'last_peered': '2024-09-10T16:05:21.760135+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:05:21.760135+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1860, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1977, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1860}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:28.247 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:28.629 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:28.629 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1860/6 objects degraded (31000.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":854,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1508941824,"bytes_avail":288401350656,"bytes_total":289910292480,"degraded_objects":1860,"degraded_total":6,"degraded_ratio":310,"recovering_objects_per_sec":107,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1287,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:28.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:28.642 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:28.943 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:28.943 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:28.955 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-09-10T16:05:28.674068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45661,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477736,"kb_used_data":129208,"kb_used_omap":12,"kb_used_meta":1348467,"kb_avail":281637784,"statfs":{"total":289910292480,"available":288397090816,"internally_reserved":0,"allocated":132308992,"data_stored":131194810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1380830213},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2158,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001882"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20501,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961194+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961194+0000","last_peered":"2024-09-10T16:05:09.961194+0000","last_clean":"2024-09-10T16:05:09.961194+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961194+0000","last_undegraded":"2024-09-10T16:05:09.961194+0000","last_fullsized":"2024-09-10T16:05:09.961194+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":26166,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-10T16:05:26.762385+0000","last_change":"2024-09-10T16:05:08.649390+0000","last_active":"2024-09-10T16:05:26.762385+0000","last_peered":"2024-09-10T16:05:26.762385+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:26.762385+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2848,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-989}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963580+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963580+0000","last_peered":"2024-09-10T16:05:09.963580+0000","last_clean":"2024-09-10T16:05:09.963580+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963580+0000","last_undegraded":"2024-09-10T16:05:09.963580+0000","last_fullsized":"2024-09-10T16:05:09.963580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45659,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130285568,"data_stored":130274729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626583,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523632,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":523067,"kb_avail":93848208,"statfs":{"total":96636764160,"available":96100564992,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":535621294},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953635,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426976,"kb_used_data":66320,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93944864,"statfs":{"total":96636764160,"available":96199540736,"internally_reserved":0,"allocated":67911680,"data_stored":67543004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63287296,"data_stored":63283683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66994176,"data_stored":66990563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:31.958 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:32.244 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:32.244 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:32.256 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-10T16:05:32.016504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473536,"kb_used_data":125008,"kb_used_omap":12,"kb_used_meta":1348467,"kb_avail":281641984,"statfs":{"total":289910292480,"available":288401391616,"internally_reserved":0,"allocated":128008192,"data_stored":126894010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1380830213},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2569,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.343694"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20502,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961231+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961231+0000","last_peered":"2024-09-10T16:05:09.961231+0000","last_clean":"2024-09-10T16:05:09.961231+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961231+0000","last_undegraded":"2024-09-10T16:05:09.961231+0000","last_fullsized":"2024-09-10T16:05:09.961231+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":28145,"reported_epoch":66,"state":"active+undersized+remapped","last_fresh":"2024-09-10T16:05:31.205448+0000","last_change":"2024-09-10T16:05:31.205617+0000","last_active":"2024-09-10T16:05:31.205448+0000","last_peered":"2024-09-10T16:05:31.205448+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:31.205448+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963580+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963580+0000","last_peered":"2024-09-10T16:05:09.963580+0000","last_clean":"2024-09-10T16:05:09.963580+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963580+0000","last_undegraded":"2024-09-10T16:05:09.963580+0000","last_fullsized":"2024-09-10T16:05:09.963580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62701568,"data_stored":62690729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626583,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523632,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":523067,"kb_avail":93848208,"statfs":{"total":96636764160,"available":96100564992,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":535621294},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953636,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422776,"kb_used_data":62120,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93949064,"statfs":{"total":96636764160,"available":96203841536,"internally_reserved":0,"allocated":63610880,"data_stored":63242204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":62693376,"data_stored":62689763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:32.257 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:32.536 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:32.536 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:32.548 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-10T16:05:32.016504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1473536,"kb_used_data":125008,"kb_used_omap":12,"kb_used_meta":1348467,"kb_avail":281641984,"statfs":{"total":289910292480,"available":288401391616,"internally_reserved":0,"allocated":128008192,"data_stored":126894010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1380830213},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2569,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.343694"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21140,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963871+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:09.963871+0000","last_peered":"2024-09-10T16:05:09.963871+0000","last_clean":"2024-09-10T16:05:09.963871+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:09.963871+0000","last_undegraded":"2024-09-10T16:05:09.963871+0000","last_fullsized":"2024-09-10T16:05:09.963871+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20502,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961231+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961231+0000","last_peered":"2024-09-10T16:05:09.961231+0000","last_clean":"2024-09-10T16:05:09.961231+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961231+0000","last_undegraded":"2024-09-10T16:05:09.961231+0000","last_fullsized":"2024-09-10T16:05:09.961231+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28799,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963678+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:09.963678+0000","last_peered":"2024-09-10T16:05:09.963678+0000","last_clean":"2024-09-10T16:05:09.963678+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:09.963678+0000","last_undegraded":"2024-09-10T16:05:09.963678+0000","last_fullsized":"2024-09-10T16:05:09.963678+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":28145,"reported_epoch":66,"state":"active+undersized+remapped","last_fresh":"2024-09-10T16:05:31.205448+0000","last_change":"2024-09-10T16:05:31.205617+0000","last_active":"2024-09-10T16:05:31.205448+0000","last_peered":"2024-09-10T16:05:31.205448+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:31.205448+0000","last_undegraded":"2024-09-10T15:53:19.530069+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":43,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963580+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:09.963580+0000","last_peered":"2024-09-10T16:05:09.963580+0000","last_clean":"2024-09-10T16:05:09.963580+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:09.963580+0000","last_undegraded":"2024-09-10T16:05:09.963580+0000","last_fullsized":"2024-09-10T16:05:09.963580+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28890,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963685+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:09.963685+0000","last_peered":"2024-09-10T16:05:09.963685+0000","last_clean":"2024-09-10T16:05:09.963685+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:09.963685+0000","last_undegraded":"2024-09-10T16:05:09.963685+0000","last_fullsized":"2024-09-10T16:05:09.963685+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":122,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963588+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:09.963588+0000","last_peered":"2024-09-10T16:05:09.963588+0000","last_clean":"2024-09-10T16:05:09.963588+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:09.963588+0000","last_undegraded":"2024-09-10T16:05:09.963588+0000","last_fullsized":"2024-09-10T16:05:09.963588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21734,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964044+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:09.964044+0000","last_peered":"2024-09-10T16:05:09.964044+0000","last_clean":"2024-09-10T16:05:09.964044+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:09.964044+0000","last_undegraded":"2024-09-10T16:05:09.964044+0000","last_fullsized":"2024-09-10T16:05:09.964044+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963847+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:09.963847+0000","last_peered":"2024-09-10T16:05:09.963847+0000","last_clean":"2024-09-10T16:05:09.963847+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:09.963847+0000","last_undegraded":"2024-09-10T16:05:09.963847+0000","last_fullsized":"2024-09-10T16:05:09.963847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21486,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:10.084334+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:10.084334+0000","last_peered":"2024-09-10T16:05:10.084334+0000","last_clean":"2024-09-10T16:05:10.084334+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:10.084334+0000","last_undegraded":"2024-09-10T16:05:10.084334+0000","last_fullsized":"2024-09-10T16:05:10.084334+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21623,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963947+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:09.963947+0000","last_peered":"2024-09-10T16:05:09.963947+0000","last_clean":"2024-09-10T16:05:09.963947+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:09.963947+0000","last_undegraded":"2024-09-10T16:05:09.963947+0000","last_fullsized":"2024-09-10T16:05:09.963947+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963618+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:09.963618+0000","last_peered":"2024-09-10T16:05:09.963618+0000","last_clean":"2024-09-10T16:05:09.963618+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:09.963618+0000","last_undegraded":"2024-09-10T16:05:09.963618+0000","last_fullsized":"2024-09-10T16:05:09.963618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.964013+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:09.964013+0000","last_peered":"2024-09-10T16:05:09.964013+0000","last_clean":"2024-09-10T16:05:09.964013+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:09.964013+0000","last_undegraded":"2024-09-10T16:05:09.964013+0000","last_fullsized":"2024-09-10T16:05:09.964013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21881,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.963778+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:09.963778+0000","last_peered":"2024-09-10T16:05:09.963778+0000","last_clean":"2024-09-10T16:05:09.963778+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:09.963778+0000","last_undegraded":"2024-09-10T16:05:09.963778+0000","last_fullsized":"2024-09-10T16:05:09.963778+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62701568,"data_stored":62690729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626583,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523632,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":523067,"kb_avail":93848208,"statfs":{"total":96636764160,"available":96100564992,"internally_reserved":0,"allocated":557056,"data_stored":184303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":535621294},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953636,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422776,"kb_used_data":62120,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93949064,"statfs":{"total":96636764160,"available":96203841536,"internally_reserved":0,"allocated":63610880,"data_stored":63242204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":62693376,"data_stored":62689763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:32.549 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:32.549 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 28145, 'reported_epoch': 66, 'state': 'active+undersized+remapped', 'last_fresh': '2024-09-10T16:05:31.205448+0000', 'last_change': '2024-09-10T16:05:31.205617+0000', 'last_active': '2024-09-10T16:05:31.205448+0000', 'last_peered': '2024-09-10T16:05:31.205448+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:05:31.205448+0000', 'last_undegraded': '2024-09-10T15:53:19.530069+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 43, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3837, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '1', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T16:05:32.550 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:32.898 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:32.899 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 989/6 objects degraded (16483.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":858,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1513201664,"bytes_avail":288397090816,"bytes_total":289910292480,"degraded_objects":989,"degraded_total":6,"degraded_ratio":164.83333333333337,"recovering_objects_per_sec":131,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1580,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:32.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:32.912 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:33.219 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:33.219 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:33.231 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2024-09-10T16:05:33.028605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465224,"kb_used_data":125016,"kb_used_omap":12,"kb_used_meta":1340147,"kb_avail":281650296,"statfs":{"total":289910292480,"available":288409903104,"internally_reserved":0,"allocated":128016384,"data_stored":126896740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1372310533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1860,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.355180"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023017+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:32.023017+0000","last_peered":"2024-09-10T16:05:32.023017+0000","last_clean":"2024-09-10T16:05:32.023017+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:32.023017+0000","last_undegraded":"2024-09-10T16:05:32.023017+0000","last_fullsized":"2024-09-10T16:05:32.023017+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20502,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961231+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961231+0000","last_peered":"2024-09-10T16:05:09.961231+0000","last_clean":"2024-09-10T16:05:09.961231+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961231+0000","last_undegraded":"2024-09-10T16:05:09.961231+0000","last_fullsized":"2024-09-10T16:05:09.961231+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28801,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022735+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:32.022735+0000","last_peered":"2024-09-10T16:05:32.022735+0000","last_clean":"2024-09-10T16:05:32.022735+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:32.022735+0000","last_undegraded":"2024-09-10T16:05:32.022735+0000","last_fullsized":"2024-09-10T16:05:32.022735+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":67,"state":"peering","last_fresh":"2024-09-10T16:05:31.205617+0000","last_change":"2024-09-10T16:05:32.598448+0000","last_active":"2024-09-10T16:05:32.598448+0000","last_peered":"2024-09-10T16:05:31.205617+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:31.205617+0000","last_undegraded":"2024-09-10T16:05:31.205617+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":67,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022713+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:32.022713+0000","last_peered":"2024-09-10T16:05:32.022713+0000","last_clean":"2024-09-10T16:05:32.022713+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:32.022713+0000","last_undegraded":"2024-09-10T16:05:32.022713+0000","last_fullsized":"2024-09-10T16:05:32.022713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28892,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022889+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:32.022889+0000","last_peered":"2024-09-10T16:05:32.022889+0000","last_clean":"2024-09-10T16:05:32.022889+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:32.022889+0000","last_undegraded":"2024-09-10T16:05:32.022889+0000","last_fullsized":"2024-09-10T16:05:32.022889+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":124,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022865+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:32.022865+0000","last_peered":"2024-09-10T16:05:32.022865+0000","last_clean":"2024-09-10T16:05:32.022865+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:32.022865+0000","last_undegraded":"2024-09-10T16:05:32.022865+0000","last_fullsized":"2024-09-10T16:05:32.022865+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21736,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023075+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:32.023075+0000","last_peered":"2024-09-10T16:05:32.023075+0000","last_clean":"2024-09-10T16:05:32.023075+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:32.023075+0000","last_undegraded":"2024-09-10T16:05:32.023075+0000","last_fullsized":"2024-09-10T16:05:32.023075+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022998+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:32.022998+0000","last_peered":"2024-09-10T16:05:32.022998+0000","last_clean":"2024-09-10T16:05:32.022998+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:32.022998+0000","last_undegraded":"2024-09-10T16:05:32.022998+0000","last_fullsized":"2024-09-10T16:05:32.022998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21488,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023024+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:32.023024+0000","last_peered":"2024-09-10T16:05:32.023024+0000","last_clean":"2024-09-10T16:05:32.023024+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:32.023024+0000","last_undegraded":"2024-09-10T16:05:32.023024+0000","last_fullsized":"2024-09-10T16:05:32.023024+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21625,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023092+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:32.023092+0000","last_peered":"2024-09-10T16:05:32.023092+0000","last_clean":"2024-09-10T16:05:32.023092+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:32.023092+0000","last_undegraded":"2024-09-10T16:05:32.023092+0000","last_fullsized":"2024-09-10T16:05:32.023092+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022934+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:32.022934+0000","last_peered":"2024-09-10T16:05:32.022934+0000","last_clean":"2024-09-10T16:05:32.022934+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:32.022934+0000","last_undegraded":"2024-09-10T16:05:32.022934+0000","last_fullsized":"2024-09-10T16:05:32.022934+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022987+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:32.022987+0000","last_peered":"2024-09-10T16:05:32.022987+0000","last_clean":"2024-09-10T16:05:32.022987+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:32.022987+0000","last_undegraded":"2024-09-10T16:05:32.022987+0000","last_fullsized":"2024-09-10T16:05:32.022987+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21883,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022925+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:32.022925+0000","last_peered":"2024-09-10T16:05:32.022925+0000","last_clean":"2024-09-10T16:05:32.022925+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:32.022925+0000","last_undegraded":"2024-09-10T16:05:32.022925+0000","last_fullsized":"2024-09-10T16:05:32.022925+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62701568,"data_stored":62690729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626584,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515320,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":514747,"kb_avail":93856520,"statfs":{"total":96636764160,"available":96109076480,"internally_reserved":0,"allocated":565248,"data_stored":187033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":527101614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953636,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422776,"kb_used_data":62120,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93949064,"statfs":{"total":96636764160,"available":96203841536,"internally_reserved":0,"allocated":63610880,"data_stored":63242204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":62693376,"data_stored":62689763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:36.232 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:36.526 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:36.526 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:36.540 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-10T16:05:34.675069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465224,"kb_used_data":125016,"kb_used_omap":12,"kb_used_meta":1340147,"kb_avail":281650296,"statfs":{"total":289910292480,"available":288409903104,"internally_reserved":0,"allocated":128016384,"data_stored":126896740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1372310533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1860,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001354"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21225,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961949+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:09.961949+0000","last_peered":"2024-09-10T16:05:09.961949+0000","last_clean":"2024-09-10T16:05:09.961949+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:09.961949+0000","last_undegraded":"2024-09-10T16:05:09.961949+0000","last_fullsized":"2024-09-10T16:05:09.961949+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023017+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:32.023017+0000","last_peered":"2024-09-10T16:05:32.023017+0000","last_clean":"2024-09-10T16:05:32.023017+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:32.023017+0000","last_undegraded":"2024-09-10T16:05:32.023017+0000","last_fullsized":"2024-09-10T16:05:32.023017+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20805,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961827+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:09.961827+0000","last_peered":"2024-09-10T16:05:09.961827+0000","last_clean":"2024-09-10T16:05:09.961827+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:09.961827+0000","last_undegraded":"2024-09-10T16:05:09.961827+0000","last_fullsized":"2024-09-10T16:05:09.961827+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20502,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961231+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961231+0000","last_peered":"2024-09-10T16:05:09.961231+0000","last_clean":"2024-09-10T16:05:09.961231+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961231+0000","last_undegraded":"2024-09-10T16:05:09.961231+0000","last_fullsized":"2024-09-10T16:05:09.961231+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28801,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022735+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:32.022735+0000","last_peered":"2024-09-10T16:05:32.022735+0000","last_clean":"2024-09-10T16:05:32.022735+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:32.022735+0000","last_undegraded":"2024-09-10T16:05:32.022735+0000","last_fullsized":"2024-09-10T16:05:32.022735+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":67,"state":"peering","last_fresh":"2024-09-10T16:05:31.205617+0000","last_change":"2024-09-10T16:05:32.598448+0000","last_active":"2024-09-10T16:05:32.598448+0000","last_peered":"2024-09-10T16:05:31.205617+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:31.205617+0000","last_undegraded":"2024-09-10T16:05:31.205617+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":67,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022713+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:32.022713+0000","last_peered":"2024-09-10T16:05:32.022713+0000","last_clean":"2024-09-10T16:05:32.022713+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:32.022713+0000","last_undegraded":"2024-09-10T16:05:32.022713+0000","last_fullsized":"2024-09-10T16:05:32.022713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28892,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022889+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:32.022889+0000","last_peered":"2024-09-10T16:05:32.022889+0000","last_clean":"2024-09-10T16:05:32.022889+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:32.022889+0000","last_undegraded":"2024-09-10T16:05:32.022889+0000","last_fullsized":"2024-09-10T16:05:32.022889+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961475+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:09.961475+0000","last_peered":"2024-09-10T16:05:09.961475+0000","last_clean":"2024-09-10T16:05:09.961475+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:09.961475+0000","last_undegraded":"2024-09-10T16:05:09.961475+0000","last_fullsized":"2024-09-10T16:05:09.961475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28629,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961529+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:09.961529+0000","last_peered":"2024-09-10T16:05:09.961529+0000","last_clean":"2024-09-10T16:05:09.961529+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:09.961529+0000","last_undegraded":"2024-09-10T16:05:09.961529+0000","last_fullsized":"2024-09-10T16:05:09.961529+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":124,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022865+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:32.022865+0000","last_peered":"2024-09-10T16:05:32.022865+0000","last_clean":"2024-09-10T16:05:32.022865+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:32.022865+0000","last_undegraded":"2024-09-10T16:05:32.022865+0000","last_fullsized":"2024-09-10T16:05:32.022865+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28297,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961331+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:09.961331+0000","last_peered":"2024-09-10T16:05:09.961331+0000","last_clean":"2024-09-10T16:05:09.961331+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:09.961331+0000","last_undegraded":"2024-09-10T16:05:09.961331+0000","last_fullsized":"2024-09-10T16:05:09.961331+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21736,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023075+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:32.023075+0000","last_peered":"2024-09-10T16:05:32.023075+0000","last_clean":"2024-09-10T16:05:32.023075+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:32.023075+0000","last_undegraded":"2024-09-10T16:05:32.023075+0000","last_fullsized":"2024-09-10T16:05:32.023075+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022998+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:32.022998+0000","last_peered":"2024-09-10T16:05:32.022998+0000","last_clean":"2024-09-10T16:05:32.022998+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:32.022998+0000","last_undegraded":"2024-09-10T16:05:32.022998+0000","last_fullsized":"2024-09-10T16:05:32.022998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21488,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023024+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:32.023024+0000","last_peered":"2024-09-10T16:05:32.023024+0000","last_clean":"2024-09-10T16:05:32.023024+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:32.023024+0000","last_undegraded":"2024-09-10T16:05:32.023024+0000","last_fullsized":"2024-09-10T16:05:32.023024+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":138,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961844+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:09.961844+0000","last_peered":"2024-09-10T16:05:09.961844+0000","last_clean":"2024-09-10T16:05:09.961844+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:09.961844+0000","last_undegraded":"2024-09-10T16:05:09.961844+0000","last_fullsized":"2024-09-10T16:05:09.961844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21625,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023092+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:32.023092+0000","last_peered":"2024-09-10T16:05:32.023092+0000","last_clean":"2024-09-10T16:05:32.023092+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:32.023092+0000","last_undegraded":"2024-09-10T16:05:32.023092+0000","last_fullsized":"2024-09-10T16:05:32.023092+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022934+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:32.022934+0000","last_peered":"2024-09-10T16:05:32.022934+0000","last_clean":"2024-09-10T16:05:32.022934+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:32.022934+0000","last_undegraded":"2024-09-10T16:05:32.022934+0000","last_fullsized":"2024-09-10T16:05:32.022934+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022987+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:32.022987+0000","last_peered":"2024-09-10T16:05:32.022987+0000","last_clean":"2024-09-10T16:05:32.022987+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:32.022987+0000","last_undegraded":"2024-09-10T16:05:32.022987+0000","last_fullsized":"2024-09-10T16:05:32.022987+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21883,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022925+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:32.022925+0000","last_peered":"2024-09-10T16:05:32.022925+0000","last_clean":"2024-09-10T16:05:32.022925+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:32.022925+0000","last_undegraded":"2024-09-10T16:05:32.022925+0000","last_fullsized":"2024-09-10T16:05:32.022925+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62701568,"data_stored":62690729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626584,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515320,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":514747,"kb_avail":93856520,"statfs":{"total":96636764160,"available":96109076480,"internally_reserved":0,"allocated":565248,"data_stored":187033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":527101614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953636,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422776,"kb_used_data":62120,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93949064,"statfs":{"total":96636764160,"available":96203841536,"internally_reserved":0,"allocated":63610880,"data_stored":63242204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":62693376,"data_stored":62689763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:36.541 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:36.822 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:36.823 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:36.834 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-09-10T16:05:36.675503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460564,"kb_used_data":120356,"kb_used_omap":12,"kb_used_meta":1340147,"kb_avail":281654956,"statfs":{"total":289910292480,"available":288414674944,"internally_reserved":0,"allocated":123244544,"data_stored":122113955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1372310533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":989,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001435"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21229,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053581+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:34.053581+0000","last_peered":"2024-09-10T16:05:34.053581+0000","last_clean":"2024-09-10T16:05:34.053581+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:34.053581+0000","last_undegraded":"2024-09-10T16:05:34.053581+0000","last_fullsized":"2024-09-10T16:05:34.053581+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023017+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:32.023017+0000","last_peered":"2024-09-10T16:05:32.023017+0000","last_clean":"2024-09-10T16:05:32.023017+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:32.023017+0000","last_undegraded":"2024-09-10T16:05:32.023017+0000","last_fullsized":"2024-09-10T16:05:32.023017+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20809,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053494+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:34.053494+0000","last_peered":"2024-09-10T16:05:34.053494+0000","last_clean":"2024-09-10T16:05:34.053494+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:34.053494+0000","last_undegraded":"2024-09-10T16:05:34.053494+0000","last_fullsized":"2024-09-10T16:05:34.053494+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20502,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961231+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961231+0000","last_peered":"2024-09-10T16:05:09.961231+0000","last_clean":"2024-09-10T16:05:09.961231+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961231+0000","last_undegraded":"2024-09-10T16:05:09.961231+0000","last_fullsized":"2024-09-10T16:05:09.961231+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28801,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022735+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:32.022735+0000","last_peered":"2024-09-10T16:05:32.022735+0000","last_clean":"2024-09-10T16:05:32.022735+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:32.022735+0000","last_undegraded":"2024-09-10T16:05:32.022735+0000","last_fullsized":"2024-09-10T16:05:32.022735+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":67,"state":"peering","last_fresh":"2024-09-10T16:05:31.205617+0000","last_change":"2024-09-10T16:05:32.598448+0000","last_active":"2024-09-10T16:05:32.598448+0000","last_peered":"2024-09-10T16:05:31.205617+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:31.205617+0000","last_undegraded":"2024-09-10T16:05:31.205617+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":67,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022713+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:32.022713+0000","last_peered":"2024-09-10T16:05:32.022713+0000","last_clean":"2024-09-10T16:05:32.022713+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:32.022713+0000","last_undegraded":"2024-09-10T16:05:32.022713+0000","last_fullsized":"2024-09-10T16:05:32.022713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28892,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022889+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:32.022889+0000","last_peered":"2024-09-10T16:05:32.022889+0000","last_clean":"2024-09-10T16:05:32.022889+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:32.022889+0000","last_undegraded":"2024-09-10T16:05:32.022889+0000","last_fullsized":"2024-09-10T16:05:32.022889+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053546+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:34.053546+0000","last_peered":"2024-09-10T16:05:34.053546+0000","last_clean":"2024-09-10T16:05:34.053546+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:34.053546+0000","last_undegraded":"2024-09-10T16:05:34.053546+0000","last_fullsized":"2024-09-10T16:05:34.053546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28633,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053571+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:34.053571+0000","last_peered":"2024-09-10T16:05:34.053571+0000","last_clean":"2024-09-10T16:05:34.053571+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:34.053571+0000","last_undegraded":"2024-09-10T16:05:34.053571+0000","last_fullsized":"2024-09-10T16:05:34.053571+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":124,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022865+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:32.022865+0000","last_peered":"2024-09-10T16:05:32.022865+0000","last_clean":"2024-09-10T16:05:32.022865+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:32.022865+0000","last_undegraded":"2024-09-10T16:05:32.022865+0000","last_fullsized":"2024-09-10T16:05:32.022865+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28301,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.050988+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:34.050988+0000","last_peered":"2024-09-10T16:05:34.050988+0000","last_clean":"2024-09-10T16:05:34.050988+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:34.050988+0000","last_undegraded":"2024-09-10T16:05:34.050988+0000","last_fullsized":"2024-09-10T16:05:34.050988+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21736,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023075+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:32.023075+0000","last_peered":"2024-09-10T16:05:32.023075+0000","last_clean":"2024-09-10T16:05:32.023075+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:32.023075+0000","last_undegraded":"2024-09-10T16:05:32.023075+0000","last_fullsized":"2024-09-10T16:05:32.023075+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022998+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:32.022998+0000","last_peered":"2024-09-10T16:05:32.022998+0000","last_clean":"2024-09-10T16:05:32.022998+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:32.022998+0000","last_undegraded":"2024-09-10T16:05:32.022998+0000","last_fullsized":"2024-09-10T16:05:32.022998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21488,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023024+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:32.023024+0000","last_peered":"2024-09-10T16:05:32.023024+0000","last_clean":"2024-09-10T16:05:32.023024+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:32.023024+0000","last_undegraded":"2024-09-10T16:05:32.023024+0000","last_fullsized":"2024-09-10T16:05:32.023024+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053583+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:34.053583+0000","last_peered":"2024-09-10T16:05:34.053583+0000","last_clean":"2024-09-10T16:05:34.053583+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:34.053583+0000","last_undegraded":"2024-09-10T16:05:34.053583+0000","last_fullsized":"2024-09-10T16:05:34.053583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21625,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023092+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:32.023092+0000","last_peered":"2024-09-10T16:05:32.023092+0000","last_clean":"2024-09-10T16:05:32.023092+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:32.023092+0000","last_undegraded":"2024-09-10T16:05:32.023092+0000","last_fullsized":"2024-09-10T16:05:32.023092+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022934+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:32.022934+0000","last_peered":"2024-09-10T16:05:32.022934+0000","last_clean":"2024-09-10T16:05:32.022934+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:32.022934+0000","last_undegraded":"2024-09-10T16:05:32.022934+0000","last_fullsized":"2024-09-10T16:05:32.022934+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022987+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:32.022987+0000","last_peered":"2024-09-10T16:05:32.022987+0000","last_clean":"2024-09-10T16:05:32.022987+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:32.022987+0000","last_undegraded":"2024-09-10T16:05:32.022987+0000","last_fullsized":"2024-09-10T16:05:32.022987+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21883,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022925+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:32.022925+0000","last_peered":"2024-09-10T16:05:32.022925+0000","last_clean":"2024-09-10T16:05:32.022925+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:32.022925+0000","last_undegraded":"2024-09-10T16:05:32.022925+0000","last_fullsized":"2024-09-10T16:05:32.022925+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58036224,"data_stored":58025385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626584,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515320,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":514747,"kb_avail":93856520,"statfs":{"total":96636764160,"available":96109076480,"internally_reserved":0,"allocated":565248,"data_stored":187033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":527101614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418116,"kb_used_data":57460,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93953724,"statfs":{"total":96636764160,"available":96208613376,"internally_reserved":0,"allocated":58839040,"data_stored":58459419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58028032,"data_stored":58024419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:36.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-10T16:05:36.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'20286", 'reported_seq': 21061, 'reported_epoch': 67, 'state': 'peering', 'last_fresh': '2024-09-10T16:05:31.205617+0000', 'last_change': '2024-09-10T16:05:32.598448+0000', 'last_active': '2024-09-10T16:05:32.598448+0000', 'last_peered': '2024-09-10T16:05:31.205617+0000', 'last_clean': '2024-09-10T15:53:17.304979+0000', 'last_became_active': '2024-09-10T15:53:19.964302+0000', 'last_became_peered': '2024-09-10T15:53:19.964302+0000', 'last_unstale': '2024-09-10T16:05:31.205617+0000', 'last_undegraded': '2024-09-10T16:05:31.205617+0000', 'last_fullsized': '2024-09-10T15:53:19.468171+0000', 'mapping_epoch': 67, 'log_start': "46'10200", 'ondisk_log_start': "46'10200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'3393", 'last_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T15:51:40.844370+0000', 'last_clean_scrub_stamp': '2024-09-10T15:53:09.068900+0000', 'objects_scrubbed': 3344, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1525, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16936, 'num_write_kb': 27184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3837, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T16:05:36.836 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T16:05:37.187 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:37.187 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":863,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":68,"num_osds":3,"num_up_osds":3,"osd_up_since":1725983597,"num_in_osds":3,"osd_in_since":1725983590,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1495617536,"bytes_avail":288414674944,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"recovering_objects_per_sec":123,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":989,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-10T15:51:14:059711+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-10T16:03:30.654499+0000","services":{}},"progress_events":{}} 2024-09-10T16:05:37.200 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-10T16:05:37.200 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:37.479 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:37.479 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:37.491 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-09-10T16:05:36.675503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460564,"kb_used_data":120356,"kb_used_omap":12,"kb_used_meta":1340147,"kb_avail":281654956,"statfs":{"total":289910292480,"available":288414674944,"internally_reserved":0,"allocated":123244544,"data_stored":122113955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1372310533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":989,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001435"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28844,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.962276+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:09.962276+0000","last_peered":"2024-09-10T16:05:09.962276+0000","last_clean":"2024-09-10T16:05:09.962276+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:09.962276+0000","last_undegraded":"2024-09-10T16:05:09.962276+0000","last_fullsized":"2024-09-10T16:05:09.962276+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21229,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053581+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:34.053581+0000","last_peered":"2024-09-10T16:05:34.053581+0000","last_clean":"2024-09-10T16:05:34.053581+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:34.053581+0000","last_undegraded":"2024-09-10T16:05:34.053581+0000","last_fullsized":"2024-09-10T16:05:34.053581+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21142,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023017+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:32.023017+0000","last_peered":"2024-09-10T16:05:32.023017+0000","last_clean":"2024-09-10T16:05:32.023017+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:32.023017+0000","last_undegraded":"2024-09-10T16:05:32.023017+0000","last_fullsized":"2024-09-10T16:05:32.023017+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20809,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053494+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:34.053494+0000","last_peered":"2024-09-10T16:05:34.053494+0000","last_clean":"2024-09-10T16:05:34.053494+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:34.053494+0000","last_undegraded":"2024-09-10T16:05:34.053494+0000","last_fullsized":"2024-09-10T16:05:34.053494+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20502,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961231+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:09.961231+0000","last_peered":"2024-09-10T16:05:09.961231+0000","last_clean":"2024-09-10T16:05:09.961231+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:09.961231+0000","last_undegraded":"2024-09-10T16:05:09.961231+0000","last_fullsized":"2024-09-10T16:05:09.961231+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28801,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022735+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:32.022735+0000","last_peered":"2024-09-10T16:05:32.022735+0000","last_clean":"2024-09-10T16:05:32.022735+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:32.022735+0000","last_undegraded":"2024-09-10T16:05:32.022735+0000","last_fullsized":"2024-09-10T16:05:32.022735+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21061,"reported_epoch":67,"state":"peering","last_fresh":"2024-09-10T16:05:31.205617+0000","last_change":"2024-09-10T16:05:32.598448+0000","last_active":"2024-09-10T16:05:32.598448+0000","last_peered":"2024-09-10T16:05:31.205617+0000","last_clean":"2024-09-10T15:53:17.304979+0000","last_became_active":"2024-09-10T15:53:19.964302+0000","last_became_peered":"2024-09-10T15:53:19.964302+0000","last_unstale":"2024-09-10T16:05:31.205617+0000","last_undegraded":"2024-09-10T16:05:31.205617+0000","last_fullsized":"2024-09-10T15:53:19.468171+0000","mapping_epoch":67,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21089,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960897+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:09.960897+0000","last_peered":"2024-09-10T16:05:09.960897+0000","last_clean":"2024-09-10T16:05:09.960897+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:09.960897+0000","last_undegraded":"2024-09-10T16:05:09.960897+0000","last_fullsized":"2024-09-10T16:05:09.960897+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022713+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:32.022713+0000","last_peered":"2024-09-10T16:05:32.022713+0000","last_clean":"2024-09-10T16:05:32.022713+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:32.022713+0000","last_undegraded":"2024-09-10T16:05:32.022713+0000","last_fullsized":"2024-09-10T16:05:32.022713+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28892,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022889+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:32.022889+0000","last_peered":"2024-09-10T16:05:32.022889+0000","last_clean":"2024-09-10T16:05:32.022889+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:32.022889+0000","last_undegraded":"2024-09-10T16:05:32.022889+0000","last_fullsized":"2024-09-10T16:05:32.022889+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053546+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:34.053546+0000","last_peered":"2024-09-10T16:05:34.053546+0000","last_clean":"2024-09-10T16:05:34.053546+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:34.053546+0000","last_undegraded":"2024-09-10T16:05:34.053546+0000","last_fullsized":"2024-09-10T16:05:34.053546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.961051+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:09.961051+0000","last_peered":"2024-09-10T16:05:09.961051+0000","last_clean":"2024-09-10T16:05:09.961051+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:09.961051+0000","last_undegraded":"2024-09-10T16:05:09.961051+0000","last_fullsized":"2024-09-10T16:05:09.961051+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T16:05:09.960992+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:09.960992+0000","last_peered":"2024-09-10T16:05:09.960992+0000","last_clean":"2024-09-10T16:05:09.960992+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:09.960992+0000","last_undegraded":"2024-09-10T16:05:09.960992+0000","last_fullsized":"2024-09-10T16:05:09.960992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28633,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053571+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:34.053571+0000","last_peered":"2024-09-10T16:05:34.053571+0000","last_clean":"2024-09-10T16:05:34.053571+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:34.053571+0000","last_undegraded":"2024-09-10T16:05:34.053571+0000","last_fullsized":"2024-09-10T16:05:34.053571+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":124,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022865+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:32.022865+0000","last_peered":"2024-09-10T16:05:32.022865+0000","last_clean":"2024-09-10T16:05:32.022865+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:32.022865+0000","last_undegraded":"2024-09-10T16:05:32.022865+0000","last_fullsized":"2024-09-10T16:05:32.022865+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28301,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.050988+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:34.050988+0000","last_peered":"2024-09-10T16:05:34.050988+0000","last_clean":"2024-09-10T16:05:34.050988+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:34.050988+0000","last_undegraded":"2024-09-10T16:05:34.050988+0000","last_fullsized":"2024-09-10T16:05:34.050988+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21736,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023075+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:32.023075+0000","last_peered":"2024-09-10T16:05:32.023075+0000","last_clean":"2024-09-10T16:05:32.023075+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:32.023075+0000","last_undegraded":"2024-09-10T16:05:32.023075+0000","last_fullsized":"2024-09-10T16:05:32.023075+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022998+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:32.022998+0000","last_peered":"2024-09-10T16:05:32.022998+0000","last_clean":"2024-09-10T16:05:32.022998+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:32.022998+0000","last_undegraded":"2024-09-10T16:05:32.022998+0000","last_fullsized":"2024-09-10T16:05:32.022998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21488,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023024+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:32.023024+0000","last_peered":"2024-09-10T16:05:32.023024+0000","last_clean":"2024-09-10T16:05:32.023024+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:32.023024+0000","last_undegraded":"2024-09-10T16:05:32.023024+0000","last_fullsized":"2024-09-10T16:05:32.023024+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053583+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:34.053583+0000","last_peered":"2024-09-10T16:05:34.053583+0000","last_clean":"2024-09-10T16:05:34.053583+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:34.053583+0000","last_undegraded":"2024-09-10T16:05:34.053583+0000","last_fullsized":"2024-09-10T16:05:34.053583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21625,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.023092+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:32.023092+0000","last_peered":"2024-09-10T16:05:32.023092+0000","last_clean":"2024-09-10T16:05:32.023092+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:32.023092+0000","last_undegraded":"2024-09-10T16:05:32.023092+0000","last_fullsized":"2024-09-10T16:05:32.023092+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022934+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:32.022934+0000","last_peered":"2024-09-10T16:05:32.022934+0000","last_clean":"2024-09-10T16:05:32.022934+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:32.022934+0000","last_undegraded":"2024-09-10T16:05:32.022934+0000","last_fullsized":"2024-09-10T16:05:32.022934+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022987+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:32.022987+0000","last_peered":"2024-09-10T16:05:32.022987+0000","last_clean":"2024-09-10T16:05:32.022987+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:32.022987+0000","last_undegraded":"2024-09-10T16:05:32.022987+0000","last_fullsized":"2024-09-10T16:05:32.022987+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21883,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T16:05:32.022925+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:32.022925+0000","last_peered":"2024-09-10T16:05:32.022925+0000","last_clean":"2024-09-10T16:05:32.022925+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:32.022925+0000","last_undegraded":"2024-09-10T16:05:32.022925+0000","last_fullsized":"2024-09-10T16:05:32.022925+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58036224,"data_stored":58025385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626584,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515320,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":514747,"kb_avail":93856520,"statfs":{"total":96636764160,"available":96109076480,"internally_reserved":0,"allocated":565248,"data_stored":187033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":527101614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527128,"kb_used_data":62344,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93844712,"statfs":{"total":96636764160,"available":96096985088,"internally_reserved":0,"allocated":63840256,"data_stored":63467503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":475917997},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418116,"kb_used_data":57460,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93953724,"statfs":{"total":96636764160,"available":96208613376,"internally_reserved":0,"allocated":58839040,"data_stored":58459419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58028032,"data_stored":58024419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:40.494 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:40.787 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:40.788 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:40.799 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-10T16:05:40.676267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1439044,"kb_used_data":58580,"kb_used_omap":12,"kb_used_meta":1380403,"kb_avail":281676476,"statfs":{"total":289910292480,"available":288436711424,"internally_reserved":0,"allocated":59985920,"data_stored":58838903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1413532677},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.659763"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28848,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030155+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:33.030155+0000","last_peered":"2024-09-10T16:05:33.030155+0000","last_clean":"2024-09-10T16:05:33.030155+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:33.030155+0000","last_undegraded":"2024-09-10T16:05:33.030155+0000","last_fullsized":"2024-09-10T16:05:33.030155+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21229,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053581+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:34.053581+0000","last_peered":"2024-09-10T16:05:34.053581+0000","last_clean":"2024-09-10T16:05:34.053581+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:34.053581+0000","last_undegraded":"2024-09-10T16:05:34.053581+0000","last_fullsized":"2024-09-10T16:05:34.053581+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21144,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032927+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:33.032927+0000","last_peered":"2024-09-10T16:05:33.032927+0000","last_clean":"2024-09-10T16:05:33.032927+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:33.032927+0000","last_undegraded":"2024-09-10T16:05:33.032927+0000","last_fullsized":"2024-09-10T16:05:33.032927+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20809,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053494+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:34.053494+0000","last_peered":"2024-09-10T16:05:34.053494+0000","last_clean":"2024-09-10T16:05:34.053494+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:34.053494+0000","last_undegraded":"2024-09-10T16:05:34.053494+0000","last_fullsized":"2024-09-10T16:05:34.053494+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20506,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029866+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:33.029866+0000","last_peered":"2024-09-10T16:05:33.029866+0000","last_clean":"2024-09-10T16:05:33.029866+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:33.029866+0000","last_undegraded":"2024-09-10T16:05:33.029866+0000","last_fullsized":"2024-09-10T16:05:33.029866+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28803,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029811+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:33.029811+0000","last_peered":"2024-09-10T16:05:33.029811+0000","last_clean":"2024-09-10T16:05:33.029811+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:33.029811+0000","last_undegraded":"2024-09-10T16:05:33.029811+0000","last_fullsized":"2024-09-10T16:05:33.029811+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.121371+0000","last_change":"2024-09-10T16:05:33.121269+0000","last_active":"2024-09-10T16:05:33.121371+0000","last_peered":"2024-09-10T16:05:33.121371+0000","last_clean":"2024-09-10T16:05:33.121371+0000","last_became_active":"2024-09-10T16:05:33.033927+0000","last_became_peered":"2024-09-10T16:05:33.033927+0000","last_unstale":"2024-09-10T16:05:33.121371+0000","last_undegraded":"2024-09-10T16:05:33.121371+0000","last_fullsized":"2024-09-10T16:05:33.121371+0000","mapping_epoch":67,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:18.459494+0000","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21093,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029997+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:33.029997+0000","last_peered":"2024-09-10T16:05:33.029997+0000","last_clean":"2024-09-10T16:05:33.029997+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:33.029997+0000","last_undegraded":"2024-09-10T16:05:33.029997+0000","last_fullsized":"2024-09-10T16:05:33.029997+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029800+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:33.029800+0000","last_peered":"2024-09-10T16:05:33.029800+0000","last_clean":"2024-09-10T16:05:33.029800+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:33.029800+0000","last_undegraded":"2024-09-10T16:05:33.029800+0000","last_fullsized":"2024-09-10T16:05:33.029800+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28894,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032865+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:33.032865+0000","last_peered":"2024-09-10T16:05:33.032865+0000","last_clean":"2024-09-10T16:05:33.032865+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:33.032865+0000","last_undegraded":"2024-09-10T16:05:33.032865+0000","last_fullsized":"2024-09-10T16:05:33.032865+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053546+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:34.053546+0000","last_peered":"2024-09-10T16:05:34.053546+0000","last_clean":"2024-09-10T16:05:34.053546+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:34.053546+0000","last_undegraded":"2024-09-10T16:05:34.053546+0000","last_fullsized":"2024-09-10T16:05:34.053546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030391+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:33.030391+0000","last_peered":"2024-09-10T16:05:33.030391+0000","last_clean":"2024-09-10T16:05:33.030391+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:33.030391+0000","last_undegraded":"2024-09-10T16:05:33.030391+0000","last_fullsized":"2024-09-10T16:05:33.030391+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030254+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:33.030254+0000","last_peered":"2024-09-10T16:05:33.030254+0000","last_clean":"2024-09-10T16:05:33.030254+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:33.030254+0000","last_undegraded":"2024-09-10T16:05:33.030254+0000","last_fullsized":"2024-09-10T16:05:33.030254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28633,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053571+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:34.053571+0000","last_peered":"2024-09-10T16:05:34.053571+0000","last_clean":"2024-09-10T16:05:34.053571+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:34.053571+0000","last_undegraded":"2024-09-10T16:05:34.053571+0000","last_fullsized":"2024-09-10T16:05:34.053571+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032856+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:33.032856+0000","last_peered":"2024-09-10T16:05:33.032856+0000","last_clean":"2024-09-10T16:05:33.032856+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:33.032856+0000","last_undegraded":"2024-09-10T16:05:33.032856+0000","last_fullsized":"2024-09-10T16:05:33.032856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28301,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.050988+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:34.050988+0000","last_peered":"2024-09-10T16:05:34.050988+0000","last_clean":"2024-09-10T16:05:34.050988+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:34.050988+0000","last_undegraded":"2024-09-10T16:05:34.050988+0000","last_fullsized":"2024-09-10T16:05:34.050988+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21739,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033096+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:33.033096+0000","last_peered":"2024-09-10T16:05:33.033096+0000","last_clean":"2024-09-10T16:05:33.033096+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:33.033096+0000","last_undegraded":"2024-09-10T16:05:33.033096+0000","last_fullsized":"2024-09-10T16:05:33.033096+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21490,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033149+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:33.033149+0000","last_peered":"2024-09-10T16:05:33.033149+0000","last_clean":"2024-09-10T16:05:33.033149+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:33.033149+0000","last_undegraded":"2024-09-10T16:05:33.033149+0000","last_fullsized":"2024-09-10T16:05:33.033149+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053583+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:34.053583+0000","last_peered":"2024-09-10T16:05:34.053583+0000","last_clean":"2024-09-10T16:05:34.053583+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:34.053583+0000","last_undegraded":"2024-09-10T16:05:34.053583+0000","last_fullsized":"2024-09-10T16:05:34.053583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21627,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032971+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:33.032971+0000","last_peered":"2024-09-10T16:05:33.032971+0000","last_clean":"2024-09-10T16:05:33.032971+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:33.032971+0000","last_undegraded":"2024-09-10T16:05:33.032971+0000","last_fullsized":"2024-09-10T16:05:33.032971+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033067+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:33.033067+0000","last_peered":"2024-09-10T16:05:33.033067+0000","last_clean":"2024-09-10T16:05:33.033067+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:33.033067+0000","last_undegraded":"2024-09-10T16:05:33.033067+0000","last_fullsized":"2024-09-10T16:05:33.033067+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":121,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033040+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:33.033040+0000","last_peered":"2024-09-10T16:05:33.033040+0000","last_clean":"2024-09-10T16:05:33.033040+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:33.033040+0000","last_undegraded":"2024-09-10T16:05:33.033040+0000","last_fullsized":"2024-09-10T16:05:33.033040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21885,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58036224,"data_stored":58025385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515328,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":514747,"kb_avail":93856512,"statfs":{"total":96636764160,"available":96109068288,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":527101614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505600,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866240,"statfs":{"total":96636764160,"available":96119029760,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":517140141},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418116,"kb_used_data":57460,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93953724,"statfs":{"total":96636764160,"available":96208613376,"internally_reserved":0,"allocated":58839040,"data_stored":58459419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58028032,"data_stored":58024419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:40.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-10T16:05:40.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-10T16:05:40.801 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:41.082 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:41.083 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:41.094 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-10T16:05:40.676267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1439044,"kb_used_data":58580,"kb_used_omap":12,"kb_used_meta":1380403,"kb_avail":281676476,"statfs":{"total":289910292480,"available":288436711424,"internally_reserved":0,"allocated":59985920,"data_stored":58838903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1413532677},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.659763"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28848,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030155+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:33.030155+0000","last_peered":"2024-09-10T16:05:33.030155+0000","last_clean":"2024-09-10T16:05:33.030155+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:33.030155+0000","last_undegraded":"2024-09-10T16:05:33.030155+0000","last_fullsized":"2024-09-10T16:05:33.030155+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21229,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053581+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:34.053581+0000","last_peered":"2024-09-10T16:05:34.053581+0000","last_clean":"2024-09-10T16:05:34.053581+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:34.053581+0000","last_undegraded":"2024-09-10T16:05:34.053581+0000","last_fullsized":"2024-09-10T16:05:34.053581+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21144,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032927+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:33.032927+0000","last_peered":"2024-09-10T16:05:33.032927+0000","last_clean":"2024-09-10T16:05:33.032927+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:33.032927+0000","last_undegraded":"2024-09-10T16:05:33.032927+0000","last_fullsized":"2024-09-10T16:05:33.032927+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20809,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053494+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:34.053494+0000","last_peered":"2024-09-10T16:05:34.053494+0000","last_clean":"2024-09-10T16:05:34.053494+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:34.053494+0000","last_undegraded":"2024-09-10T16:05:34.053494+0000","last_fullsized":"2024-09-10T16:05:34.053494+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20506,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029866+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:33.029866+0000","last_peered":"2024-09-10T16:05:33.029866+0000","last_clean":"2024-09-10T16:05:33.029866+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:33.029866+0000","last_undegraded":"2024-09-10T16:05:33.029866+0000","last_fullsized":"2024-09-10T16:05:33.029866+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28803,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029811+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:33.029811+0000","last_peered":"2024-09-10T16:05:33.029811+0000","last_clean":"2024-09-10T16:05:33.029811+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:33.029811+0000","last_undegraded":"2024-09-10T16:05:33.029811+0000","last_fullsized":"2024-09-10T16:05:33.029811+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.121371+0000","last_change":"2024-09-10T16:05:33.121269+0000","last_active":"2024-09-10T16:05:33.121371+0000","last_peered":"2024-09-10T16:05:33.121371+0000","last_clean":"2024-09-10T16:05:33.121371+0000","last_became_active":"2024-09-10T16:05:33.033927+0000","last_became_peered":"2024-09-10T16:05:33.033927+0000","last_unstale":"2024-09-10T16:05:33.121371+0000","last_undegraded":"2024-09-10T16:05:33.121371+0000","last_fullsized":"2024-09-10T16:05:33.121371+0000","mapping_epoch":67,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:18.459494+0000","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21093,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029997+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:33.029997+0000","last_peered":"2024-09-10T16:05:33.029997+0000","last_clean":"2024-09-10T16:05:33.029997+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:33.029997+0000","last_undegraded":"2024-09-10T16:05:33.029997+0000","last_fullsized":"2024-09-10T16:05:33.029997+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029800+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:33.029800+0000","last_peered":"2024-09-10T16:05:33.029800+0000","last_clean":"2024-09-10T16:05:33.029800+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:33.029800+0000","last_undegraded":"2024-09-10T16:05:33.029800+0000","last_fullsized":"2024-09-10T16:05:33.029800+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28894,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032865+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:33.032865+0000","last_peered":"2024-09-10T16:05:33.032865+0000","last_clean":"2024-09-10T16:05:33.032865+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:33.032865+0000","last_undegraded":"2024-09-10T16:05:33.032865+0000","last_fullsized":"2024-09-10T16:05:33.032865+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053546+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:34.053546+0000","last_peered":"2024-09-10T16:05:34.053546+0000","last_clean":"2024-09-10T16:05:34.053546+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:34.053546+0000","last_undegraded":"2024-09-10T16:05:34.053546+0000","last_fullsized":"2024-09-10T16:05:34.053546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030391+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:33.030391+0000","last_peered":"2024-09-10T16:05:33.030391+0000","last_clean":"2024-09-10T16:05:33.030391+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:33.030391+0000","last_undegraded":"2024-09-10T16:05:33.030391+0000","last_fullsized":"2024-09-10T16:05:33.030391+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030254+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:33.030254+0000","last_peered":"2024-09-10T16:05:33.030254+0000","last_clean":"2024-09-10T16:05:33.030254+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:33.030254+0000","last_undegraded":"2024-09-10T16:05:33.030254+0000","last_fullsized":"2024-09-10T16:05:33.030254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28633,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053571+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:34.053571+0000","last_peered":"2024-09-10T16:05:34.053571+0000","last_clean":"2024-09-10T16:05:34.053571+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:34.053571+0000","last_undegraded":"2024-09-10T16:05:34.053571+0000","last_fullsized":"2024-09-10T16:05:34.053571+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032856+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:33.032856+0000","last_peered":"2024-09-10T16:05:33.032856+0000","last_clean":"2024-09-10T16:05:33.032856+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:33.032856+0000","last_undegraded":"2024-09-10T16:05:33.032856+0000","last_fullsized":"2024-09-10T16:05:33.032856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28301,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.050988+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:34.050988+0000","last_peered":"2024-09-10T16:05:34.050988+0000","last_clean":"2024-09-10T16:05:34.050988+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:34.050988+0000","last_undegraded":"2024-09-10T16:05:34.050988+0000","last_fullsized":"2024-09-10T16:05:34.050988+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21739,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033096+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:33.033096+0000","last_peered":"2024-09-10T16:05:33.033096+0000","last_clean":"2024-09-10T16:05:33.033096+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:33.033096+0000","last_undegraded":"2024-09-10T16:05:33.033096+0000","last_fullsized":"2024-09-10T16:05:33.033096+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21490,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033149+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:33.033149+0000","last_peered":"2024-09-10T16:05:33.033149+0000","last_clean":"2024-09-10T16:05:33.033149+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:33.033149+0000","last_undegraded":"2024-09-10T16:05:33.033149+0000","last_fullsized":"2024-09-10T16:05:33.033149+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053583+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:34.053583+0000","last_peered":"2024-09-10T16:05:34.053583+0000","last_clean":"2024-09-10T16:05:34.053583+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:34.053583+0000","last_undegraded":"2024-09-10T16:05:34.053583+0000","last_fullsized":"2024-09-10T16:05:34.053583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21627,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032971+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:33.032971+0000","last_peered":"2024-09-10T16:05:33.032971+0000","last_clean":"2024-09-10T16:05:33.032971+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:33.032971+0000","last_undegraded":"2024-09-10T16:05:33.032971+0000","last_fullsized":"2024-09-10T16:05:33.032971+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033067+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:33.033067+0000","last_peered":"2024-09-10T16:05:33.033067+0000","last_clean":"2024-09-10T16:05:33.033067+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:33.033067+0000","last_undegraded":"2024-09-10T16:05:33.033067+0000","last_fullsized":"2024-09-10T16:05:33.033067+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":121,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033040+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:33.033040+0000","last_peered":"2024-09-10T16:05:33.033040+0000","last_clean":"2024-09-10T16:05:33.033040+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:33.033040+0000","last_undegraded":"2024-09-10T16:05:33.033040+0000","last_fullsized":"2024-09-10T16:05:33.033040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21885,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58036224,"data_stored":58025385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515328,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":514747,"kb_avail":93856512,"statfs":{"total":96636764160,"available":96109068288,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":527101614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505600,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866240,"statfs":{"total":96636764160,"available":96119029760,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":517140141},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418116,"kb_used_data":57460,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93953724,"statfs":{"total":96636764160,"available":96208613376,"internally_reserved":0,"allocated":58839040,"data_stored":58459419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58028032,"data_stored":58024419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:41.096 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:41.376 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:41.376 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:41.388 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-10T16:05:40.676267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1439044,"kb_used_data":58580,"kb_used_omap":12,"kb_used_meta":1380403,"kb_avail":281676476,"statfs":{"total":289910292480,"available":288436711424,"internally_reserved":0,"allocated":59985920,"data_stored":58838903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1413532677},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.659763"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28848,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030155+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:33.030155+0000","last_peered":"2024-09-10T16:05:33.030155+0000","last_clean":"2024-09-10T16:05:33.030155+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:33.030155+0000","last_undegraded":"2024-09-10T16:05:33.030155+0000","last_fullsized":"2024-09-10T16:05:33.030155+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21229,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053581+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:34.053581+0000","last_peered":"2024-09-10T16:05:34.053581+0000","last_clean":"2024-09-10T16:05:34.053581+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:34.053581+0000","last_undegraded":"2024-09-10T16:05:34.053581+0000","last_fullsized":"2024-09-10T16:05:34.053581+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21144,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032927+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:33.032927+0000","last_peered":"2024-09-10T16:05:33.032927+0000","last_clean":"2024-09-10T16:05:33.032927+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:33.032927+0000","last_undegraded":"2024-09-10T16:05:33.032927+0000","last_fullsized":"2024-09-10T16:05:33.032927+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20809,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053494+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:34.053494+0000","last_peered":"2024-09-10T16:05:34.053494+0000","last_clean":"2024-09-10T16:05:34.053494+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:34.053494+0000","last_undegraded":"2024-09-10T16:05:34.053494+0000","last_fullsized":"2024-09-10T16:05:34.053494+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20506,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029866+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:33.029866+0000","last_peered":"2024-09-10T16:05:33.029866+0000","last_clean":"2024-09-10T16:05:33.029866+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:33.029866+0000","last_undegraded":"2024-09-10T16:05:33.029866+0000","last_fullsized":"2024-09-10T16:05:33.029866+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28803,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029811+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:33.029811+0000","last_peered":"2024-09-10T16:05:33.029811+0000","last_clean":"2024-09-10T16:05:33.029811+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:33.029811+0000","last_undegraded":"2024-09-10T16:05:33.029811+0000","last_fullsized":"2024-09-10T16:05:33.029811+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.121371+0000","last_change":"2024-09-10T16:05:33.121269+0000","last_active":"2024-09-10T16:05:33.121371+0000","last_peered":"2024-09-10T16:05:33.121371+0000","last_clean":"2024-09-10T16:05:33.121371+0000","last_became_active":"2024-09-10T16:05:33.033927+0000","last_became_peered":"2024-09-10T16:05:33.033927+0000","last_unstale":"2024-09-10T16:05:33.121371+0000","last_undegraded":"2024-09-10T16:05:33.121371+0000","last_fullsized":"2024-09-10T16:05:33.121371+0000","mapping_epoch":67,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:18.459494+0000","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21093,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029997+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:33.029997+0000","last_peered":"2024-09-10T16:05:33.029997+0000","last_clean":"2024-09-10T16:05:33.029997+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:33.029997+0000","last_undegraded":"2024-09-10T16:05:33.029997+0000","last_fullsized":"2024-09-10T16:05:33.029997+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029800+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:33.029800+0000","last_peered":"2024-09-10T16:05:33.029800+0000","last_clean":"2024-09-10T16:05:33.029800+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:33.029800+0000","last_undegraded":"2024-09-10T16:05:33.029800+0000","last_fullsized":"2024-09-10T16:05:33.029800+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28894,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032865+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:33.032865+0000","last_peered":"2024-09-10T16:05:33.032865+0000","last_clean":"2024-09-10T16:05:33.032865+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:33.032865+0000","last_undegraded":"2024-09-10T16:05:33.032865+0000","last_fullsized":"2024-09-10T16:05:33.032865+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053546+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:34.053546+0000","last_peered":"2024-09-10T16:05:34.053546+0000","last_clean":"2024-09-10T16:05:34.053546+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:34.053546+0000","last_undegraded":"2024-09-10T16:05:34.053546+0000","last_fullsized":"2024-09-10T16:05:34.053546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030391+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:33.030391+0000","last_peered":"2024-09-10T16:05:33.030391+0000","last_clean":"2024-09-10T16:05:33.030391+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:33.030391+0000","last_undegraded":"2024-09-10T16:05:33.030391+0000","last_fullsized":"2024-09-10T16:05:33.030391+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030254+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:33.030254+0000","last_peered":"2024-09-10T16:05:33.030254+0000","last_clean":"2024-09-10T16:05:33.030254+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:33.030254+0000","last_undegraded":"2024-09-10T16:05:33.030254+0000","last_fullsized":"2024-09-10T16:05:33.030254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28633,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053571+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:34.053571+0000","last_peered":"2024-09-10T16:05:34.053571+0000","last_clean":"2024-09-10T16:05:34.053571+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:34.053571+0000","last_undegraded":"2024-09-10T16:05:34.053571+0000","last_fullsized":"2024-09-10T16:05:34.053571+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032856+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:33.032856+0000","last_peered":"2024-09-10T16:05:33.032856+0000","last_clean":"2024-09-10T16:05:33.032856+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:33.032856+0000","last_undegraded":"2024-09-10T16:05:33.032856+0000","last_fullsized":"2024-09-10T16:05:33.032856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28301,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.050988+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:34.050988+0000","last_peered":"2024-09-10T16:05:34.050988+0000","last_clean":"2024-09-10T16:05:34.050988+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:34.050988+0000","last_undegraded":"2024-09-10T16:05:34.050988+0000","last_fullsized":"2024-09-10T16:05:34.050988+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21739,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033096+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:33.033096+0000","last_peered":"2024-09-10T16:05:33.033096+0000","last_clean":"2024-09-10T16:05:33.033096+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:33.033096+0000","last_undegraded":"2024-09-10T16:05:33.033096+0000","last_fullsized":"2024-09-10T16:05:33.033096+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21490,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033149+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:33.033149+0000","last_peered":"2024-09-10T16:05:33.033149+0000","last_clean":"2024-09-10T16:05:33.033149+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:33.033149+0000","last_undegraded":"2024-09-10T16:05:33.033149+0000","last_fullsized":"2024-09-10T16:05:33.033149+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053583+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:34.053583+0000","last_peered":"2024-09-10T16:05:34.053583+0000","last_clean":"2024-09-10T16:05:34.053583+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:34.053583+0000","last_undegraded":"2024-09-10T16:05:34.053583+0000","last_fullsized":"2024-09-10T16:05:34.053583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21627,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032971+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:33.032971+0000","last_peered":"2024-09-10T16:05:33.032971+0000","last_clean":"2024-09-10T16:05:33.032971+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:33.032971+0000","last_undegraded":"2024-09-10T16:05:33.032971+0000","last_fullsized":"2024-09-10T16:05:33.032971+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033067+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:33.033067+0000","last_peered":"2024-09-10T16:05:33.033067+0000","last_clean":"2024-09-10T16:05:33.033067+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:33.033067+0000","last_undegraded":"2024-09-10T16:05:33.033067+0000","last_fullsized":"2024-09-10T16:05:33.033067+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":121,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033040+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:33.033040+0000","last_peered":"2024-09-10T16:05:33.033040+0000","last_clean":"2024-09-10T16:05:33.033040+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:33.033040+0000","last_undegraded":"2024-09-10T16:05:33.033040+0000","last_fullsized":"2024-09-10T16:05:33.033040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21885,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58036224,"data_stored":58025385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515328,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":514747,"kb_avail":93856512,"statfs":{"total":96636764160,"available":96109068288,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":527101614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505600,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866240,"statfs":{"total":96636764160,"available":96119029760,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":517140141},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418116,"kb_used_data":57460,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93953724,"statfs":{"total":96636764160,"available":96208613376,"internally_reserved":0,"allocated":58839040,"data_stored":58459419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58028032,"data_stored":58024419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:41.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-10T16:05:41.389 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T16:05:41.405 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T16:05:41.405 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:41.686 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:41.687 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:41.699 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-10T16:05:40.676267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1439044,"kb_used_data":58580,"kb_used_omap":12,"kb_used_meta":1380403,"kb_avail":281676476,"statfs":{"total":289910292480,"available":288436711424,"internally_reserved":0,"allocated":59985920,"data_stored":58838903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1413532677},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.659763"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28848,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030155+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:33.030155+0000","last_peered":"2024-09-10T16:05:33.030155+0000","last_clean":"2024-09-10T16:05:33.030155+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:33.030155+0000","last_undegraded":"2024-09-10T16:05:33.030155+0000","last_fullsized":"2024-09-10T16:05:33.030155+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21229,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053581+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:34.053581+0000","last_peered":"2024-09-10T16:05:34.053581+0000","last_clean":"2024-09-10T16:05:34.053581+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:34.053581+0000","last_undegraded":"2024-09-10T16:05:34.053581+0000","last_fullsized":"2024-09-10T16:05:34.053581+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21144,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032927+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:33.032927+0000","last_peered":"2024-09-10T16:05:33.032927+0000","last_clean":"2024-09-10T16:05:33.032927+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:33.032927+0000","last_undegraded":"2024-09-10T16:05:33.032927+0000","last_fullsized":"2024-09-10T16:05:33.032927+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20809,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053494+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:34.053494+0000","last_peered":"2024-09-10T16:05:34.053494+0000","last_clean":"2024-09-10T16:05:34.053494+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:34.053494+0000","last_undegraded":"2024-09-10T16:05:34.053494+0000","last_fullsized":"2024-09-10T16:05:34.053494+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20506,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029866+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:33.029866+0000","last_peered":"2024-09-10T16:05:33.029866+0000","last_clean":"2024-09-10T16:05:33.029866+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:33.029866+0000","last_undegraded":"2024-09-10T16:05:33.029866+0000","last_fullsized":"2024-09-10T16:05:33.029866+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28803,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029811+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:33.029811+0000","last_peered":"2024-09-10T16:05:33.029811+0000","last_clean":"2024-09-10T16:05:33.029811+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:33.029811+0000","last_undegraded":"2024-09-10T16:05:33.029811+0000","last_fullsized":"2024-09-10T16:05:33.029811+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.121371+0000","last_change":"2024-09-10T16:05:33.121269+0000","last_active":"2024-09-10T16:05:33.121371+0000","last_peered":"2024-09-10T16:05:33.121371+0000","last_clean":"2024-09-10T16:05:33.121371+0000","last_became_active":"2024-09-10T16:05:33.033927+0000","last_became_peered":"2024-09-10T16:05:33.033927+0000","last_unstale":"2024-09-10T16:05:33.121371+0000","last_undegraded":"2024-09-10T16:05:33.121371+0000","last_fullsized":"2024-09-10T16:05:33.121371+0000","mapping_epoch":67,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:18.459494+0000","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21093,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029997+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:33.029997+0000","last_peered":"2024-09-10T16:05:33.029997+0000","last_clean":"2024-09-10T16:05:33.029997+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:33.029997+0000","last_undegraded":"2024-09-10T16:05:33.029997+0000","last_fullsized":"2024-09-10T16:05:33.029997+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029800+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:33.029800+0000","last_peered":"2024-09-10T16:05:33.029800+0000","last_clean":"2024-09-10T16:05:33.029800+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:33.029800+0000","last_undegraded":"2024-09-10T16:05:33.029800+0000","last_fullsized":"2024-09-10T16:05:33.029800+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28894,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032865+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:33.032865+0000","last_peered":"2024-09-10T16:05:33.032865+0000","last_clean":"2024-09-10T16:05:33.032865+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:33.032865+0000","last_undegraded":"2024-09-10T16:05:33.032865+0000","last_fullsized":"2024-09-10T16:05:33.032865+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053546+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:34.053546+0000","last_peered":"2024-09-10T16:05:34.053546+0000","last_clean":"2024-09-10T16:05:34.053546+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:34.053546+0000","last_undegraded":"2024-09-10T16:05:34.053546+0000","last_fullsized":"2024-09-10T16:05:34.053546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030391+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:33.030391+0000","last_peered":"2024-09-10T16:05:33.030391+0000","last_clean":"2024-09-10T16:05:33.030391+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:33.030391+0000","last_undegraded":"2024-09-10T16:05:33.030391+0000","last_fullsized":"2024-09-10T16:05:33.030391+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030254+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:33.030254+0000","last_peered":"2024-09-10T16:05:33.030254+0000","last_clean":"2024-09-10T16:05:33.030254+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:33.030254+0000","last_undegraded":"2024-09-10T16:05:33.030254+0000","last_fullsized":"2024-09-10T16:05:33.030254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28633,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053571+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:34.053571+0000","last_peered":"2024-09-10T16:05:34.053571+0000","last_clean":"2024-09-10T16:05:34.053571+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:34.053571+0000","last_undegraded":"2024-09-10T16:05:34.053571+0000","last_fullsized":"2024-09-10T16:05:34.053571+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032856+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:33.032856+0000","last_peered":"2024-09-10T16:05:33.032856+0000","last_clean":"2024-09-10T16:05:33.032856+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:33.032856+0000","last_undegraded":"2024-09-10T16:05:33.032856+0000","last_fullsized":"2024-09-10T16:05:33.032856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28301,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.050988+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:34.050988+0000","last_peered":"2024-09-10T16:05:34.050988+0000","last_clean":"2024-09-10T16:05:34.050988+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:34.050988+0000","last_undegraded":"2024-09-10T16:05:34.050988+0000","last_fullsized":"2024-09-10T16:05:34.050988+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21739,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033096+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:33.033096+0000","last_peered":"2024-09-10T16:05:33.033096+0000","last_clean":"2024-09-10T16:05:33.033096+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:33.033096+0000","last_undegraded":"2024-09-10T16:05:33.033096+0000","last_fullsized":"2024-09-10T16:05:33.033096+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21490,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033149+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:33.033149+0000","last_peered":"2024-09-10T16:05:33.033149+0000","last_clean":"2024-09-10T16:05:33.033149+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:33.033149+0000","last_undegraded":"2024-09-10T16:05:33.033149+0000","last_fullsized":"2024-09-10T16:05:33.033149+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053583+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:34.053583+0000","last_peered":"2024-09-10T16:05:34.053583+0000","last_clean":"2024-09-10T16:05:34.053583+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:34.053583+0000","last_undegraded":"2024-09-10T16:05:34.053583+0000","last_fullsized":"2024-09-10T16:05:34.053583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21627,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032971+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:33.032971+0000","last_peered":"2024-09-10T16:05:33.032971+0000","last_clean":"2024-09-10T16:05:33.032971+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:33.032971+0000","last_undegraded":"2024-09-10T16:05:33.032971+0000","last_fullsized":"2024-09-10T16:05:33.032971+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033067+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:33.033067+0000","last_peered":"2024-09-10T16:05:33.033067+0000","last_clean":"2024-09-10T16:05:33.033067+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:33.033067+0000","last_undegraded":"2024-09-10T16:05:33.033067+0000","last_fullsized":"2024-09-10T16:05:33.033067+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":121,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033040+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:33.033040+0000","last_peered":"2024-09-10T16:05:33.033040+0000","last_clean":"2024-09-10T16:05:33.033040+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:33.033040+0000","last_undegraded":"2024-09-10T16:05:33.033040+0000","last_fullsized":"2024-09-10T16:05:33.033040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21885,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58036224,"data_stored":58025385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515328,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":514747,"kb_avail":93856512,"statfs":{"total":96636764160,"available":96109068288,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":527101614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505600,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866240,"statfs":{"total":96636764160,"available":96119029760,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":517140141},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418116,"kb_used_data":57460,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93953724,"statfs":{"total":96636764160,"available":96208613376,"internally_reserved":0,"allocated":58839040,"data_stored":58459419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58028032,"data_stored":58024419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:41.700 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:41.981 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:41.981 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:41.992 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-10T16:05:40.676267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1439044,"kb_used_data":58580,"kb_used_omap":12,"kb_used_meta":1380403,"kb_avail":281676476,"statfs":{"total":289910292480,"available":288436711424,"internally_reserved":0,"allocated":59985920,"data_stored":58838903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1413532677},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.659763"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28848,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030155+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:33.030155+0000","last_peered":"2024-09-10T16:05:33.030155+0000","last_clean":"2024-09-10T16:05:33.030155+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:33.030155+0000","last_undegraded":"2024-09-10T16:05:33.030155+0000","last_fullsized":"2024-09-10T16:05:33.030155+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21229,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053581+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:34.053581+0000","last_peered":"2024-09-10T16:05:34.053581+0000","last_clean":"2024-09-10T16:05:34.053581+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:34.053581+0000","last_undegraded":"2024-09-10T16:05:34.053581+0000","last_fullsized":"2024-09-10T16:05:34.053581+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21144,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032927+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:33.032927+0000","last_peered":"2024-09-10T16:05:33.032927+0000","last_clean":"2024-09-10T16:05:33.032927+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:33.032927+0000","last_undegraded":"2024-09-10T16:05:33.032927+0000","last_fullsized":"2024-09-10T16:05:33.032927+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20809,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053494+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:34.053494+0000","last_peered":"2024-09-10T16:05:34.053494+0000","last_clean":"2024-09-10T16:05:34.053494+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:34.053494+0000","last_undegraded":"2024-09-10T16:05:34.053494+0000","last_fullsized":"2024-09-10T16:05:34.053494+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20506,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029866+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:33.029866+0000","last_peered":"2024-09-10T16:05:33.029866+0000","last_clean":"2024-09-10T16:05:33.029866+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:33.029866+0000","last_undegraded":"2024-09-10T16:05:33.029866+0000","last_fullsized":"2024-09-10T16:05:33.029866+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28803,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029811+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:33.029811+0000","last_peered":"2024-09-10T16:05:33.029811+0000","last_clean":"2024-09-10T16:05:33.029811+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:33.029811+0000","last_undegraded":"2024-09-10T16:05:33.029811+0000","last_fullsized":"2024-09-10T16:05:33.029811+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.121371+0000","last_change":"2024-09-10T16:05:33.121269+0000","last_active":"2024-09-10T16:05:33.121371+0000","last_peered":"2024-09-10T16:05:33.121371+0000","last_clean":"2024-09-10T16:05:33.121371+0000","last_became_active":"2024-09-10T16:05:33.033927+0000","last_became_peered":"2024-09-10T16:05:33.033927+0000","last_unstale":"2024-09-10T16:05:33.121371+0000","last_undegraded":"2024-09-10T16:05:33.121371+0000","last_fullsized":"2024-09-10T16:05:33.121371+0000","mapping_epoch":67,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:18.459494+0000","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21093,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029997+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:33.029997+0000","last_peered":"2024-09-10T16:05:33.029997+0000","last_clean":"2024-09-10T16:05:33.029997+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:33.029997+0000","last_undegraded":"2024-09-10T16:05:33.029997+0000","last_fullsized":"2024-09-10T16:05:33.029997+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029800+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:33.029800+0000","last_peered":"2024-09-10T16:05:33.029800+0000","last_clean":"2024-09-10T16:05:33.029800+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:33.029800+0000","last_undegraded":"2024-09-10T16:05:33.029800+0000","last_fullsized":"2024-09-10T16:05:33.029800+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28894,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032865+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:33.032865+0000","last_peered":"2024-09-10T16:05:33.032865+0000","last_clean":"2024-09-10T16:05:33.032865+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:33.032865+0000","last_undegraded":"2024-09-10T16:05:33.032865+0000","last_fullsized":"2024-09-10T16:05:33.032865+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053546+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:34.053546+0000","last_peered":"2024-09-10T16:05:34.053546+0000","last_clean":"2024-09-10T16:05:34.053546+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:34.053546+0000","last_undegraded":"2024-09-10T16:05:34.053546+0000","last_fullsized":"2024-09-10T16:05:34.053546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030391+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:33.030391+0000","last_peered":"2024-09-10T16:05:33.030391+0000","last_clean":"2024-09-10T16:05:33.030391+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:33.030391+0000","last_undegraded":"2024-09-10T16:05:33.030391+0000","last_fullsized":"2024-09-10T16:05:33.030391+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030254+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:33.030254+0000","last_peered":"2024-09-10T16:05:33.030254+0000","last_clean":"2024-09-10T16:05:33.030254+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:33.030254+0000","last_undegraded":"2024-09-10T16:05:33.030254+0000","last_fullsized":"2024-09-10T16:05:33.030254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28633,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053571+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:34.053571+0000","last_peered":"2024-09-10T16:05:34.053571+0000","last_clean":"2024-09-10T16:05:34.053571+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:34.053571+0000","last_undegraded":"2024-09-10T16:05:34.053571+0000","last_fullsized":"2024-09-10T16:05:34.053571+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032856+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:33.032856+0000","last_peered":"2024-09-10T16:05:33.032856+0000","last_clean":"2024-09-10T16:05:33.032856+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:33.032856+0000","last_undegraded":"2024-09-10T16:05:33.032856+0000","last_fullsized":"2024-09-10T16:05:33.032856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28301,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.050988+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:34.050988+0000","last_peered":"2024-09-10T16:05:34.050988+0000","last_clean":"2024-09-10T16:05:34.050988+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:34.050988+0000","last_undegraded":"2024-09-10T16:05:34.050988+0000","last_fullsized":"2024-09-10T16:05:34.050988+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21739,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033096+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:33.033096+0000","last_peered":"2024-09-10T16:05:33.033096+0000","last_clean":"2024-09-10T16:05:33.033096+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:33.033096+0000","last_undegraded":"2024-09-10T16:05:33.033096+0000","last_fullsized":"2024-09-10T16:05:33.033096+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21490,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033149+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:33.033149+0000","last_peered":"2024-09-10T16:05:33.033149+0000","last_clean":"2024-09-10T16:05:33.033149+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:33.033149+0000","last_undegraded":"2024-09-10T16:05:33.033149+0000","last_fullsized":"2024-09-10T16:05:33.033149+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053583+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:34.053583+0000","last_peered":"2024-09-10T16:05:34.053583+0000","last_clean":"2024-09-10T16:05:34.053583+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:34.053583+0000","last_undegraded":"2024-09-10T16:05:34.053583+0000","last_fullsized":"2024-09-10T16:05:34.053583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21627,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032971+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:33.032971+0000","last_peered":"2024-09-10T16:05:33.032971+0000","last_clean":"2024-09-10T16:05:33.032971+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:33.032971+0000","last_undegraded":"2024-09-10T16:05:33.032971+0000","last_fullsized":"2024-09-10T16:05:33.032971+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033067+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:33.033067+0000","last_peered":"2024-09-10T16:05:33.033067+0000","last_clean":"2024-09-10T16:05:33.033067+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:33.033067+0000","last_undegraded":"2024-09-10T16:05:33.033067+0000","last_fullsized":"2024-09-10T16:05:33.033067+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":121,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033040+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:33.033040+0000","last_peered":"2024-09-10T16:05:33.033040+0000","last_clean":"2024-09-10T16:05:33.033040+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:33.033040+0000","last_undegraded":"2024-09-10T16:05:33.033040+0000","last_fullsized":"2024-09-10T16:05:33.033040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21885,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58036224,"data_stored":58025385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515328,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":514747,"kb_avail":93856512,"statfs":{"total":96636764160,"available":96109068288,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":527101614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505600,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866240,"statfs":{"total":96636764160,"available":96119029760,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":517140141},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418116,"kb_used_data":57460,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93953724,"statfs":{"total":96636764160,"available":96208613376,"internally_reserved":0,"allocated":58839040,"data_stored":58459419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58028032,"data_stored":58024419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:41.994 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T16:05:41.994 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:42.278 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:42.279 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:42.290 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-10T16:05:40.676267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1439044,"kb_used_data":58580,"kb_used_omap":12,"kb_used_meta":1380403,"kb_avail":281676476,"statfs":{"total":289910292480,"available":288436711424,"internally_reserved":0,"allocated":59985920,"data_stored":58838903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1413532677},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.659763"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28848,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030155+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:33.030155+0000","last_peered":"2024-09-10T16:05:33.030155+0000","last_clean":"2024-09-10T16:05:33.030155+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:33.030155+0000","last_undegraded":"2024-09-10T16:05:33.030155+0000","last_fullsized":"2024-09-10T16:05:33.030155+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21229,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053581+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:34.053581+0000","last_peered":"2024-09-10T16:05:34.053581+0000","last_clean":"2024-09-10T16:05:34.053581+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:34.053581+0000","last_undegraded":"2024-09-10T16:05:34.053581+0000","last_fullsized":"2024-09-10T16:05:34.053581+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21144,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032927+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:33.032927+0000","last_peered":"2024-09-10T16:05:33.032927+0000","last_clean":"2024-09-10T16:05:33.032927+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:33.032927+0000","last_undegraded":"2024-09-10T16:05:33.032927+0000","last_fullsized":"2024-09-10T16:05:33.032927+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20809,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053494+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:34.053494+0000","last_peered":"2024-09-10T16:05:34.053494+0000","last_clean":"2024-09-10T16:05:34.053494+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:34.053494+0000","last_undegraded":"2024-09-10T16:05:34.053494+0000","last_fullsized":"2024-09-10T16:05:34.053494+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20506,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029866+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:33.029866+0000","last_peered":"2024-09-10T16:05:33.029866+0000","last_clean":"2024-09-10T16:05:33.029866+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:33.029866+0000","last_undegraded":"2024-09-10T16:05:33.029866+0000","last_fullsized":"2024-09-10T16:05:33.029866+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28803,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029811+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:33.029811+0000","last_peered":"2024-09-10T16:05:33.029811+0000","last_clean":"2024-09-10T16:05:33.029811+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:33.029811+0000","last_undegraded":"2024-09-10T16:05:33.029811+0000","last_fullsized":"2024-09-10T16:05:33.029811+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.121371+0000","last_change":"2024-09-10T16:05:33.121269+0000","last_active":"2024-09-10T16:05:33.121371+0000","last_peered":"2024-09-10T16:05:33.121371+0000","last_clean":"2024-09-10T16:05:33.121371+0000","last_became_active":"2024-09-10T16:05:33.033927+0000","last_became_peered":"2024-09-10T16:05:33.033927+0000","last_unstale":"2024-09-10T16:05:33.121371+0000","last_undegraded":"2024-09-10T16:05:33.121371+0000","last_fullsized":"2024-09-10T16:05:33.121371+0000","mapping_epoch":67,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:18.459494+0000","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21093,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029997+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:33.029997+0000","last_peered":"2024-09-10T16:05:33.029997+0000","last_clean":"2024-09-10T16:05:33.029997+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:33.029997+0000","last_undegraded":"2024-09-10T16:05:33.029997+0000","last_fullsized":"2024-09-10T16:05:33.029997+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029800+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:33.029800+0000","last_peered":"2024-09-10T16:05:33.029800+0000","last_clean":"2024-09-10T16:05:33.029800+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:33.029800+0000","last_undegraded":"2024-09-10T16:05:33.029800+0000","last_fullsized":"2024-09-10T16:05:33.029800+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28894,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032865+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:33.032865+0000","last_peered":"2024-09-10T16:05:33.032865+0000","last_clean":"2024-09-10T16:05:33.032865+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:33.032865+0000","last_undegraded":"2024-09-10T16:05:33.032865+0000","last_fullsized":"2024-09-10T16:05:33.032865+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053546+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:34.053546+0000","last_peered":"2024-09-10T16:05:34.053546+0000","last_clean":"2024-09-10T16:05:34.053546+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:34.053546+0000","last_undegraded":"2024-09-10T16:05:34.053546+0000","last_fullsized":"2024-09-10T16:05:34.053546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030391+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:33.030391+0000","last_peered":"2024-09-10T16:05:33.030391+0000","last_clean":"2024-09-10T16:05:33.030391+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:33.030391+0000","last_undegraded":"2024-09-10T16:05:33.030391+0000","last_fullsized":"2024-09-10T16:05:33.030391+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030254+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:33.030254+0000","last_peered":"2024-09-10T16:05:33.030254+0000","last_clean":"2024-09-10T16:05:33.030254+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:33.030254+0000","last_undegraded":"2024-09-10T16:05:33.030254+0000","last_fullsized":"2024-09-10T16:05:33.030254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28633,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053571+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:34.053571+0000","last_peered":"2024-09-10T16:05:34.053571+0000","last_clean":"2024-09-10T16:05:34.053571+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:34.053571+0000","last_undegraded":"2024-09-10T16:05:34.053571+0000","last_fullsized":"2024-09-10T16:05:34.053571+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032856+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:33.032856+0000","last_peered":"2024-09-10T16:05:33.032856+0000","last_clean":"2024-09-10T16:05:33.032856+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:33.032856+0000","last_undegraded":"2024-09-10T16:05:33.032856+0000","last_fullsized":"2024-09-10T16:05:33.032856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28301,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.050988+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:34.050988+0000","last_peered":"2024-09-10T16:05:34.050988+0000","last_clean":"2024-09-10T16:05:34.050988+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:34.050988+0000","last_undegraded":"2024-09-10T16:05:34.050988+0000","last_fullsized":"2024-09-10T16:05:34.050988+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21739,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033096+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:33.033096+0000","last_peered":"2024-09-10T16:05:33.033096+0000","last_clean":"2024-09-10T16:05:33.033096+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:33.033096+0000","last_undegraded":"2024-09-10T16:05:33.033096+0000","last_fullsized":"2024-09-10T16:05:33.033096+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21490,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033149+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:33.033149+0000","last_peered":"2024-09-10T16:05:33.033149+0000","last_clean":"2024-09-10T16:05:33.033149+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:33.033149+0000","last_undegraded":"2024-09-10T16:05:33.033149+0000","last_fullsized":"2024-09-10T16:05:33.033149+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053583+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:34.053583+0000","last_peered":"2024-09-10T16:05:34.053583+0000","last_clean":"2024-09-10T16:05:34.053583+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:34.053583+0000","last_undegraded":"2024-09-10T16:05:34.053583+0000","last_fullsized":"2024-09-10T16:05:34.053583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21627,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032971+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:33.032971+0000","last_peered":"2024-09-10T16:05:33.032971+0000","last_clean":"2024-09-10T16:05:33.032971+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:33.032971+0000","last_undegraded":"2024-09-10T16:05:33.032971+0000","last_fullsized":"2024-09-10T16:05:33.032971+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033067+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:33.033067+0000","last_peered":"2024-09-10T16:05:33.033067+0000","last_clean":"2024-09-10T16:05:33.033067+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:33.033067+0000","last_undegraded":"2024-09-10T16:05:33.033067+0000","last_fullsized":"2024-09-10T16:05:33.033067+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":121,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033040+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:33.033040+0000","last_peered":"2024-09-10T16:05:33.033040+0000","last_clean":"2024-09-10T16:05:33.033040+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:33.033040+0000","last_undegraded":"2024-09-10T16:05:33.033040+0000","last_fullsized":"2024-09-10T16:05:33.033040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21885,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58036224,"data_stored":58025385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626585,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515328,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":514747,"kb_avail":93856512,"statfs":{"total":96636764160,"available":96109068288,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":527101614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505600,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866240,"statfs":{"total":96636764160,"available":96119029760,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":517140141},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418116,"kb_used_data":57460,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93953724,"statfs":{"total":96636764160,"available":96208613376,"internally_reserved":0,"allocated":58839040,"data_stored":58459419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58028032,"data_stored":58024419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:42.292 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T16:05:42.574 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":68,"fsid":"199e91a6-cca4-4c41-a5fa-011e971838b5","created":"2024-09-10T15:51:14.060262+0000","modified":"2024-09-10T16:05:33.019817+0000","last_up_change":"2024-09-10T15:53:17.454878+0000","last_in_change":"2024-09-10T15:53:10.382924+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-10T15:51:29.314004+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-10T15:51:40.401037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"53b58107-009b-46ac-921d-2e8d9580081e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":22,"up_from":26,"up_thru":66,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6800","nonce":824549766}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6801","nonce":824549766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6803","nonce":824549766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6802","nonce":824549766}]},"public_addr":"172.21.15.121:6800/824549766","cluster_addr":"172.21.15.121:6801/824549766","heartbeat_back_addr":"172.21.15.121:6803/824549766","heartbeat_front_addr":"172.21.15.121:6802/824549766","state":["exists","up"]},{"osd":1,"uuid":"aced9681-6a62-49a4-a1e2-6ff7d9a544ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":27,"up_from":42,"up_thru":67,"down_at":28,"lost_at":28,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6808","nonce":2529147590}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6809","nonce":2529147590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6811","nonce":2529147590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6810","nonce":2529147590}]},"public_addr":"172.21.15.121:6808/2529147590","cluster_addr":"172.21.15.121:6809/2529147590","heartbeat_back_addr":"172.21.15.121:6811/2529147590","heartbeat_front_addr":"172.21.15.121:6810/2529147590","state":["exists","up"]},{"osd":2,"uuid":"0cfba235-9896-429e-8396-69dfeb7a44bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":18,"up_from":32,"up_thru":56,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6804","nonce":4255745125}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6805","nonce":4255745125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6807","nonce":4255745125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.121:6806","nonce":4255745125}]},"public_addr":"172.21.15.121:6804/4255745125","cluster_addr":"172.21.15.121:6805/4255745125","heartbeat_back_addr":"172.21.15.121:6807/4255745125","heartbeat_front_addr":"172.21.15.121:6806/4255745125","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:22.164346+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.644539+0000","dead_epoch":28},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T15:51:21.413080+0000","dead_epoch":20}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T16:05:43.587 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T16:05:43.588 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T16:05:43.721 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T16:05:43.722 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T16:05:43.722 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T16:05:43.733 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T16:05:44.017 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T16:05:44.030 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T16:05:44.030 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T16:05:44.151 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T16:05:44.151 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T16:05:44.151 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T16:05:44.161 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T16:05:44.442 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T16:05:44.455 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T16:05:44.456 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T16:05:44.576 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-10T16:05:44.577 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T16:05:44.577 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-10T16:05:44.587 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T16:05:44.874 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T16:05:44.887 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:05:45.175 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:05:45.175 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:05:45.186 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-10T16:05:44.676867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1434484,"kb_used_data":54020,"kb_used_omap":12,"kb_used_meta":1380403,"kb_avail":281681036,"statfs":{"total":289910292480,"available":288441380864,"internally_reserved":0,"allocated":55316480,"data_stored":54169463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1413532677},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.648262"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28849,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030195+0000","last_change":"2024-09-10T16:02:45.045648+0000","last_active":"2024-09-10T16:05:33.030195+0000","last_peered":"2024-09-10T16:05:33.030195+0000","last_clean":"2024-09-10T16:05:33.030195+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:33.030195+0000","last_undegraded":"2024-09-10T16:05:33.030195+0000","last_fullsized":"2024-09-10T16:05:33.030195+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:55.206969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21229,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053581+0000","last_change":"2024-09-10T15:52:49.843355+0000","last_active":"2024-09-10T16:05:34.053581+0000","last_peered":"2024-09-10T16:05:34.053581+0000","last_clean":"2024-09-10T16:05:34.053581+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:34.053581+0000","last_undegraded":"2024-09-10T16:05:34.053581+0000","last_fullsized":"2024-09-10T16:05:34.053581+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1699","last_scrub_stamp":"2024-09-10T15:52:49.843203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:49.843203+0000","objects_scrubbed":1658,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:15.921304+0000","scrub_duration":940,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21144,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032927+0000","last_change":"2024-09-10T16:03:46.400559+0000","last_active":"2024-09-10T16:05:33.032927+0000","last_peered":"2024-09-10T16:05:33.032927+0000","last_clean":"2024-09-10T16:05:33.032927+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:33.032927+0000","last_undegraded":"2024-09-10T16:05:33.032927+0000","last_fullsized":"2024-09-10T16:05:33.032927+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1763","last_scrub_stamp":"2024-09-10T15:52:50.491042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:50.491042+0000","objects_scrubbed":1728,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:59.247001+0000","scrub_duration":573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20809,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053494+0000","last_change":"2024-09-10T15:52:31.156786+0000","last_active":"2024-09-10T16:05:34.053494+0000","last_peered":"2024-09-10T16:05:34.053494+0000","last_clean":"2024-09-10T16:05:34.053494+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:34.053494+0000","last_undegraded":"2024-09-10T16:05:34.053494+0000","last_fullsized":"2024-09-10T16:05:34.053494+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:51:40.844370+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:59.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20506,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029866+0000","last_change":"2024-09-10T15:53:04.071364+0000","last_active":"2024-09-10T16:05:33.029866+0000","last_peered":"2024-09-10T16:05:33.029866+0000","last_clean":"2024-09-10T16:05:33.029866+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:33.029866+0000","last_undegraded":"2024-09-10T16:05:33.029866+0000","last_fullsized":"2024-09-10T16:05:33.029866+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2967","last_scrub_stamp":"2024-09-10T15:53:04.071206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.071206+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.737934+0000","scrub_duration":1446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28803,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029811+0000","last_change":"2024-09-10T15:58:12.888662+0000","last_active":"2024-09-10T16:05:33.029811+0000","last_peered":"2024-09-10T16:05:33.029811+0000","last_clean":"2024-09-10T16:05:33.029811+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:33.029811+0000","last_undegraded":"2024-09-10T16:05:33.029811+0000","last_fullsized":"2024-09-10T16:05:33.029811+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3327","last_scrub_stamp":"2024-09-10T15:53:07.938168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:07.938168+0000","objects_scrubbed":3277,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:33.645398+0000","scrub_duration":1368,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21070,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.121371+0000","last_change":"2024-09-10T16:05:33.121269+0000","last_active":"2024-09-10T16:05:33.121371+0000","last_peered":"2024-09-10T16:05:33.121371+0000","last_clean":"2024-09-10T16:05:33.121371+0000","last_became_active":"2024-09-10T16:05:33.033927+0000","last_became_peered":"2024-09-10T16:05:33.033927+0000","last_unstale":"2024-09-10T16:05:33.121371+0000","last_undegraded":"2024-09-10T16:05:33.121371+0000","last_fullsized":"2024-09-10T16:05:33.121371+0000","mapping_epoch":67,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3393","last_scrub_stamp":"2024-09-10T15:53:09.068900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.068900+0000","objects_scrubbed":3344,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:18.459494+0000","scrub_duration":1525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21093,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029997+0000","last_change":"2024-09-10T15:53:06.815322+0000","last_active":"2024-09-10T16:05:33.029997+0000","last_peered":"2024-09-10T16:05:33.029997+0000","last_clean":"2024-09-10T16:05:33.029997+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:33.029997+0000","last_undegraded":"2024-09-10T16:05:33.029997+0000","last_fullsized":"2024-09-10T16:05:33.029997+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3296","last_scrub_stamp":"2024-09-10T15:53:06.814832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:06.814832+0000","objects_scrubbed":3239,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:10.820660+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.029800+0000","last_change":"2024-09-10T15:53:19.475878+0000","last_active":"2024-09-10T16:05:33.029800+0000","last_peered":"2024-09-10T16:05:33.029800+0000","last_clean":"2024-09-10T16:05:33.029800+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:33.029800+0000","last_undegraded":"2024-09-10T16:05:33.029800+0000","last_fullsized":"2024-09-10T16:05:33.029800+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:35.265828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28894,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032865+0000","last_change":"2024-09-10T15:55:20.691429+0000","last_active":"2024-09-10T16:05:33.032865+0000","last_peered":"2024-09-10T16:05:33.032865+0000","last_clean":"2024-09-10T16:05:33.032865+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:33.032865+0000","last_undegraded":"2024-09-10T16:05:33.032865+0000","last_fullsized":"2024-09-10T16:05:33.032865+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2072","last_scrub_stamp":"2024-09-10T15:52:52.645684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:52.645684+0000","objects_scrubbed":2016,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:39.745678+0000","scrub_duration":750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053546+0000","last_change":"2024-09-10T15:52:31.160243+0000","last_active":"2024-09-10T16:05:34.053546+0000","last_peered":"2024-09-10T16:05:34.053546+0000","last_clean":"2024-09-10T16:05:34.053546+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:34.053546+0000","last_undegraded":"2024-09-10T16:05:34.053546+0000","last_fullsized":"2024-09-10T16:05:34.053546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:49.301528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030391+0000","last_change":"2024-09-10T16:04:28.962763+0000","last_active":"2024-09-10T16:05:33.030391+0000","last_peered":"2024-09-10T16:05:33.030391+0000","last_clean":"2024-09-10T16:05:33.030391+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:33.030391+0000","last_undegraded":"2024-09-10T16:05:33.030391+0000","last_fullsized":"2024-09-10T16:05:33.030391+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3191","last_scrub_stamp":"2024-09-10T15:53:05.799133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:05.799133+0000","objects_scrubbed":3128,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:31.765687+0000","scrub_duration":1157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.030254+0000","last_change":"2024-09-10T15:53:19.891730+0000","last_active":"2024-09-10T16:05:33.030254+0000","last_peered":"2024-09-10T16:05:33.030254+0000","last_clean":"2024-09-10T16:05:33.030254+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:33.030254+0000","last_undegraded":"2024-09-10T16:05:33.030254+0000","last_fullsized":"2024-09-10T16:05:33.030254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:30.798710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28633,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053571+0000","last_change":"2024-09-10T16:00:39.008606+0000","last_active":"2024-09-10T16:05:34.053571+0000","last_peered":"2024-09-10T16:05:34.053571+0000","last_clean":"2024-09-10T16:05:34.053571+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:34.053571+0000","last_undegraded":"2024-09-10T16:05:34.053571+0000","last_fullsized":"2024-09-10T16:05:34.053571+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2062","last_scrub_stamp":"2024-09-10T15:52:53.665808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:53.665808+0000","objects_scrubbed":2021,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:55.376251+0000","scrub_duration":746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032856+0000","last_change":"2024-09-10T15:53:19.558473+0000","last_active":"2024-09-10T16:05:33.032856+0000","last_peered":"2024-09-10T16:05:33.032856+0000","last_clean":"2024-09-10T16:05:33.032856+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:33.032856+0000","last_undegraded":"2024-09-10T16:05:33.032856+0000","last_fullsized":"2024-09-10T16:05:33.032856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:25.688838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28301,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.050988+0000","last_change":"2024-09-10T16:03:06.169932+0000","last_active":"2024-09-10T16:05:34.050988+0000","last_peered":"2024-09-10T16:05:34.050988+0000","last_clean":"2024-09-10T16:05:34.050988+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:34.050988+0000","last_undegraded":"2024-09-10T16:05:34.050988+0000","last_fullsized":"2024-09-10T16:05:34.050988+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2154","last_scrub_stamp":"2024-09-10T15:52:54.903413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:52:54.903413+0000","objects_scrubbed":2125,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.198108+0000","scrub_duration":979,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21739,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033096+0000","last_change":"2024-09-10T16:03:24.238911+0000","last_active":"2024-09-10T16:05:33.033096+0000","last_peered":"2024-09-10T16:05:33.033096+0000","last_clean":"2024-09-10T16:05:33.033096+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:33.033096+0000","last_undegraded":"2024-09-10T16:05:33.033096+0000","last_fullsized":"2024-09-10T16:05:33.033096+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2845","last_scrub_stamp":"2024-09-10T15:53:02.560275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:02.560275+0000","objects_scrubbed":2806,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:59.574685+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T15:53:19.599488+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:58.369868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21490,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033149+0000","last_change":"2024-09-10T16:05:10.083771+0000","last_active":"2024-09-10T16:05:33.033149+0000","last_peered":"2024-09-10T16:05:33.033149+0000","last_clean":"2024-09-10T16:05:33.033149+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:33.033149+0000","last_undegraded":"2024-09-10T16:05:33.033149+0000","last_fullsized":"2024-09-10T16:05:33.033149+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3640","last_scrub_stamp":"2024-09-10T15:53:13.900343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:13.900343+0000","objects_scrubbed":3574,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:52.717505+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":142,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:34.053583+0000","last_change":"2024-09-10T15:52:31.157317+0000","last_active":"2024-09-10T16:05:34.053583+0000","last_peered":"2024-09-10T16:05:34.053583+0000","last_clean":"2024-09-10T16:05:34.053583+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:34.053583+0000","last_undegraded":"2024-09-10T16:05:34.053583+0000","last_fullsized":"2024-09-10T16:05:34.053583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.596639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21627,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032971+0000","last_change":"2024-09-10T16:04:08.909863+0000","last_active":"2024-09-10T16:05:33.032971+0000","last_peered":"2024-09-10T16:05:33.032971+0000","last_clean":"2024-09-10T16:05:33.032971+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:33.032971+0000","last_undegraded":"2024-09-10T16:05:33.032971+0000","last_fullsized":"2024-09-10T16:05:33.032971+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3433","last_scrub_stamp":"2024-09-10T15:53:09.687452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:09.687452+0000","objects_scrubbed":3390,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:11.992166+0000","scrub_duration":1195,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":116,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033067+0000","last_change":"2024-09-10T15:53:19.598225+0000","last_active":"2024-09-10T16:05:33.033067+0000","last_peered":"2024-09-10T16:05:33.033067+0000","last_clean":"2024-09-10T16:05:33.033067+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:33.033067+0000","last_undegraded":"2024-09-10T16:05:33.033067+0000","last_fullsized":"2024-09-10T16:05:33.033067+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:33.321295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":121,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.033040+0000","last_change":"2024-09-10T15:53:19.473629+0000","last_active":"2024-09-10T16:05:33.033040+0000","last_peered":"2024-09-10T16:05:33.033040+0000","last_clean":"2024-09-10T16:05:33.033040+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:33.033040+0000","last_undegraded":"2024-09-10T16:05:33.033040+0000","last_fullsized":"2024-09-10T16:05:33.033040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:29.596349+0000","last_clean_scrub_stamp":"2024-09-10T15:51:29.596349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:26.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'21098","reported_seq":21885,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:33.032877+0000","last_change":"2024-09-10T16:04:51.053976+0000","last_active":"2024-09-10T16:05:33.032877+0000","last_peered":"2024-09-10T16:05:33.032877+0000","last_clean":"2024-09-10T16:05:33.032877+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:33.032877+0000","last_undegraded":"2024-09-10T16:05:33.032877+0000","last_fullsized":"2024-09-10T16:05:33.032877+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3241","last_scrub_stamp":"2024-09-10T15:53:04.965466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T15:51:40.844370+0000","last_clean_scrub_stamp":"2024-09-10T15:53:04.965466+0000","objects_scrubbed":3170,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:23.444669+0000","scrub_duration":1293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53121024,"data_stored":53110185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626586,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515328,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":514747,"kb_avail":93856512,"statfs":{"total":96636764160,"available":96109068288,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":527101614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505600,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866240,"statfs":{"total":96636764160,"available":96119029760,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":517140141},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953638,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413556,"kb_used_data":52900,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93958284,"statfs":{"total":96636764160,"available":96213282816,"internally_reserved":0,"allocated":54169600,"data_stored":53789979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53112832,"data_stored":53109219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:05:45.191 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-10T15:51:40.844370+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=51, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.191 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-10T15:52:49.843203+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=52, tm_sec=49, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.191 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-10T15:52:50.491042+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=52, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.191 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-10T15:51:40.844370+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=51, tm_sec=40, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.191 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-10T15:53:04.071206+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=53, tm_sec=4, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.192 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-10T15:53:07.938168+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=53, tm_sec=7, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.192 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-10T15:53:09.068900+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=53, tm_sec=9, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.192 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-10T15:53:06.814832+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=53, tm_sec=6, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.192 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T15:51:29.596349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=51, tm_sec=29, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.192 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-10T15:52:52.645684+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=52, tm_sec=52, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.192 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-10T15:51:29.596349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=51, tm_sec=29, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.192 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-10T15:53:05.799133+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=53, tm_sec=5, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.192 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-10T15:51:29.596349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=51, tm_sec=29, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.192 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-10T15:52:53.665808+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=52, tm_sec=53, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.193 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-10T15:51:29.596349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=51, tm_sec=29, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.193 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-10T15:52:54.903413+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=52, tm_sec=54, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.193 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-10T15:53:02.560275+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=53, tm_sec=2, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.193 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-10T15:51:29.596349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=51, tm_sec=29, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.193 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-10T15:53:13.900343+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=53, tm_sec=13, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.193 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-10T15:51:29.596349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=51, tm_sec=29, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.193 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-10T15:53:09.687452+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=53, tm_sec=9, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.193 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-10T15:51:29.596349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=51, tm_sec=29, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.193 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-10T15:51:29.596349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=51, tm_sec=29, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.194 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-10T15:53:04.965466+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=15, tm_min=53, tm_sec=4, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=16, tm_min=5, tm_sec=42, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T16:05:45.194 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T16:06:05.195 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T16:06:05.489 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-10T16:06:05.490 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-09-10T16:06:05.501 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-09-10T16:06:04.681239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293051,"num_write_kb":509303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46650,"num_bytes_recovered":37408787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160934,"ondisk_log_size":160934,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1417656,"kb_used_data":37192,"kb_used_omap":12,"kb_used_meta":1380403,"kb_avail":281697864,"statfs":{"total":289910292480,"available":288458612736,"internally_reserved":0,"allocated":38084608,"data_stored":36937591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13307,"internal_metadata":1413532677},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002695"},"pg_stats":[{"pgid":"2.f","version":"50'20770","reported_seq":28857,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:49.050774+0000","last_change":"2024-09-10T16:05:49.050774+0000","last_active":"2024-09-10T16:05:49.050774+0000","last_peered":"2024-09-10T16:05:49.050774+0000","last_clean":"2024-09-10T16:05:49.050774+0000","last_became_active":"2024-09-10T16:02:44.949385+0000","last_became_peered":"2024-09-10T16:02:44.949385+0000","last_unstale":"2024-09-10T16:05:49.050774+0000","last_undegraded":"2024-09-10T16:05:49.050774+0000","last_fullsized":"2024-09-10T16:05:49.050774+0000","mapping_epoch":51,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20770","last_scrub_stamp":"2024-09-10T16:05:49.050651+0000","last_deep_scrub":"50'20770","last_deep_scrub_stamp":"2024-09-10T16:05:49.050651+0000","last_clean_scrub_stamp":"2024-09-10T16:05:49.050651+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:42.761630+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20770,"num_write_kb":41540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'21074","reported_seq":21237,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:48.989814+0000","last_change":"2024-09-10T16:05:48.989814+0000","last_active":"2024-09-10T16:05:48.989814+0000","last_peered":"2024-09-10T16:05:48.989814+0000","last_clean":"2024-09-10T16:05:48.989814+0000","last_became_active":"2024-09-10T15:52:31.156572+0000","last_became_peered":"2024-09-10T15:52:31.156572+0000","last_unstale":"2024-09-10T16:05:48.989814+0000","last_undegraded":"2024-09-10T16:05:48.989814+0000","last_fullsized":"2024-09-10T16:05:48.989814+0000","mapping_epoch":36,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'21074","last_scrub_stamp":"2024-09-10T16:05:48.989725+0000","last_deep_scrub":"50'21074","last_deep_scrub_stamp":"2024-09-10T16:05:48.989725+0000","last_clean_scrub_stamp":"2024-09-10T16:05:48.989725+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:22.509263+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19410,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'20630","reported_seq":21153,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:54.798591+0000","last_change":"2024-09-10T16:05:54.798519+0000","last_active":"2024-09-10T16:05:54.798591+0000","last_peered":"2024-09-10T16:05:54.798591+0000","last_clean":"2024-09-10T16:05:54.798591+0000","last_became_active":"2024-09-10T16:03:46.314792+0000","last_became_peered":"2024-09-10T16:03:46.314792+0000","last_unstale":"2024-09-10T16:05:54.798591+0000","last_undegraded":"2024-09-10T16:05:54.798591+0000","last_fullsized":"2024-09-10T16:05:54.798591+0000","mapping_epoch":57,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20630","last_scrub_stamp":"2024-09-10T16:05:54.798406+0000","last_deep_scrub":"50'20630","last_deep_scrub_stamp":"2024-09-10T16:05:54.798406+0000","last_clean_scrub_stamp":"2024-09-10T16:05:54.798406+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:12.251283+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18898,"num_write_kb":34337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'20682","reported_seq":20818,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.054841+0000","last_change":"2024-09-10T16:05:47.054757+0000","last_active":"2024-09-10T16:05:47.054841+0000","last_peered":"2024-09-10T16:05:47.054841+0000","last_clean":"2024-09-10T16:05:47.054841+0000","last_became_active":"2024-09-10T15:52:31.156506+0000","last_became_peered":"2024-09-10T15:52:31.156506+0000","last_unstale":"2024-09-10T16:05:47.054841+0000","last_undegraded":"2024-09-10T16:05:47.054841+0000","last_fullsized":"2024-09-10T16:05:47.054841+0000","mapping_epoch":36,"log_start":"46'10600","ondisk_log_start":"46'10600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20682","last_scrub_stamp":"2024-09-10T16:05:47.054634+0000","last_deep_scrub":"50'20682","last_deep_scrub_stamp":"2024-09-10T16:05:47.054634+0000","last_clean_scrub_stamp":"2024-09-10T16:05:47.054634+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:12:25.730166+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20682,"num_write_kb":41364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'20331","reported_seq":20514,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:48.075361+0000","last_change":"2024-09-10T16:05:48.075361+0000","last_active":"2024-09-10T16:05:48.075361+0000","last_peered":"2024-09-10T16:05:48.075361+0000","last_clean":"2024-09-10T16:05:48.075361+0000","last_became_active":"2024-09-10T15:52:30.151909+0000","last_became_peered":"2024-09-10T15:52:30.151909+0000","last_unstale":"2024-09-10T16:05:48.075361+0000","last_undegraded":"2024-09-10T16:05:48.075361+0000","last_fullsized":"2024-09-10T16:05:48.075361+0000","mapping_epoch":35,"log_start":"46'10300","ondisk_log_start":"46'10300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20331","last_scrub_stamp":"2024-09-10T16:05:48.075269+0000","last_deep_scrub":"50'20331","last_deep_scrub_stamp":"2024-09-10T16:05:48.075269+0000","last_clean_scrub_stamp":"2024-09-10T16:05:48.075269+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:23:05.448307+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17402,"num_write_kb":28960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'20867","reported_seq":28811,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:55.774687+0000","last_change":"2024-09-10T16:05:55.774687+0000","last_active":"2024-09-10T16:05:55.774687+0000","last_peered":"2024-09-10T16:05:55.774687+0000","last_clean":"2024-09-10T16:05:55.774687+0000","last_became_active":"2024-09-10T15:58:12.769183+0000","last_became_peered":"2024-09-10T15:58:12.769183+0000","last_unstale":"2024-09-10T16:05:55.774687+0000","last_undegraded":"2024-09-10T16:05:55.774687+0000","last_fullsized":"2024-09-10T16:05:55.774687+0000","mapping_epoch":47,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20867","last_scrub_stamp":"2024-09-10T16:05:55.774574+0000","last_deep_scrub":"50'20867","last_deep_scrub_stamp":"2024-09-10T16:05:55.774574+0000","last_clean_scrub_stamp":"2024-09-10T16:05:55.774574+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:03.953365+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17585,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15843328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"50'20286","reported_seq":21078,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:56.812383+0000","last_change":"2024-09-10T16:05:56.812383+0000","last_active":"2024-09-10T16:05:56.812383+0000","last_peered":"2024-09-10T16:05:56.812383+0000","last_clean":"2024-09-10T16:05:56.812383+0000","last_became_active":"2024-09-10T16:05:33.033927+0000","last_became_peered":"2024-09-10T16:05:33.033927+0000","last_unstale":"2024-09-10T16:05:56.812383+0000","last_undegraded":"2024-09-10T16:05:56.812383+0000","last_fullsized":"2024-09-10T16:05:56.812383+0000","mapping_epoch":67,"log_start":"46'10200","ondisk_log_start":"46'10200","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20286","last_scrub_stamp":"2024-09-10T16:05:56.812288+0000","last_deep_scrub":"50'20286","last_deep_scrub_stamp":"2024-09-10T16:05:56.812288+0000","last_clean_scrub_stamp":"2024-09-10T16:05:56.812288+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:51.076723+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3837,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"50'20914","reported_seq":21101,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.048899+0000","last_change":"2024-09-10T16:05:47.048899+0000","last_active":"2024-09-10T16:05:47.048899+0000","last_peered":"2024-09-10T16:05:47.048899+0000","last_clean":"2024-09-10T16:05:47.048899+0000","last_became_active":"2024-09-10T15:52:30.152273+0000","last_became_peered":"2024-09-10T15:52:30.152273+0000","last_unstale":"2024-09-10T16:05:47.048899+0000","last_undegraded":"2024-09-10T16:05:47.048899+0000","last_fullsized":"2024-09-10T16:05:47.048899+0000","mapping_epoch":35,"log_start":"46'10900","ondisk_log_start":"46'10900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20914","last_scrub_stamp":"2024-09-10T16:05:47.048795+0000","last_deep_scrub":"50'20914","last_deep_scrub_stamp":"2024-09-10T16:05:47.048795+0000","last_clean_scrub_stamp":"2024-09-10T16:05:47.048795+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:41.736895+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17673,"num_write_kb":28868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":125,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:50.754736+0000","last_change":"2024-09-10T16:05:50.754736+0000","last_active":"2024-09-10T16:05:50.754736+0000","last_peered":"2024-09-10T16:05:50.754736+0000","last_clean":"2024-09-10T16:05:50.754736+0000","last_became_active":"2024-09-10T15:53:19.475546+0000","last_became_peered":"2024-09-10T15:53:19.475546+0000","last_unstale":"2024-09-10T16:05:50.754736+0000","last_undegraded":"2024-09-10T16:05:50.754736+0000","last_fullsized":"2024-09-10T16:05:50.754736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T16:05:50.754627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T16:05:50.754627+0000","last_clean_scrub_stamp":"2024-09-10T16:05:50.754627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:02.322192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000809975,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'20719","reported_seq":28903,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:48.818220+0000","last_change":"2024-09-10T16:05:48.818112+0000","last_active":"2024-09-10T16:05:48.818220+0000","last_peered":"2024-09-10T16:05:48.818220+0000","last_clean":"2024-09-10T16:05:48.818220+0000","last_became_active":"2024-09-10T15:55:20.685090+0000","last_became_peered":"2024-09-10T15:55:20.685090+0000","last_unstale":"2024-09-10T16:05:48.818220+0000","last_undegraded":"2024-09-10T16:05:48.818220+0000","last_fullsized":"2024-09-10T16:05:48.818220+0000","mapping_epoch":45,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20719","last_scrub_stamp":"2024-09-10T16:05:48.817964+0000","last_deep_scrub":"50'20719","last_deep_scrub_stamp":"2024-09-10T16:05:48.817964+0000","last_clean_scrub_stamp":"2024-09-10T16:05:48.817964+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:14.761466+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18696,"num_write_kb":33360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3994,"num_bytes_recovered":16351232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":150,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:45.092329+0000","last_change":"2024-09-10T16:05:45.092329+0000","last_active":"2024-09-10T16:05:45.092329+0000","last_peered":"2024-09-10T16:05:45.092329+0000","last_clean":"2024-09-10T16:05:45.092329+0000","last_became_active":"2024-09-10T15:52:31.159978+0000","last_became_peered":"2024-09-10T15:52:31.159978+0000","last_unstale":"2024-09-10T16:05:45.092329+0000","last_undegraded":"2024-09-10T16:05:45.092329+0000","last_fullsized":"2024-09-10T16:05:45.092329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T16:05:45.092260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T16:05:45.092260+0000","last_clean_scrub_stamp":"2024-09-10T16:05:45.092260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:33.664478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042619100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'20578","reported_seq":28499,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:46.048357+0000","last_change":"2024-09-10T16:05:46.048357+0000","last_active":"2024-09-10T16:05:46.048357+0000","last_peered":"2024-09-10T16:05:46.048357+0000","last_clean":"2024-09-10T16:05:46.048357+0000","last_became_active":"2024-09-10T16:04:28.868079+0000","last_became_peered":"2024-09-10T16:04:28.868079+0000","last_unstale":"2024-09-10T16:05:46.048357+0000","last_undegraded":"2024-09-10T16:05:46.048357+0000","last_fullsized":"2024-09-10T16:05:46.048357+0000","mapping_epoch":61,"log_start":"46'10500","ondisk_log_start":"46'10500","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20578","last_scrub_stamp":"2024-09-10T16:05:46.048257+0000","last_deep_scrub":"50'20578","last_deep_scrub_stamp":"2024-09-10T16:05:46.048257+0000","last_clean_scrub_stamp":"2024-09-10T16:05:46.048257+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:16.318293+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17442,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"68'3","reported_seq":158,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:45.042281+0000","last_change":"2024-09-10T16:05:45.042281+0000","last_active":"2024-09-10T16:05:45.042281+0000","last_peered":"2024-09-10T16:05:45.042281+0000","last_clean":"2024-09-10T16:05:45.042281+0000","last_became_active":"2024-09-10T15:53:19.891339+0000","last_became_peered":"2024-09-10T15:53:19.891339+0000","last_unstale":"2024-09-10T16:05:45.042281+0000","last_undegraded":"2024-09-10T16:05:45.042281+0000","last_fullsized":"2024-09-10T16:05:45.042281+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'3","last_scrub_stamp":"2024-09-10T16:05:45.042185+0000","last_deep_scrub":"68'3","last_deep_scrub_stamp":"2024-09-10T16:05:45.042185+0000","last_clean_scrub_stamp":"2024-09-10T16:05:45.042185+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:18.940437+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0015585600000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'20785","reported_seq":28641,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:48.036658+0000","last_change":"2024-09-10T16:05:48.036658+0000","last_active":"2024-09-10T16:05:48.036658+0000","last_peered":"2024-09-10T16:05:48.036658+0000","last_clean":"2024-09-10T16:05:48.036658+0000","last_became_active":"2024-09-10T16:00:38.953524+0000","last_became_peered":"2024-09-10T16:00:38.953524+0000","last_unstale":"2024-09-10T16:05:48.036658+0000","last_undegraded":"2024-09-10T16:05:48.036658+0000","last_fullsized":"2024-09-10T16:05:48.036658+0000","mapping_epoch":49,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20785","last_scrub_stamp":"2024-09-10T16:05:48.036535+0000","last_deep_scrub":"50'20785","last_deep_scrub_stamp":"2024-09-10T16:05:48.036535+0000","last_clean_scrub_stamp":"2024-09-10T16:05:48.036535+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:07.804808+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18759,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":5206016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"68'2","reported_seq":136,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:44.719293+0000","last_change":"2024-09-10T16:05:44.719209+0000","last_active":"2024-09-10T16:05:44.719293+0000","last_peered":"2024-09-10T16:05:44.719293+0000","last_clean":"2024-09-10T16:05:44.719293+0000","last_became_active":"2024-09-10T15:53:19.558054+0000","last_became_peered":"2024-09-10T15:53:19.558054+0000","last_unstale":"2024-09-10T16:05:44.719293+0000","last_undegraded":"2024-09-10T16:05:44.719293+0000","last_fullsized":"2024-09-10T16:05:44.719293+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'2","last_scrub_stamp":"2024-09-10T16:05:44.719120+0000","last_deep_scrub":"68'2","last_deep_scrub_stamp":"2024-09-10T16:05:44.719120+0000","last_clean_scrub_stamp":"2024-09-10T16:05:44.719120+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:25.916995+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00076395100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'20456","reported_seq":28309,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:50.005480+0000","last_change":"2024-09-10T16:05:50.005480+0000","last_active":"2024-09-10T16:05:50.005480+0000","last_peered":"2024-09-10T16:05:50.005480+0000","last_clean":"2024-09-10T16:05:50.005480+0000","last_became_active":"2024-09-10T16:03:06.090543+0000","last_became_peered":"2024-09-10T16:03:06.090543+0000","last_unstale":"2024-09-10T16:05:50.005480+0000","last_undegraded":"2024-09-10T16:05:50.005480+0000","last_fullsized":"2024-09-10T16:05:50.005480+0000","mapping_epoch":53,"log_start":"46'10400","ondisk_log_start":"46'10400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20456","last_scrub_stamp":"2024-09-10T16:05:50.005389+0000","last_deep_scrub":"50'20456","last_deep_scrub_stamp":"2024-09-10T16:05:50.005389+0000","last_clean_scrub_stamp":"2024-09-10T16:05:50.005389+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:41.014266+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18325,"num_write_kb":32400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3831,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'20854","reported_seq":21747,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:52.770483+0000","last_change":"2024-09-10T16:05:52.770483+0000","last_active":"2024-09-10T16:05:52.770483+0000","last_peered":"2024-09-10T16:05:52.770483+0000","last_clean":"2024-09-10T16:05:52.770483+0000","last_became_active":"2024-09-10T16:03:24.140701+0000","last_became_peered":"2024-09-10T16:03:24.140701+0000","last_unstale":"2024-09-10T16:05:52.770483+0000","last_undegraded":"2024-09-10T16:05:52.770483+0000","last_fullsized":"2024-09-10T16:05:52.770483+0000","mapping_epoch":55,"log_start":"46'10800","ondisk_log_start":"46'10800","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20854","last_scrub_stamp":"2024-09-10T16:05:52.770391+0000","last_deep_scrub":"50'20854","last_deep_scrub_stamp":"2024-09-10T16:05:52.770391+0000","last_clean_scrub_stamp":"2024-09-10T16:05:52.770391+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:34.867210+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18046,"num_write_kb":30480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":125,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:46.800740+0000","last_change":"2024-09-10T16:05:46.800683+0000","last_active":"2024-09-10T16:05:46.800740+0000","last_peered":"2024-09-10T16:05:46.800740+0000","last_clean":"2024-09-10T16:05:46.800740+0000","last_became_active":"2024-09-10T15:53:19.599164+0000","last_became_peered":"2024-09-10T15:53:19.599164+0000","last_unstale":"2024-09-10T16:05:46.800740+0000","last_undegraded":"2024-09-10T16:05:46.800740+0000","last_fullsized":"2024-09-10T16:05:46.800740+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T16:05:46.800588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T16:05:46.800588+0000","last_clean_scrub_stamp":"2024-09-10T16:05:46.800588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:48.409752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051074699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'20718","reported_seq":21498,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:53.764559+0000","last_change":"2024-09-10T16:05:53.764559+0000","last_active":"2024-09-10T16:05:53.764559+0000","last_peered":"2024-09-10T16:05:53.764559+0000","last_clean":"2024-09-10T16:05:53.764559+0000","last_became_active":"2024-09-10T16:05:09.969485+0000","last_became_peered":"2024-09-10T16:05:09.969485+0000","last_unstale":"2024-09-10T16:05:53.764559+0000","last_undegraded":"2024-09-10T16:05:53.764559+0000","last_fullsized":"2024-09-10T16:05:53.764559+0000","mapping_epoch":65,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20718","last_scrub_stamp":"2024-09-10T16:05:53.764466+0000","last_deep_scrub":"50'20718","last_deep_scrub_stamp":"2024-09-10T16:05:53.764466+0000","last_clean_scrub_stamp":"2024-09-10T16:05:53.764466+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:42.615014+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17136,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":150,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:46.059480+0000","last_change":"2024-09-10T16:05:46.059480+0000","last_active":"2024-09-10T16:05:46.059480+0000","last_peered":"2024-09-10T16:05:46.059480+0000","last_clean":"2024-09-10T16:05:46.059480+0000","last_became_active":"2024-09-10T15:52:31.157054+0000","last_became_peered":"2024-09-10T15:52:31.157054+0000","last_unstale":"2024-09-10T16:05:46.059480+0000","last_undegraded":"2024-09-10T16:05:46.059480+0000","last_fullsized":"2024-09-10T16:05:46.059480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T16:05:46.059377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T16:05:46.059377+0000","last_clean_scrub_stamp":"2024-09-10T16:05:46.059377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:02.304928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'20766","reported_seq":21635,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:51.774505+0000","last_change":"2024-09-10T16:05:51.774505+0000","last_active":"2024-09-10T16:05:51.774505+0000","last_peered":"2024-09-10T16:05:51.774505+0000","last_clean":"2024-09-10T16:05:51.774505+0000","last_became_active":"2024-09-10T16:04:08.810870+0000","last_became_peered":"2024-09-10T16:04:08.810870+0000","last_unstale":"2024-09-10T16:05:51.774505+0000","last_undegraded":"2024-09-10T16:05:51.774505+0000","last_fullsized":"2024-09-10T16:05:51.774505+0000","mapping_epoch":59,"log_start":"46'10700","ondisk_log_start":"46'10700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'20766","last_scrub_stamp":"2024-09-10T16:05:51.774400+0000","last_deep_scrub":"50'20766","last_deep_scrub_stamp":"2024-09-10T16:05:51.774400+0000","last_clean_scrub_stamp":"2024-09-10T16:05:51.774400+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:01.041932+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17372,"num_write_kb":27964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3880,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":124,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:45.760297+0000","last_change":"2024-09-10T16:05:45.760297+0000","last_active":"2024-09-10T16:05:45.760297+0000","last_peered":"2024-09-10T16:05:45.760297+0000","last_clean":"2024-09-10T16:05:45.760297+0000","last_became_active":"2024-09-10T15:53:19.597845+0000","last_became_peered":"2024-09-10T15:53:19.597845+0000","last_unstale":"2024-09-10T16:05:45.760297+0000","last_undegraded":"2024-09-10T16:05:45.760297+0000","last_fullsized":"2024-09-10T16:05:45.760297+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T16:05:45.760158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T16:05:45.760158+0000","last_clean_scrub_stamp":"2024-09-10T16:05:45.760158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:57.303731+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00049597499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":129,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:49.788327+0000","last_change":"2024-09-10T16:05:49.788327+0000","last_active":"2024-09-10T16:05:49.788327+0000","last_peered":"2024-09-10T16:05:49.788327+0000","last_clean":"2024-09-10T16:05:49.788327+0000","last_became_active":"2024-09-10T15:53:19.473305+0000","last_became_peered":"2024-09-10T15:53:19.473305+0000","last_unstale":"2024-09-10T16:05:49.788327+0000","last_undegraded":"2024-09-10T16:05:49.788327+0000","last_fullsized":"2024-09-10T16:05:49.788327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T16:05:49.788163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T16:05:49.788163+0000","last_clean_scrub_stamp":"2024-09-10T16:05:49.788163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:28.918249+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001001947,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"68'21099","reported_seq":21894,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T16:05:47.858639+0000","last_change":"2024-09-10T16:05:47.858639+0000","last_active":"2024-09-10T16:05:47.858639+0000","last_peered":"2024-09-10T16:05:47.858639+0000","last_clean":"2024-09-10T16:05:47.858639+0000","last_became_active":"2024-09-10T16:04:50.936548+0000","last_became_peered":"2024-09-10T16:04:50.936548+0000","last_unstale":"2024-09-10T16:05:47.858639+0000","last_undegraded":"2024-09-10T16:05:47.858639+0000","last_fullsized":"2024-09-10T16:05:47.858639+0000","mapping_epoch":63,"log_start":"46'11000","ondisk_log_start":"46'11000","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'21099","last_scrub_stamp":"2024-09-10T16:05:47.858563+0000","last_deep_scrub":"68'21099","last_deep_scrub_stamp":"2024-09-10T16:05:47.858563+0000","last_clean_scrub_stamp":"2024-09-10T16:05:47.858563+0000","objects_scrubbed":1,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:41.963343+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17917,"num_write_kb":29496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4006,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":293049,"num_write_kb":509301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46648,"num_bytes_recovered":37408768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36134912,"data_stored":36124073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160929,"ondisk_log_size":160929,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":42,"seq":180388626590,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515328,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":514747,"kb_avail":93856512,"statfs":{"total":96636764160,"available":96109068288,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":527101614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505600,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866240,"statfs":{"total":96636764160,"available":96119029760,"internally_reserved":0,"allocated":573440,"data_stored":189742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":517140141},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":32,"seq":137438953642,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":396728,"kb_used_data":36072,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93975112,"statfs":{"total":96636764160,"available":96230514688,"internally_reserved":0,"allocated":36937728,"data_stored":36558107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36126720,"data_stored":36123107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T16:06:05.503 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T16:06:05.805 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T16:06:05.806 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T16:06:05.806 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T16:06:05.806 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:06:10.044 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-10T16:06:11.908 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T16:06:11.908 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T16:06:11.909 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:06:11.978 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T16:06:11.978 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T16:06:11.978 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:06:15.447 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-10T16:06:15.447 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-10T16:06:18.079 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T16:06:18.080 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T16:06:18.080 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T16:06:18.080 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:06:18.111 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T16:06:18.111 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T16:06:18.111 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T16:06:18.111 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:06:18.165 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T16:06:18.165 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T16:06:18.165 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:06:18.218 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T16:06:18.219 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T16:06:18.219 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T16:06:18.272 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T16:06:18.272 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T16:06:18.272 DEBUG:teuthology.orchestra.run.smithi121:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v '\(MON_DOWN\)' | head -n 1 2024-09-10T16:06:18.368 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi121.front.sepia.ceph.com 2024-09-10T16:06:18.369 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T16:06:18.595 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi121.front.sepia.ceph.com 2024-09-10T16:06:18.595 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T16:06:18.714 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi121.front.sepia.ceph.com 2024-09-10T16:06:18.714 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T16:06:18.793 INFO:tasks.ceph:Archiving mon data... 2024-09-10T16:06:18.794 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899253/data/mon.a.tgz 2024-09-10T16:06:18.794 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-09-10T16:06:18.822 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.RRSo7pt2mV 2024-09-10T16:06:18.823 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.RRSo7pt2mV 2024-09-10T16:06:18.952 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.RRSo7pt2mV 2024-09-10T16:06:19.061 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.RRSo7pt2mV is 310KB 2024-09-10T16:06:19.131 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.RRSo7pt2mV 2024-09-10T16:06:19.146 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899253/data/mon.b.tgz 2024-09-10T16:06:19.146 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-09-10T16:06:19.201 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.WiHRb2NWpz 2024-09-10T16:06:19.202 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.WiHRb2NWpz 2024-09-10T16:06:19.316 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.WiHRb2NWpz 2024-09-10T16:06:19.399 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.WiHRb2NWpz is 310KB 2024-09-10T16:06:19.465 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.WiHRb2NWpz 2024-09-10T16:06:19.480 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899253/data/mon.c.tgz 2024-09-10T16:06:19.480 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-09-10T16:06:19.534 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.JGqKUIRQsq 2024-09-10T16:06:19.535 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.JGqKUIRQsq 2024-09-10T16:06:19.650 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.JGqKUIRQsq 2024-09-10T16:06:19.733 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.JGqKUIRQsq is 310KB 2024-09-10T16:06:19.799 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.JGqKUIRQsq 2024-09-10T16:06:19.814 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T16:06:19.814 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T16:06:19.934 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T16:06:19.934 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T16:06:19.936 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899253/remote/smithi121/crash 2024-09-10T16:06:19.937 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T16:06:19.998 INFO:tasks.ceph:Compressing logs... 2024-09-10T16:06:19.998 DEBUG:teuthology.orchestra.run.smithi121:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T16:06:20.342 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68241.log 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62526.log 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44090.log 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68241.log.gz 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53503.log 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62526.log.gz 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53461.log 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44090.log.gz 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64460.log 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68829.log 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50479.log 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64460.log.gz 2024-09-10T16:06:20.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57703.log 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68829.log.gz 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63115.log 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37232.log 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59859.log 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63115.log.gz 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57493.log 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56401.log 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39110.log 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38962.log 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70131.log 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61334.log 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63409.log 2024-09-10T16:06:20.344 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70131.log.gz 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52453.log 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68703.log 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50479.log.gz 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63409.log.gz 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45686.log 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52453.log.gz 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49387.log 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68703.log.gz 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53461.log.gz 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53503.log.gz 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57493.log.gz 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59859.log.gz 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56401.log.gz 2024-09-10T16:06:20.345 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61334.log.gz 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45686.log.gz 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57703.log.gz 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65594.log 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49387.log.gz 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46338.log 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43610.log 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65594.log.gz 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54889.log 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46338.log.gz 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57955.log 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70887.log 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54889.log.gz 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44174.log 2024-09-10T16:06:20.346 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57955.log.gz 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53545.log 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70887.log.gz 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57283.log 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44174.log.gz 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66561.log 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53545.log.gz 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69165.log 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57283.log.gz 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62736.log 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66561.log.gz 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53209.log 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69165.log.gz 2024-09-10T16:06:20.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52705.log 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62736.log.gz 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57367.log 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53209.log.gz 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56023.log 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52705.log.gz 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52117.log 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57367.log.gz 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36998.log 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56023.log.gz 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70509.log 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52117.log.gz 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64838.log 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58291.log 2024-09-10T16:06:20.348 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70509.log.gz 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55897.log 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64838.log.gz 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50353.log 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58291.log.gz 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68199.log 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55897.log.gz 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44888.log 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50353.log.gz 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58207.log 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68199.log.gz 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43274.log 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44888.log.gz 2024-09-10T16:06:20.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51403.log 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58207.log.gz 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54091.log 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38703.log 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51403.log.gz 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63073.log 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54091.log.gz 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57199.log 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38703.log: /var/log/ceph/ceph-client.admin.63073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38851.log 0.0% 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63073.log.gz 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36442.log 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57199.log.gz 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52537.log 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63199.log 2024-09-10T16:06:20.350 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36442.log.gz 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42148.log 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52537.log.gz 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57787.log 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63199.log.gz 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66603.log 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51235.log 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57787.log.gz 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67737.log 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66603.log.gz 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66393.log 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51235.log.gz 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63241.log 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67737.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67737.log.gz 2024-09-10T16:06:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71979.log 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66393.log.gz 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59355.log 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63241.log.gz 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70173.log 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71979.log.gz 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58977.log 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59355.log.gz 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45820.log 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70173.log.gz 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40156.log 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58977.log.gz 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59733.log 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45820.log.gz 2024-09-10T16:06:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55267.log 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69627.log 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59733.log.gz 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59649.log 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55267.log.gz 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64208.log 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69627.log.gz 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:gzip/var/log/ceph/ceph-client.admin.59649.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38555.log 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59649.log.gz 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39823.log 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64208.log.gz 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37736.log 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60494.log 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65426.log 2024-09-10T16:06:20.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63997.log 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60494.log.gz 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55015.log 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37232.log.gz 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65426.log.gz 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36998.log.gz 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39823.log.gz 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38703.log.gz -- replaced with /var/log/ceph/ceph-client.admin.43610.log.gz 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37736.log.gz 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39110.log.gz 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38555.log.gz 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67989.log 2024-09-10T16:06:20.354 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38851.log.gz 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40156.log.gz 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38962.log.gz 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42148.log.gz 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63997.log.gz 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43274.log.gz 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52873.log 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55015.log.gz 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71181.log 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67989.log.gz 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52747.log 0.0% 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52873.log.gz 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36724.log 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71181.log.gz 2024-09-10T16:06:20.355 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64880.log 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52747.log.gz 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36724.log.gz 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66225.log 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64880.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64880.log.gz 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66225.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.66225.log.gz 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/ceph-client.admin.66309.log 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57409.log 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63577.log 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66309.log.gz 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37568.log 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57409.log.gz 2024-09-10T16:06:20.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54847.log 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63577.log.gz 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39971.log 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37568.log.gz 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39406.log 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54847.log.gz 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67905.log 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39971.log.gz 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44720.log 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39406.log.gz 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58851.log 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67905.log.gz 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50731.log 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44720.log.gz 2024-09-10T16:06:20.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55519.log 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58851.log.gz 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50731.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50731.log.gz 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37778.log 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61376.log 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55519.log.gz 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55939.log 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37778.log.gz 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53419.log 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61376.log.gz 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48463.log 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55939.log.gz 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47875.log 2024-09-10T16:06:20.358 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53419.log.gz 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43693.log 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48463.log.gz 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60788.log 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47875.log.gz 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47749.log 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43693.log.gz 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37481.log 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60788.log.gz 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71307.log 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47749.log.gz 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46560.log 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37481.log.gz 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56107.log 2024-09-10T16:06:20.359 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71307.log.gz 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41663.log 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46560.log.gz 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71559.log 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56107.log.gz 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64081.log 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41663.log.gz 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32240.log 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71559.log.gz 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48799.log 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64081.log.gz 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.32240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59985.log 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32240.log.gz 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44972.log 2024-09-10T16:06:20.360 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48799.log.gz 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45602.log 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59985.log.gz 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55603.log 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44972.log.gz 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57619.log 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45602.log.gz 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63535.log 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55603.log.gz 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66519.log 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57619.log.gz 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60998.log 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63535.log.gz 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56149.log 2024-09-10T16:06:20.361 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66519.log.gz 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71055.log 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60998.log.gz 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39712.log 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56149.log.gz 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60284.log 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71055.log.gz 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61670.log 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39712.log.gz 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39073.log 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60284.log.gz 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37482.log 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61670.log.gz 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52915.log 2024-09-10T16:06:20.362 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39073.log.gz 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66939.log 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37482.log.gz 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52915.log: /var/log/ceph/ceph-client.admin.54679.log 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52915.log.gz 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53839.log 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66939.log.gz 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59019.log 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54679.log.gz 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60704.log 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53839.log.gz 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48673.log 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59019.log.gz 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60662.log 2024-09-10T16:06:20.363 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60704.log.gz 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64334.log 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48673.log.gz 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70677.log 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60662.log.gz 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58767.log 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64334.log.gz 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45857.log 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70677.log.gz 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70929.log 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58767.log.gz 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49093.log 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45857.log.gz 2024-09-10T16:06:20.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64123.log 2024-09-10T16:06:20.365 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70929.log.gz 2024-09-10T16:06:20.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65930.log 2024-09-10T16:06:20.365 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49093.log.gz 2024-09-10T16:06:20.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64123.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.64123.log.gz 2024-09-10T16:06:20.365 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59901.log 2024-09-10T16:06:20.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71895.log 2024-09-10T16:06:20.365 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65930.log.gz 2024-09-10T16:06:20.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56695.log 2024-09-10T16:06:20.365 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59901.log.gz 2024-09-10T16:06:20.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68661.log 2024-09-10T16:06:20.365 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71895.log.gz 2024-09-10T16:06:20.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56695.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.56695.log.gz --verbose 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/ceph-client.admin.62568.log 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68661.log.gz 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44258.log 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50815.log 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62568.log.gz 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52411.log 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44258.log.gz 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60158.log 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50815.log.gz 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54637.log 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52411.log.gz 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48295.log 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60158.log.gz 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54343.log 2024-09-10T16:06:20.366 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54637.log.gz 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36560.log 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48295.log.gz 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40341.log 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54343.log.gz 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46486.log 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36560.log.gz 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64754.log 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40341.log.gz 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53251.log 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46486.log.gz 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65678.log 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64754.log.gz 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62105.log 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53251.log.gz 2024-09-10T16:06:20.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49513.log 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65678.log.gz 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57157.log 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62105.log.gz 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50899.log 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49513.log.gz 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72271.log 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57157.log.gz 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50311.log 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50899.log.gz 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.72271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54973.log 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72271.log.gz 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58809.log 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50311.log.gz 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60746.log 2024-09-10T16:06:20.368 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54973.log.gz 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58809.log.gz 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67023.log 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49555.log 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60746.log.gz 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51865.log 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67023.log.gz 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36958.log 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49555.log.gz 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60830.log 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51865.log.gz 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60116.log 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36958.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36958.log.gz 2024-09-10T16:06:20.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53923.log 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60830.log.gz 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46005.log 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60116.log.gz 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47917.log 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53923.log.gz 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49261.log 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46005.log.gz 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53125.log 0.0% 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47917.log.gz 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70257.log 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49261.log.gz 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45968.log 2024-09-10T16:06:20.370 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53125.log.gz 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69669.log 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70257.log.gz 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45968.log.gz 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48715.log 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70803.log 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69669.log.gz 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55561.log 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48715.log.gz 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58377.log 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70803.log.gz 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65300.log 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55561.log.gz 2024-09-10T16:06:20.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55225.log 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58377.log.gz 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52033.log 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65300.log.gz 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71601.log 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55225.log.gz 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43189.log 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52033.log.gz 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37274.log 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71601.log.gz 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43189.log.gz 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50437.log 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49219.log 2024-09-10T16:06:20.372 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37274.log.gz 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57031.log 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50437.log.gz 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58725.log 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49219.log.gz 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55477.log 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57031.log.gz 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69879.log 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58725.log.gz 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66435.log 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55477.log.gz 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39860.log 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69879.log.gz 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48589.log 2024-09-10T16:06:20.373 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66435.log.gz 2024-09-10T16:06:20.374 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65006.log 2024-09-10T16:06:20.374 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39860.log.gz 2024-09-10T16:06:20.374 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48631.log 2024-09-10T16:06:20.374 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48589.log.gz 2024-09-10T16:06:20.374 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63493.log 2024-09-10T16:06:20.374 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65006.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65006.log.gz 2024-09-10T16:06:20.374 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47959.log 2024-09-10T16:06:20.374 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48631.log.gz 2024-09-10T16:06:20.374 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48085.log 2024-09-10T16:06:20.374 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63493.log.gz 2024-09-10T16:06:20.374 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63955.log 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47959.log.gz 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61040.log 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48085.log.gz 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53083.log 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63955.log.gz 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37930.log 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61040.log.gz 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54007.log 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53083.log.gz 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67821.log 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37930.log.gz 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43506.log 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54007.log.gz 2024-09-10T16:06:20.375 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36644.log 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67821.log.gz 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39517.log 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43506.log.gz 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62779.log 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36644.log.gz 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36386.log 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39517.log.gz 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49429.log 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62779.log.gz 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50605.log 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36386.log.gz 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49303.log 2024-09-10T16:06:20.376 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49429.log.gz 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64796.log 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50605.log.gz 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71391.log 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49303.log.gz 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40452.log 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64796.log.gz 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55309.log 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71391.log.gz 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51487.log 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40452.log.gz 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40378.log 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55309.log.gz 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49933.log 2024-09-10T16:06:20.377 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51487.log.gz 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59439.log 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40378.log.gz 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55435.log 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49933.log.gz 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44510.log 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59439.log.gz 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71433.log 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55435.log.gz 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44594.log 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44510.log.gz 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55393.log 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71433.log.gz 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54595.log 2024-09-10T16:06:20.378 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44594.log.gz 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32399.log 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55393.log.gz 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62442.log 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54595.log.gz 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.32399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65636.log 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32399.log.gz 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43777.log 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62442.log.gz 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61124.log 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65636.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65636.log.gz 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44636.log 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43777.log.gz 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39897.log 2024-09-10T16:06:20.379 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61124.log.gz 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57871.log 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44636.log.gz 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45014.log 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39897.log.gz 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49597.log 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57871.log.gz 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63283.log 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45014.log.gz 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49597.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49597.log.gz 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41832.log 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63283.log.gz 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69963.log 2024-09-10T16:06:20.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32359.log 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41832.log.gz 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59061.log 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69963.log.gz 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62063.log 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.32359.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32359.log.gz 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50773.log 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59061.log.gz 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70761.log 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62063.log.gz 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63619.log 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50773.log.gz 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54133.log 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70761.log.gz 2024-09-10T16:06:20.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48841.log 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63619.log.gz 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60074.log 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54133.log.gz 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69711.log 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48841.log.gz 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60074.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60074.log.gz 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46301.log 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42064.log 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69711.log.gz 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37190.log 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46301.log.gz 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38090.log 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42064.log.gz 2024-09-10T16:06:20.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37190.log.gz 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64670.log 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43190.log 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38090.log.gz 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70047.log 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64670.log.gz 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59145.log 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gz 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:gzip/var/log/ceph/ceph-client.admin.70047.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.67569.log 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70047.log.gz 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64712.log 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59145.log.gz 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69921.log 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67569.log.gz 2024-09-10T16:06:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36852.log 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64712.log.gz 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58557.log 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69921.log.gz 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71223.log 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36852.log.gz 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68745.log 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58557.log.gz 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52243.log 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71223.log.gz 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52579.log 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68745.log.gz 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70341.log 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52243.log.gz 2024-09-10T16:06:20.384 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39036.log 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52579.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52579.log.gz 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58515.log 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70341.log.gz 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49765.log 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39036.log.gz 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56611.log 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58515.log.gz 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63157.log 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49765.log.gz 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35961.log 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56611.log.gz 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40008.log 2024-09-10T16:06:20.385 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63157.log.gz 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.35961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52999.log 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35961.log.gz 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56485.log 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40008.log.gz 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39573.log 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52999.log.gz 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65804.log 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56485.log.gz 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58599.log 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39573.log.gz 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66813.log 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65804.log.gz 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44048.log 2024-09-10T16:06:20.386 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58599.log.gz 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38322.log 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66813.log.gz 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66981.log 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44048.log.gz 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45560.log 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38322.log.gz 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43422.log 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66981.log.gz 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48421.log 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45560.log.gz 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52369.log 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43422.log.gz 2024-09-10T16:06:20.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48421.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48421.log.gz 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37907.log 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59397.log 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52369.log.gz 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53587.log 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37907.log.gz 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43609.log 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59397.log.gz 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60872.log 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53587.log.gz 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59775.log 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43609.log.gz 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62231.log 2024-09-10T16:06:20.388 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60872.log.gz 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59775.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.56779.log 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59775.log.gz 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52621.log 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62231.log.gz 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57535.log 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56779.log.gz 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46449.log 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52621.log.gz 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56569.log 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57535.log.gz 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39638.log 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46449.log.gz 2024-09-10T16:06:20.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56905.log 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56569.log.gz 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52327.log 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39638.log.gz 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40230.log 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56905.log.gz 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61628.log 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52327.log.gz 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45894.log 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40230.log.gz 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46153.log 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61628.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61628.log.gz 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67191.log 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45894.log.gz 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69375.log 2024-09-10T16:06:20.390 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46153.log.gz 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48253.log 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67191.log.gz 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41664.log 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69375.log.gz 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68577.log 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48253.log.gz 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67359.log 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41664.log.gz 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68577.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68577.log.gz 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38258.log 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66897.log 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67359.log.gz 2024-09-10T16:06:20.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72231.log 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38258.log.gz 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68031.log 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66897.log.gz 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.72231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65342.log 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72231.log.gz 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58641.log 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68031.log.gz 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41747.log 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65342.log.gz 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47707.log 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58641.log.gz 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71475.log 2024-09-10T16:06:20.392 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41747.log.gz 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60200.log 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47707.log.gz 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65384.log 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71475.log.gz 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71685.log 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60200.log.gz 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68787.log 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65384.log.gz 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39221.log 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71685.log.gz 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56989.log 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68787.log.gz 2024-09-10T16:06:20.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57997.log 2024-09-10T16:06:20.394 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39221.log.gz 2024-09-10T16:06:20.394 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72063.log 2024-09-10T16:06:20.394 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56989.log.gz 2024-09-10T16:06:20.394 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48043.log 2024-09-10T16:06:20.394 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57997.log.gz 2024-09-10T16:06:20.394 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.72063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51151.log 2024-09-10T16:06:20.394 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72063.log.gz 2024-09-10T16:06:20.394 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32605.log 2024-09-10T16:06:20.394 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48043.log.gz 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60914.log 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51151.log.gz 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.32605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37356.log 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32605.log.gz 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69459.log 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60914.log.gz 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62357.log 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37356.log.gz 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38629.log 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69459.log.gz 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55645.log 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62357.log.gz 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67233.log 2024-09-10T16:06:20.395 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38629.log.gz 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50983.log 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55645.log.gz 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69081.log 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67233.log.gz 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54721.log 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50983.log.gz 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38999.log 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69081.log.gz 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5/var/log/ceph/ceph-client.admin.54721.log: --verbose -- /var/log/ceph/ceph-client.admin.71811.log 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54721.log.gz 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69501.log 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38999.log.gz 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56527.log 2024-09-10T16:06:20.396 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71811.log.gz 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41831.log 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69501.log.gz 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68913.log 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56527.log.gz 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51781.log 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41831.log.gz 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62147.log 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68913.log.gz 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51781.log.gz 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67317.log 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58123.log 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62147.log.gz 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61895.log 2024-09-10T16:06:20.397 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67317.log.gz 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60578.log 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58123.log.gz 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61895.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.61895.log.gz -5 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38490.log 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50395.log 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60578.log.gz 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36851.log 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38490.log.gz 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37480.log 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50395.log.gz 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr:gzip/var/log/ceph/ceph-client.admin.36851.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.72147.log 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36851.log.gz 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62273.log 2024-09-10T16:06:20.398 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37480.log.gz 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.72147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55771.log 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72147.log.gz 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44426.log 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62273.log.gz 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58461.log 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55771.log.gz 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70299.log 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44426.log.gz 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45140.log 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58461.log.gz 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68367.log 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70299.log.gz 2024-09-10T16:06:20.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49891.log 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45140.log.gz 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71349.log 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68367.log.gz 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48337.log 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49891.log.gz 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62610.log 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71349.log.gz 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43922.log 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48337.log.gz 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63913.log 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62610.log.gz 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr:gzip/var/log/ceph/ceph-client.admin.43922.log: -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T16:06:20.400 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43922.log.gz 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48127.log 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63913.log.gz 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5/var/log/ceph/ceph-mon.c.log: --verbose -- /var/log/ceph/ceph-client.admin.69795.log 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44762.log 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48127.log.gz 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57745.log 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69795.log.gz 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55141.log 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44762.log.gz 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45244.log 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57745.log.gz 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49051.log 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55141.log.gz 2024-09-10T16:06:20.401 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49009.log 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45244.log.gz 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56947.log 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49051.log.gz 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61250.log 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49009.log.gz 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39786.log 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56947.log.gz 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45412.log 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61250.log.gz 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46523.log 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39786.log.gz 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66183.log 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45412.log.gz 2024-09-10T16:06:20.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38532.log 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46523.log.gz 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr: 95.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55351.log 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66183.log.gz 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55687.log 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38532.log.gz 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65888.log 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55351.log.gz 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57115.log 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55687.log.gz 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60031.log 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65888.log.gz 2024-09-10T16:06:20.403 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.57115.log: -- /var/log/ceph/ceph-client.admin.32538.log 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57115.log.gz 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36518.log 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60031.log.gz 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.32538.log: /var/log/ceph/ceph-client.admin.39369.log 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32538.log.gz 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48211.log 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36518.log.gz 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56737.log 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39369.log.gz 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61082.log 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48211.log.gz 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45476.log 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56737.log.gz 2024-09-10T16:06:20.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61082.log.gz 2024-09-10T16:06:20.405 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38448.log 2024-09-10T16:06:20.405 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61796.log 2024-09-10T16:06:20.405 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45476.log.gz 2024-09-10T16:06:20.405 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50647.log 2024-09-10T16:06:20.405 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38448.log.gz 2024-09-10T16:06:20.405 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61292.log 2024-09-10T16:06:20.405 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61796.log.gz 2024-09-10T16:06:20.405 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65468.log 2024-09-10T16:06:20.405 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50647.log.gz 2024-09-10T16:06:20.405 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45098.log 2024-09-10T16:06:20.405 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61292.log.gz 2024-09-10T16:06:20.405 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58334.log 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65468.log.gz 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr:gzip/var/log/ceph/ceph-client.admin.45098.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55813.log 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45098.log.gz 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57325.log 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58334.log.gz 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43694.log 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55813.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55813.log.gz 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60956.log 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57325.log.gz 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64376.log 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43694.log.gz 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67401.log 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60956.log.gz 2024-09-10T16:06:20.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58935.log 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64376.log.gz 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67401.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.67401.log.gz 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/ceph-client.admin.59481.log 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36467.log 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58935.log.gz 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65972.log 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59481.log.gz 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5/var/log/ceph/ceph-client.admin.36467.log: --verbose -- /var/log/ceph/ceph-client.admin.61166.log 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36467.log.gz 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65972.log.gz 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60326.log 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68157.log 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61166.log.gz 2024-09-10T16:06:20.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43357.log 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60326.log.gz 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65258.log 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68157.log.gz 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39147.log 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43357.log.gz 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62694.log 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65258.log.gz 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54469.log 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39147.log.gz 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38174.log 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62694.log.gz 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61544.log 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54469.log.gz 2024-09-10T16:06:20.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40415.log 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38174.log.gz 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32064.log 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61544.log.gz 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72189.log 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40415.log.gz 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.32064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72105.log 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32064.log.gz 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.72189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44300.log 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72189.log.gz 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.72105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69585.log 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72105.log.gz 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44300.log: /var/log/ceph/ceph-client.admin.66141.log 2024-09-10T16:06:20.409 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44300.log.gz 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55981.log 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69585.log.gz 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66141.log.gz 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56443.log 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68283.log 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55981.log.gz 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68997.log 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56443.log.gz 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43273.log 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68283.log.gz 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42106.log 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68997.log.gz 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48547.log 2024-09-10T16:06:20.410 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43273.log.gz 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38888.log 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42106.log.gz 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39295.log 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48547.log.gz 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39615.log 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38888.log.gz 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72021.log 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39295.log.gz 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53335.log 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39615.log.gz 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.72021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52957.log 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72021.log.gz 2024-09-10T16:06:20.411 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68871.log 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53335.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53335.log.gz 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71643.log 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52957.log.gz 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37354.log 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68871.log.gz 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51739.log 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71643.log.gz 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56317.log 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37354.log.gz 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53881.log 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51739.log.gz 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51445.log 2024-09-10T16:06:20.412 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56317.log.gz 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53881.log.gz 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53041.log 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:gzip/var/log/ceph/ceph-client.admin.51445.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.67695.log 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51445.log.gz 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41602.log 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53041.log.gz 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68073.log 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67695.log.gz 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41602.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41602.log.gz 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60620.log 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68073.log.gz 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50269.log 2024-09-10T16:06:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38005.log 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60620.log.gz 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50227.log 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50269.log.gz 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60536.log 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38005.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38005.log.gz 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41560.log 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50227.log.gz 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45328.log 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60536.log.gz 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63745.log 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41560.log.gz 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45328.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45328.log.gz 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/ceph-client.admin.37608.log 2024-09-10T16:06:20.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40526.log 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63745.log.gz 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53797.log 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37608.log.gz 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56275.log 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40526.log.gz 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40304.log 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53797.log.gz 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40045.log 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56275.log.gz 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38925.log 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40304.log.gz 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49975.log 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40045.log.gz 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49807.log 2024-09-10T16:06:20.415 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38925.log.gz 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63367.log 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49975.log.gz 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65720.log 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49807.log.gz 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53293.log 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63367.log.gz 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65720.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65720.log.gz -5 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61502.log 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54259.log 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53293.log.gz 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48757.log 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61502.log.gz 2024-09-10T16:06:20.416 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.54259.log: -- /var/log/ceph/ceph-client.admin.36602.log 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54259.log.gz 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68325.log 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48757.log.gz 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36602.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36602.log.gz 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63703.log 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48967.log 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68325.log.gz 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41980.log 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63703.log.gz 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54049.log 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48967.log.gz 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5/var/log/ceph/ceph-client.admin.41980.log: --verbose -- /var/log/ceph/ceph-client.admin.66014.log 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41980.log.gz 2024-09-10T16:06:20.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40489.log 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54049.log.gz 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58081.log 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66014.log.gz 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57661.log 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40489.log.gz 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65174.log 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58081.log.gz 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58683.log 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57661.log.gz 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44342.log 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65174.log.gz 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57073.log 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58683.log.gz 2024-09-10T16:06:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60410.log 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44342.log.gz 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.57073.log: /var/log/ceph/ceph-client.admin.50563.log 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57073.log.gz 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40119.log 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60410.log.gz 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64418.log 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50563.log.gz 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54427.log 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40119.log.gz 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54553.log 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64418.log.gz 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55729.log 2024-09-10T16:06:20.419 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54427.log.gz 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51025.log 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54553.log.gz 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41748.log 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55729.log.gz 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44216.log 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51025.log.gz 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40563.log 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41748.log.gz 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67149.log 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44216.log.gz 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52285.log 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40563.log.gz 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67149.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T16:06:20.420 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67149.log.gz 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65090.log 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52285.log.gz 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70467.log 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51991.log 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65090.log.gz 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65510.log 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70467.log.gz 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57913.log 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51991.log.gz 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48169.log 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65510.log.gz 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70719.log 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57913.log.gz 2024-09-10T16:06:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60452.log 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48169.log.gz 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68535.log 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70719.log.gz 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65762.log 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60452.log.gz 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64292.log 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68535.log.gz 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71853.log 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65762.log.gz 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58165.log 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64292.log.gz 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64964.log 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71853.log.gz 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51949.log 2024-09-10T16:06:20.422 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58165.log.gz 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68493.log 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64964.log.gz 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54217.log 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51949.log.gz 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70089.log 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68493.log.gz 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50017.log 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54217.log.gz 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55099.log 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70089.log.gz 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62652.log 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50017.log.gz 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58893.log 2024-09-10T16:06:20.423 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55099.log.gz 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37148.log 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62652.log.gz 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39332.log 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58893.log.gz 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67443.log 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37148.log.gz 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63661.log 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39332.log.gz 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52075.log 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67443.log.gz 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5/var/log/ceph/ceph-client.admin.63661.log: --verbose -- /var/log/ceph/ceph-client.admin.46264.log 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63661.log.gz 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51529.log 2024-09-10T16:06:20.424 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52075.log.gz 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62863.log 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46264.log.gz 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48883.log 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51529.log.gz 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39934.log 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62863.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62863.log.gz 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66645.log 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48883.log.gz 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37823.log 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39934.log.gz 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61979.log 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66645.log.gz 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40193.log 2024-09-10T16:06:20.425 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37823.log.gz 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68451.log 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61979.log.gz 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40193.log.gz 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71013.log 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40082.log 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68451.log.gz 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64502.log 0.0% 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71013.log.gz 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65552.log 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40082.log.gz 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54763.log 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64502.log.gz 2024-09-10T16:06:20.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51277.log 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65552.log.gz 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67863.log 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54763.log.gz 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70425.log 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51277.log.gz 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46634.log 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67863.log.gz 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64250.log 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70425.log.gz 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55855.log 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46634.log.gz 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65216.log 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64250.log.gz 2024-09-10T16:06:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39443.log 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55855.log.gz 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69039.log 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65216.log.gz 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38666.log 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39443.log.gz 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69039.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69039.log.gz 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38364.log 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38666.log.gz 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69207.log 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52495.log 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38364.log.gz 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69207.log.gzgzip 2024-09-10T16:06:20.428 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55057.log 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52495.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.56653.log 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52495.log.gz 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66687.log 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55057.log.gz 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69333.log 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56653.log.gz 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57451.log 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66687.log.gz 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45931.log 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69333.log.gz 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50689.log 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57451.log.gz 2024-09-10T16:06:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37694.log 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45931.log.gz 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60368.log 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50689.log.gz 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45783.log 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37694.log.gz 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60368.log.gz 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67275.log 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69543.log 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45783.log.gz 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67275.log.gz 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56191.log 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39675.log 2024-09-10T16:06:20.430 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69543.log.gz 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56191.log.gz 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68955.log 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56359.log 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39675.log.gz 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63787.log 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68955.log.gz 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48379.log 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56359.log.gz 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32439.log 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63787.log.gz 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr: 95.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48379.log.gz 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51907.log 2024-09-10T16:06:20.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.32439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62821.log 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32439.log.gz 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64039.log 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51907.log.gz 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44384.log 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62821.log.gz 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59313.log 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64039.log.gz 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45243.log 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44384.log.gz 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51655.log 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59313.log.gz 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43778.log 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45243.log.gz 2024-09-10T16:06:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51655.log.gz 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50941.log 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67527.log 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43778.log.gz 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47665.log 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50941.log.gz 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67527.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67527.log.gz 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44132.log 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47665.log.gz 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58419.log 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41915.log 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44132.log.gz 2024-09-10T16:06:20.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50101.log 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58419.log.gz 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65132.log 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41915.log.gz 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59817.log 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50101.log.gz 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70593.log 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65132.log.gz 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43865.log 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59817.log.gz 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59103.log 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70593.log.gz 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70635.log 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52201.log 2024-09-10T16:06:20.434 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59103.log.gz 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54385.log 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70635.log.gz 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51319.log 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52201.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52201.log.gz 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69249.log 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54385.log.gz 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62315.log 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51319.log.gz 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62021.log 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69249.log.gz 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38777.log 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62315.log.gz 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61208.log 2024-09-10T16:06:20.435 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62021.log.gz 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38257.log 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38777.log.gz 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61712.log 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61208.log.gz 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62947.log 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38257.log.gz 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58039.log 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61712.log.gz 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66729.log 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62947.log.gz 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71139.log 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58039.log.gz 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55183.log 2024-09-10T16:06:20.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66729.log.gz 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45709.log 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71139.log.gz 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.55183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54805.log 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55183.log.gz 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57829.log 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45709.log.gz 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64165.log 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54805.log.gz 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67611.log 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57829.log.gz 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49177.log 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64165.log.gz 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52159.log 2024-09-10T16:06:20.437 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67611.log.gz 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69837.log 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49177.log.gz 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59187.log 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52159.log.gz 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70845.log 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69837.log.gz 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59187.log.gz 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53167.log 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63451.log 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70845.log.gz 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46116.log 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53167.log.gz 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45392.log 2024-09-10T16:06:20.438 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63451.log.gz 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39480.log 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46116.log.gz 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58249.log 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45392.log.gz 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46042.log 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39480.log.gz 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.58249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67947.log 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58249.log.gz 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67485.log 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46042.log.gz 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67653.log 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67947.log.gz 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44846.log 2024-09-10T16:06:20.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67485.log.gz 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46597.log 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67653.log.gz 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49345.log 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44846.log.gz 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46412.log 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46597.log.gz 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49345.log.gz 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37355.log 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63829.log 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46412.log.gz 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46190.log 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37355.log.gz 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr:gzip/var/log/ceph/ceph-client.admin.63829.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43842.log 2024-09-10T16:06:20.440 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63829.log.gz 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49849.log 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46190.log.gz 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44804.log 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43842.log.gz 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5/var/log/ceph/ceph-client.admin.49849.log: --verbose -- /var/log/ceph/ceph-client.admin.44006.log 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49849.log.gz 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45308.log 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44804.log.gz 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45746.log 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44006.log.gz 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51823.log 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45308.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45308.log.gz 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54301.log 2024-09-10T16:06:20.441 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45746.log.gz 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51613.log 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51823.log.gz 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52831.log 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54301.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54301.log.gz 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53671.log 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51613.log.gz 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60242.log 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52831.log.gz 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64628.log 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53671.log.gz 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.60242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70551.log 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60242.log.gz 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40267.log 2024-09-10T16:06:20.442 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64628.log.gz 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64544.log 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70551.log.gz 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47833.log 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40267.log.gz 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51193.log 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64544.log.gz 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62484.log 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47833.log.gz 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51193.log: -- /var/log/ceph/ceph-client.admin.68619.log 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51193.log.gz 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62484.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62484.log.gz 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67065.log 2024-09-10T16:06:20.443 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68619.log.gz 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52663.log 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.67065.log: -- /var/log/ceph/ceph-client.admin.45182.log 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67065.log.gz 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59523.log 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52663.log.gz 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62989.log 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45182.log.gz 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37607.log 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59523.log.gz 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38814.log 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62989.log.gz 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37106.log 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37607.log.gz 2024-09-10T16:06:20.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44678.log 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38814.log.gz 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69123.log 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37106.log.gz 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61460.log 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44678.log.gz 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59229.log 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69123.log.gz 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43548.log 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61460.log.gz 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66771.log 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59229.log.gz 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44552.log 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43548.log.gz 2024-09-10T16:06:20.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59943.log 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66771.log.gz 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62189.log 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44552.log.gz 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50059.log 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59943.log.gz 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50185.log 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62189.log.gz 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44468.log 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50059.log.gz 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71769.log 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50185.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50185.log.gz 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67779.log 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44468.log.gz 2024-09-10T16:06:20.446 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48001.log 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71769.log.gz 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41916.log 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67779.log.gz 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54511.log 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48001.log.gz 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.41916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40600.log 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41916.log.gz 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45518.log 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54511.log.gz 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.40600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66477.log 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40600.log.gz 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45518.log.gz 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39258.log 2024-09-10T16:06:20.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46227.log 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66477.log.gz 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45056.log 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39258.log.gz 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59691.log 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46227.log.gz 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51571.log 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45056.log.gz 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38592.log 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59691.log.gz 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46671.log 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51571.log.gz 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48505.log 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38592.log.gz 2024-09-10T16:06:20.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45644.log 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46671.log.gz 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38089.log 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48505.log.gz 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.45644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38006.log 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45644.log.gz 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39184.log 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38089.log.gz 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38006.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38006.log.gz 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59271.log 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46375.log 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39184.log.gz 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67107.log 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59271.log.gz 2024-09-10T16:06:20.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49681.log 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46375.log.gz 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.67107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67107.log.gz 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49135.log 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49723.log 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49681.log.gz 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63325.log 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49135.log.gz 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61754.log 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49723.log.gz 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44930.log 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63325.log.gz 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46079.log 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61754.log.gz 2024-09-10T16:06:20.450 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69417.log 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.44930.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44930.log.gz 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54931.log 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46079.log.gz 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62400.log 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69417.log.gz 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48925.log 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54931.log.gz 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62400.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62400.log.gz 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51109.log 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65048.log 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.48925.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48925.log.gz 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56065.log 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51109.log.gz 2024-09-10T16:06:20.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54175.log 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65048.log.gz 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36725.log 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56065.log.gz 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.54175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54175.log.gz 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50143.log 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70383.log 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36725.log.gz 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61418.log 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50143.log.gz 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70383.log.gz 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68115.log 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56821.log 2024-09-10T16:06:20.452 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61418.log.gz 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64586.log 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68115.log.gz 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68409.log 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56821.log.gz 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61838.log 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64586.log.gz 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.68409.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68409.log.gz 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71517.log 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69753.log 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61838.log.gz 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66267.log 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71517.log.gz 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52789.log 2024-09-10T16:06:20.453 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69753.log.gz 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42022.log 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66267.log.gz 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36938.log 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.52789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52789.log.gz 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.42022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37865.log 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42022.log.gz 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71097.log 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36938.log.gz 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43358.log 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37865.log.gz 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39749.log 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71097.log.gz 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43358.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43358.log.gz 2024-09-10T16:06:20.454 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51361.log 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.39749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43464.log 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39749.log.gz 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59565.log 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51361.log.gz 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38740.log 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43464.log.gz 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59565.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.59565.log.gz 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56863.log 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43086.log 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38740.log.gz 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43964.log 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56863.log.gz 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46727.log 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43086.log.gz 2024-09-10T16:06:20.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70971.log 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43964.log.gz 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.46727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53755.log 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46727.log.gz 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71727.log 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70971.log.gz 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43128.log 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53755.log.gz 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38406.log 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71727.log.gz 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.43128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70215.log 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43128.log.gz 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53629.log 2024-09-10T16:06:20.456 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38406.log.gz 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70215.log.gz 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64922.log 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63871.log 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53629.log.gz 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.64922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49639.log 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64922.log.gz 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61586.log 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63871.log.gz 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57577.log 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49639.log.gz 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50857.log 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61586.log.gz 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49471.log 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57577.log.gz 2024-09-10T16:06:20.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66057.log 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50857.log.gz 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.49471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61937.log 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49471.log.gz 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71265.log 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66057.log.gz 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.61937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70005.log 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61937.log.gz 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71265.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.71265.log.gz --verbose 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/ceph-client.admin.53713.log 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.70005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53377.log 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70005.log.gz 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50521.log 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53713.log.gz 2024-09-10T16:06:20.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69291.log 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53377.log.gz 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.50521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38173.log 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50521.log.gz 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.69291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47791.log 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69291.log.gz 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.38173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71937.log 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38173.log.gz 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.47791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66099.log 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47791.log.gz 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.71937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37064.log 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71937.log.gz 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66099.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.53965.log -- replaced with /var/log/ceph/ceph-client.admin.66099.log.gz 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-09-10T16:06:20.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.37064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66855.log 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37064.log.gz 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.53965.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27642.log 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53965.log.gz 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63031.log 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66855.log.gz 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph.tmp-client.admin.27642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51067.log 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27642.log.gz 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.63031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62905.log 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63031.log.gz 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65846.log 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51067.log.gz 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.62905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51697.log 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62905.log.gz 2024-09-10T16:06:20.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.65846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65846.log.gz 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66351.log 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.51697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59607.log 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51697.log.gz 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.66351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57241.log 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66351.log.gz 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.59607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56233.log 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59607.log.gz 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.57241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36812.log 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57241.log.gz 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.56233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36726.log 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56233.log.gz 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36812.log.gz 2024-09-10T16:06:20.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.36726.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36726.log.gz 2024-09-10T16:06:20.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-10T16:06:21.201 INFO:teuthology.orchestra.run.smithi121.stderr: 96.8% -- replaced with /var/log/ceph/ceph-client.admin.43865.log.gz 2024-09-10T16:06:21.384 INFO:teuthology.orchestra.run.smithi121.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T16:06:21.748 INFO:teuthology.orchestra.run.smithi121.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T16:06:21.881 INFO:teuthology.orchestra.run.smithi121.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T16:06:24.677 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T16:10:35.867 INFO:teuthology.orchestra.run.smithi121.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T16:12:56.632 INFO:teuthology.orchestra.run.smithi121.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T16:14:08.231 INFO:teuthology.orchestra.run.smithi121.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T16:14:08.234 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-09-10T16:14:08.234 INFO:teuthology.orchestra.run.smithi121.stderr:real 7m48.180s 2024-09-10T16:14:08.234 INFO:teuthology.orchestra.run.smithi121.stderr:user 5m31.101s 2024-09-10T16:14:08.234 INFO:teuthology.orchestra.run.smithi121.stderr:sys 0m12.731s 2024-09-10T16:14:08.234 INFO:tasks.ceph:Archiving logs... 2024-09-10T16:14:08.235 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899253/remote/smithi121/log 2024-09-10T16:14:08.236 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T16:15:10.451 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T16:15:10.463 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T16:15:10.463 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T16:15:10.535 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T16:15:10.544 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T16:15:10.544 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T16:15:10.562 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-09-10T16:15:10.589 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T16:15:10.589 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-09-10T16:15:10.589 INFO:teuthology.orchestra.run.smithi121.stdout:^* hv01.front.sepia.ceph.com 3 6 377 29 +513us[ +512us] +/- 47ms 2024-09-10T16:15:10.589 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -122us[ -123us] +/- 55ms 2024-09-10T16:15:10.589 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -626us[ -627us] +/- 71ms 2024-09-10T16:15:10.589 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T16:15:10.589 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T16:15:10.598 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T16:15:10.599 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T16:15:10.629 DEBUG:teuthology.orchestra.run.smithi121:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T16:15:10.756 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T16:15:10.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T16:15:10.787 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T16:15:10.796 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T16:15:10.828 INFO:teuthology.task.internal:Duration was 1811.323638 seconds 2024-09-10T16:15:10.828 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T16:15:10.913 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T16:15:10.914 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T16:15:10.979 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T16:15:11.272 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T16:15:11.272 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2024-09-10T16:15:11.272 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T16:15:11.292 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T16:15:11.293 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T16:15:11.358 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T16:15:11.359 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T16:15:11.540 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T16:15:11.549 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T16:15:11.549 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T16:15:11.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T16:15:11.599 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T16:15:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2024-09-10T16:15:11.686 DEBUG:teuthology.orchestra.run.smithi121:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T16:15:11.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T16:15:11.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T16:15:11.751 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T16:15:11.751 DEBUG:teuthology.misc:Transferring archived files from smithi121:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-10_15:30:27-rados-wip-bharath7-testing-2024-09-10-1409-distro-default-smithi/7899253/remote/smithi121 2024-09-10T16:15:11.752 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T16:15:11.816 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T16:15:11.816 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T16:15:11.866 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T16:15:11.875 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T16:15:11.875 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T16:15:11.905 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T16:15:11.905 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T16:15:11.921 INFO:teuthology.orchestra.run.smithi121.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 16:15 /home/ubuntu/cephtest 2024-09-10T16:15:11.922 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T16:15:11.938 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T16:15:12.057 INFO:teuthology.run:Summary data: description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-low-osd-mem-target} rados supported-random-distro$/{centos_latest}} duration: 1811.3236377239227 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-10T16:15:12.057 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T16:15:12.141 INFO:teuthology.run:pass